CN119512434A

CN119512434A - Touch screen pointer control method and device

Info

Publication number: CN119512434A
Application number: CN202510083628.0A
Authority: CN
Inventors: 隰佳杰; 韩宝阳
Original assignee: Beijing Singularity Jump Technology Co ltd
Current assignee: Beijing Singularity Jump Technology Co ltd
Priority date: 2025-01-20
Filing date: 2025-01-20
Publication date: 2025-02-25

Abstract

The present invention discloses a control method and device for a touch screen pointer, the method comprising: obtaining in real time the pointer dragging operation data performed by a user on an air operation device; displaying a pointer moving in response to the pointer dragging operation on the touch screen; the pointer dragging operation data including the pointer dragging movement amount, pointer dragging direction and pointer dragging time interval of each frame in the dragging operation process recorded frame by frame; after the dragging is released, the pointer dragging movement amount is reversely displaced based on the pointer dragging time interval and pointer dragging direction for each frame of data in the order or reverse order of the frame records in the pointer dragging operation data. The present invention is used to improve the control efficiency of the touch screen and improve the user experience.

Description

Control method and device for touch screen pointer

Technical Field

The invention relates to the technical field of mobile terminal touch control, in particular to a control method and device of a touch screen pointer.

Background

This section is intended to provide a background or context to the presented embodiments of the invention. The description herein is not admitted to be prior art by inclusion in this section.

With the development of computer technology, the user interface interaction mode has undergone the evolution from the "pointer" system in the PC age to the "touch" system in the mobile internet age. The traditional computer operating system is based on pointer system design, uses a mouse as a main control carrier, and provides high-precision control experience. However, after entering the mobile internet era, a general mobile phone operating system is optimized for touch control, and is characterized in that interface elements (such as icons and fonts) occupy a large area in a screen, and most operations can be directly completed by fingers.

In recent years, more and more mobile screen devices (such as mobile phones, tablet computers, vehicle-mounted screens, education tablets, etc.) employ mobile phone operating systems and are in a scene where users cannot directly touch the screen. In these cases, remote control is typically required using an overhead operating device (e.g., touch pad, air mouse, etc.). The operation mode of such devices differs from conventional touch operations, particularly in terms of sliding and dragging operations, in that the user needs to frequently move the pointer from one position to another and then manually move the pointer back to the starting position, which significantly reduces the operation efficiency and affects the user experience.

The main problem currently existing is that in the process of operating the touch optimized interface through the pointer control system when the air operation device is used away from the touch screen, the user needs to frequently and manually move the pointer from the end point back to the start point. The concrete steps are as follows:

1. the operation efficiency is low, namely when the user needs to perform sliding or dragging operation from the point A to the point B for a plurality of times, the pointer needs to be manually moved back from the point B to the point A after each operation, so that the repetitive action of the user is increased, and the operation efficiency is reduced.

2. The user experience is poor, which makes the use experience of the air-operated device less intuitive and natural than direct touch, due to the additional manual operation required to return the pointer.

3. The lack of cross-platform compatibility is that existing solutions may not be suitable for all operating systems and devices, limiting the widespread use of control schemes for touch screens.

Disclosure of Invention

The embodiment of the invention provides a control method of a pointer of a touch screen, which is used for solving the problem of low efficiency of manually moving back the pointer, improving the control efficiency of the touch screen, improving the user experience and improving the cross-platform compatibility, and comprises the following steps:

The method comprises the steps of acquiring pointer drag operation data of a user on aerial operation equipment in real time, wherein the pointer drag operation data comprise pointer drag movement quantity, pointer drag direction and pointer drag time interval of each frame in the drag operation process recorded frame by frame, wherein the pointer drag operation data is used for displaying a pointer which moves in response to the pointer drag operation in the touch screen;

After the drag is released, the data of each frame is subjected to reverse displacement of the pointer drag movement amount based on the pointer drag time interval and the pointer drag direction in the sequence or reverse sequence of frame records in the pointer drag operation data.

The embodiment of the invention also provides a control device of the pointer of the touch screen, which is used for solving the problem of low efficiency of manually moving back the pointer, improving the control efficiency of the touch screen, improving the user experience and improving the cross-platform compatibility, and comprises:

The device comprises a pointer drag operation data acquisition module, a touch screen and a display module, wherein the pointer drag operation data acquisition module is used for acquiring pointer drag operation data of a user on an aerial operation device in real time, and the pointer drag operation data comprises pointer drag movement quantity, pointer drag direction and pointer drag time interval of each frame in the drag operation process recorded frame by frame;

and the backward displacement module is used for performing backward displacement of the pointer drag movement amount on the data of each frame based on the pointer drag time interval and the pointer drag direction in the sequence or the reverse sequence of frame records in the pointer drag operation data after drag release.

The embodiment of the invention also provides computer equipment, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor realizes the control method of the touch screen pointer when executing the computer program.

The embodiment of the invention also provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program realizes the control method of the touch screen pointer when being executed by a processor.

The embodiment of the invention also provides a computer program product, which comprises a computer program, and the computer program realizes the control method of the touch screen pointer when being executed by a processor.

According to the embodiment of the invention, pointer drag operation data of a user on an aerial operation device are obtained in real time, pointers which move in response to the pointer drag operation are displayed in the touch screen, the pointer drag operation data comprise pointer drag movement amounts, pointer drag directions and pointer drag time intervals of each frame in the drag operation process recorded frame by frame, and after drag release, reverse displacement of the pointer drag movement amounts is carried out on the data of each frame according to the pointer drag time intervals and the pointer drag directions in the sequence or reverse sequence recorded in the frame in the pointer drag operation data. According to the embodiment of the invention, the detailed data of each frame, including the pointer drag movement amount, the direction and the time interval, of the pointer drag operation is obtained in real time when the user performs the pointer drag operation on the air operation equipment, the operation track of the user can be completely and accurately recorded, when the user finishes the drag operation, the reverse displacement operation is performed based on the recorded data, the user does not need to manually return the pointer, so that the complicated manual return operation is avoided, the operation efficiency is directly improved, the reverse displacement of the pointer drag movement amount is performed in the sequence or reverse sequence recorded by the frames in the pointer drag operation data, the smooth experience similar to direct contact control is simulated, the satisfaction degree of the user for controlling the touch screen by using the air operation equipment in the non-touch screen scene is improved, the user experience is remarkably improved, the relative position change of the pointer in the operation process is focused, the absolute position information of the equipment is not relied, the unified pointer control logic is realized in different operation systems and various mobile screen equipment, the solution of cross-platform and high compatibility is provided, and the pointer control requirement of the diversified equipment in the non-touch operation scene is met.

Drawings

In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art. In the drawings:

FIG. 1 is a flow chart of a method for controlling a pointer of a touch screen according to an embodiment of the invention;

FIG. 2 is a diagram illustrating a method for controlling a pointer of a touch screen according to an embodiment of the present invention;

FIG. 3 is a diagram illustrating a method for controlling a pointer of a touch screen according to an embodiment of the present invention;

FIG. 4 is a diagram illustrating a method for controlling a pointer of a touch screen according to an embodiment of the present invention;

FIG. 5 is a diagram illustrating a method for controlling a pointer of a touch screen according to an embodiment of the present invention;

FIG. 6 is a diagram illustrating a method for controlling a pointer of a touch screen according to an embodiment of the present invention;

FIG. 7 is a diagram illustrating a sliding recording stage in a method for controlling a pointer of a touch screen according to an embodiment of the present invention;

FIG. 8 is a diagram illustrating a specific example of a data processing stage and a rebound execution stage in a method for controlling a pointer of a touch screen according to an embodiment of the present invention;

FIG. 9 is a schematic diagram of a control device for a touch screen pointer according to an embodiment of the present invention;

FIG. 10 is a schematic diagram of a computer device for controlling a pointer of a touch screen according to an embodiment of the present invention.

Detailed Description

For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the embodiments of the present invention will be described in further detail with reference to the accompanying drawings. The exemplary embodiments of the present invention and their descriptions herein are for the purpose of explaining the present invention, but are not to be construed as limiting the invention.

The term "and/or" is used herein to describe only one relationship, and means that three relationships may exist, for example, A and/or B, and that three cases exist, A alone, A and B together, and B alone. In addition, the term "at least one" herein means any one of a plurality or any combination of at least two of a plurality, for example, including at least one of A, B, C, may mean including any one or more elements selected from the group consisting of A, B and C.

In the description of the present specification, the terms "comprising," "including," "having," "containing," and the like are open-ended terms, meaning including, but not limited to. The description of the reference terms "one embodiment," "a particular embodiment," "some embodiments," "for example," etc., means that a particular feature, structure, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the application. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiments or examples. Furthermore, the particular features, structures, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. The order of steps involved in the embodiments is illustrative of the practice of the application, and is not limited and may be suitably modified as desired.

The data acquisition, storage, use, processing and the like in the technical scheme of the application all meet the relevant regulations of the relevant laws and regulations. The information collected in the application is information and data which are authorized by a user or are fully authorized by all parties, and the processing such as collection, storage, use, processing, transmission, provision, disclosure, application and the like of related data is in compliance with related laws and regulations and standards, necessary security measures are taken without violating the public welfare, and corresponding operation entrance is provided for the user to select authorization or rejection. In addition, the application provides a corresponding operation entrance for the user to choose to agree or reject the automatic decision result, and if the user chooses to reject, the expert decision flow can be entered.

It should be noted that, in the embodiments of the present application, some existing solutions in the industry such as software, components, models, etc. may be mentioned, and, if some existing software tools, components, algorithm models, or solutions widely known in the technical field may be cited, they should be considered as exemplary, only for illustrating the feasibility of implementing the technical solution of the present application, these references should be understood as typical examples, and their core aims are to illustrate and verify the rationality and feasibility of implementing the technical solution proposed by the present application. But does not imply that the applicant has or must not have used this solution, such citations do not imply that the applicant has in fact adopted these existing solutions or that it must in the future adopt these methods during their technical implementation, in other words that these references serve only for illustrative purposes, help to understand the relevance and surpassing of the innovative point of the present application with respect to the prior art, and do not constitute an admission or dependency statement regarding specific prior art products.

In the development history of computer technology, the user interface interaction mode has undergone the evolution from the "pointer" system in the PC age to the "touch" system in the mobile internet age. The operating system of the PC end is based on the design of a pointer system, uses a mouse as a main control carrier and realizes high-precision control. The mobile phone operation system in the mobile internet era aims at touch control optimization, and is characterized in that interface elements (such as icons and fonts) occupy a large area in a screen, and most operations can be directly completed through fingers.

However, with the development of technology, mobile screen devices (such as a mobile phone, a tablet, a vehicle-mounted screen, an education tablet, etc.) increasingly adopting a mobile phone operating system are appeared in a scene where a user cannot directly touch the screen. In this case, an air operation device (such as a touch pad, a mouse, etc.) is required for remote control. And the system is transversely sliding and longitudinally sliding, which are high-frequency operation of users. In these scenarios, the existing pointer system has a significant disadvantage that when the user needs to perform a sliding or dragging operation from point a to point B multiple times, the pointer needs to be manually moved from point B back to point a after each operation, which greatly reduces the operation efficiency.

In order to solve the above problems, an embodiment of the present invention provides a method for controlling a pointer of a touch screen, which is used for solving the problem of low efficiency of manually moving back the pointer, improving control efficiency of the touch screen, improving user experience, and improving cross-platform compatibility, and referring to fig. 1, fig. 1 is a flow diagram of a method for controlling a pointer of a touch screen according to an embodiment of the present invention, and the method may include:

step 101, acquiring pointer drag operation data of a user on an aerial operation device in real time, wherein a pointer which moves in response to the pointer drag operation is displayed in a touch screen, and the pointer drag operation data comprises pointer drag movement quantity, pointer drag direction and pointer drag time interval of each frame in the drag operation process recorded frame by frame;

and 102, after the drag is released, performing reverse displacement of the pointer drag movement amount on the data of each frame based on the pointer drag time interval and the pointer drag direction in the sequence or reverse sequence recorded by the frames in the pointer drag operation data.

According to the embodiment of the invention, pointer drag operation data of a user on an aerial operation device are acquired in real time, pointers which move in response to the pointer drag operation are displayed in the touch screen, the pointer drag operation data comprise pointer drag movement amounts, pointer drag directions and pointer drag time intervals of each frame in the drag operation process recorded frame by frame, and reverse displacement of the pointer drag movement amounts is carried out on the data of each frame based on the pointer drag time intervals and the pointer drag directions in the sequence or reverse sequence recorded in the pointer drag operation data. According to the embodiment of the invention, detailed data of each frame, including pointer dragging movement amount, direction and time interval, of a user when the user drags the pointer on the aerial operation device are obtained in real time, operation tracks of the user can be completely and accurately recorded, when the user finishes the dragging operation, backward displacement operation is performed based on the recorded data, the user does not need to manually return the pointer, so that complicated manual return operation is avoided, the operation efficiency is directly improved, backward displacement of the pointer dragging movement amount is performed in the sequence or reverse sequence recorded by the frames in the pointer dragging operation data, smooth experience similar to direct contact control is simulated, satisfaction degree of the user for controlling a touch screen by using the aerial operation device in a non-touch screen scene is improved, user experience is remarkably improved, and unified pointer control logic can be realized in different operation systems and various types of mobile screen devices by focusing on relative position change of the pointer without depending on absolute position information of the device, and therefore, the automatic rebound control of the pointer is realized according to a set rebound mode association relation, the requirements of the pointer under the non-touch operation scene of diversified devices are met.

In the specific implementation, step 101 is firstly performed, namely pointer drag operation data of a user on an aerial operation device are obtained in real time, pointers which move in response to the pointer drag operation are displayed in the touch screen, and the pointer drag operation data comprise pointer drag movement amounts, pointer drag directions and pointer drag time intervals of each frame in the drag operation process recorded frame by frame.

In an embodiment, an interaction signal between the air operation device and the touch screen is continuously monitored. When detecting that a user operates the air operation device to generate a signal conforming to the initial characteristic of the pointer drag operation (such as the condition that a finger on a touch pad is pressed and starts to move, or a specific key on an air mouse is pressed and moves along with the pressing, etc.), immediately starting a data acquisition flow. Meanwhile, the related data recording module is initialized, so that preparation is made for recording the subsequent operation data frame by frame.

And precisely acquiring current time information and marking the current time information as a time stamp at each time node in the operation process. The time stamp is used for calculating the time interval in the process of pointer dragging operation, the recording precision of the time stamp is required to be accurate to the extent that the time variation of pointer operation can be accurately reflected, for example, the time stamp is accurate to the millisecond level, and therefore a powerful basis is provided for accurately analyzing the pointer operation rhythm.

And synchronously acquiring the corresponding position information of the pointer on the touch screen. The movement amount of the pointer in different directions is calculated by comparing the changes of the pointer positions of the front frame and the rear frame. Assuming that the current frame pointer position coordinates are one set of values and the previous frame pointer position coordinates are the other set of values, the movement amount of the pointer in the horizontal direction and the vertical direction can be obtained by subtracting the coordinate values of the current frame pointer position coordinates and the previous frame pointer position coordinates in the horizontal direction and the vertical direction. The drag direction of the pointer can be determined based on the positive and negative of these movement amounts. If the horizontal movement is positive, the pointer moves right in the horizontal direction, and if the horizontal movement is negative, the pointer moves left in the horizontal direction. Similarly, the positive and negative of the vertical movement amount can also clarify the movement direction of the pointer in the vertical direction.

The pointer movement amount (including horizontal and vertical directions), the pointer drag direction (determined according to the positive and negative movement amounts) and the corresponding time stamp acquired at each time point are integrated into a complete set of pointer drag operation data. And then storing the data in a cache area designated by the system or a special data storage structure according to the time sequence, and reserving complete original data for subsequent data processing and analysis.

And continuously focusing on the pointer operation state, and immediately stopping the data acquisition operation when detecting that a user operates the air operation equipment to generate a signal conforming to the ending characteristic of the pointer dragging operation (such as the condition that a finger on a touch control plate is lifted or a specific key on an air mouse is released). At this time, frame-by-frame data of the whole process from the beginning to the end of the pointer drag operation of the user is completely recorded.

The pointer display in the touch screen is closely related to the operation of the air operation device. When the air operation device sends out a pointer movement instruction, the touch screen updates the position of the pointer on the screen according to the relevant parameters in the instruction after receiving the instruction signal. The movement of the pointer on the screen is achieved by adjusting the coordinate values in the screen coordinate system. For example, in a two-dimensional plane touch screen coordinate system, the initial position of the pointer has a specific coordinate value, when receiving an instruction that there is a movement in a horizontal direction and there is no movement in a vertical direction, the coordinate value of the pointer in the horizontal direction is updated correspondingly according to the movement, and the coordinate value in the vertical direction is kept unchanged, so as to realize the horizontal movement of the pointer on the screen.

The display of the pointer on the touch screen provides intuitive operational feedback to the user. The user can clearly know the operation effect of the user on the air operation equipment by observing the position change of the pointer on the screen. The display attributes such as the shape and the color of the pointer can be adjusted according to system settings so as to adapt to different screen backgrounds and operation scenes, and the pointer is ensured to have good visibility under various conditions. For example, when the vehicle-mounted screen is used in a bright outdoor environment, the pointer can adopt thicker lines and vivid colors so that a driver can easily see the position of the pointer, and when the education flat board is used indoors, the pointer pattern can be customized according to teaching contents and interface styles so as to coordinate with the overall visual effect, and meanwhile, the observation and operation perception of the pointer movement by a user are not influenced. The close association between the pointer display and the user operation and a good visual feedback mechanism ensure that the user can accurately and efficiently control the pointer of the touch screen through the air operation equipment, and smooth interaction operation with the content of the touch screen is realized.

In one embodiment, fig. 2 is a specific example diagram of a control method of a touch screen pointer in an embodiment of the present invention, as shown in fig. 2, the method for acquiring pointer drag operation data performed by a user on an air operation device in real time includes:

Step 201, when detecting that a user starts pointer drag operation on an aerial operation device, generating a starting time stamp;

Step 202, recording pointer drag movement amount and pointer drag time interval in the drag operation process frame by frame until the user is detected to stop carrying out the pointer drag operation on the aerial operation device, and simultaneously adding a corresponding time stamp for each frame of record data.

In an embodiment, monitoring of the interaction condition of the air operation device and the touch screen is always maintained. Once the signal sent by the aerial operation device is detected to be matched with the characteristic that a user starts to carry out pointer drag operation, for example, the touch pad detects that the finger is pressed in a specific mode and generates a movement trend, or a specific key on the aerial mouse is pressed and detects a movement signal of the finger, the time acquisition operation is immediately triggered, the time information of the current moment is accurately acquired, and the time information is defined as a starting time stamp. The start time stamp is used as a time reference point of the whole pointer drag operation data record, and provides a key reference start value for the subsequent calculation of the pointer drag related time interval.

The state of the pointer is sampled and recorded at regular time intervals (e.g., one frame every several milliseconds) from the moment the start of the pointer drag operation is detected. And at each sampling moment, acquiring corresponding position information of the pointer on the touch screen, and calculating the movement amount of the pointer relative to the position of the previous frame. And simultaneously, recording the time stamp of the current sampling time, comparing the time stamp with the time stamp recorded in the previous frame, and calculating the time interval between the time stamp and the time stamp, wherein the time interval is the pointer dragging time interval.

Such a frame-by-frame recording operation is continuously performed, and the movement amount of the pointer and the time interval data are continuously updated until it is detected that the user stops the pointer drag operation. This process ensures a complete record of the entire pointer drag operation trajectory, providing detailed raw data for subsequent data processing and analysis.

The corresponding time stamp is added to the frame data while the moving amount of the pointer of one frame and the time interval data are recorded. This timestamp is used not only to calculate the pointer drag time interval, but also as a time stamp for the entire data record, so that each set of data can be accurately located on the time axis. Integrating the pointer drag movement amount, the pointer drag time interval and the corresponding time stamp into a complete data unit, and sequentially storing the complete data unit in a pre-initialized data structure according to time sequence.

In one embodiment, fig. 3 is a specific example diagram of a control method of a touch screen pointer in an embodiment of the present invention, as shown in fig. 3, recording a pointer drag movement amount and a pointer drag direction in a frame-by-frame drag operation process, including:

Step 301, recording relative coordinate information corresponding to a pointer in the process of dragging operation frame by frame;

And 302, respectively taking the change value and the positive and negative change value of the relative coordinate information of each frame and the relative coordinate information of the frame next to the frame as the pointer drag moving amount and the pointer drag direction corresponding to the frame.

In one embodiment, fig. 4 is a specific example diagram of a control method of a touch screen pointer in an embodiment of the present invention, as shown in fig. 4, in which a pointer drag time interval during a drag operation is recorded frame by frame, including:

step 401, recording a time stamp corresponding to a pointer in the process of dragging operation frame by frame;

Step 402, using the time interval between the time stamp of each frame and the time stamp of the frame next to the frame as the pointer drag time interval corresponding to the frame.

In the embodiment, in the process of recording the pointer drag operation frame by frame, at each sampling time, the relative coordinate information of the pointer in the touch screen coordinate system is obtained through a special position detection means. The relative coordinate information is based on a specific fixed reference point (such as the top left corner vertex of the screen) or the pointer position of the previous frame on the screen, and the coordinate offset of the pointer relative to the reference in the horizontal and vertical directions is calculated. For each sampling frame, the coordinate offsets of the pointer in the horizontal and vertical directions are carefully recorded, thereby forming complete relative coordinate information of the frame.

In order to ensure the consistency and accuracy of the relative coordinate information record in the whole pointer dragging operation process, a specific sampling strategy and a data processing flow are adopted. Even if the pointer moves at a relatively high speed or has small jitter, the sampling strategy can ensure that the pointer position is sampled at a proper frequency, and each frame can acquire accurate relative coordinate information. The data processing flow is responsible for sorting and storing the sampled data, and preventing data loss or error recording. By the mode, the pointer position at each time point can be accurately captured, and the reliability of relative coordinate information recording is ensured.

For the relative coordinate information recorded for each frame, the difference between the relative coordinates of the current frame and the relative coordinates of the following frame in the horizontal direction and the vertical direction is calculated, so that the movement amount of the pointer in the two directions is determined. Assuming that the relative coordinates of the pointers of the current frame are one group of values, and the relative coordinates of the pointers of the next frame are the other group of values, the moving amounts in the horizontal direction and the vertical direction are obtained through corresponding calculation. These shift values can accurately reflect the magnitude of the position change of the pointer between two adjacent frames.

And determining the dragging direction of the pointer according to the positive and negative values of the calculated moving quantity. If the horizontal movement is positive, the pointer moves right in the horizontal direction, and if the horizontal movement is negative, the pointer moves left in the horizontal direction. Similarly, for the amount of movement in the vertical direction, a positive value indicates that the pointer is moved downward in the vertical direction, and a negative value indicates that it is moved upward. The method for judging the direction through the positive and negative values of the movement amount is simple and effective, and can accurately reflect the movement direction of the pointer on the two-dimensional plane.

When the pointer drag operation data is recorded frame by frame, the current time information is acquired at each sampling time by using a timing device and marked as a time stamp. The timing device has certain precision, can reach millisecond level or higher, ensure the accurate record of pointer operation time.

And continuously recording each frame of time stamp along with the continuous operation of pointer drag to form a time stamp sequence arranged in time sequence. The sequence completely records the time track of the pointer in the whole operation process, and each time stamp is associated with corresponding pointer position information, movement amount and other data to jointly form a complete data record of pointer dragging operation.

Taking a power dispatching center as an example for controlling a large-screen power system monitoring pointer, the recorded time stamp sequence can clearly show the operation time sequence of a dispatcher when monitoring the power running condition and adjusting the power parameters. Each point in time from when attention to a regional power load is started to when an adjustment decision is made is accurately recorded, which helps review the operational process.

For a recorded sequence of time stamps, the difference between adjacent time stamps is calculated to determine the pointer drag time interval. And setting the current frame time stamp as a specific time value, and setting the next frame time stamp as another time value, wherein the difference value of the current frame time stamp and the next frame time stamp is the pointer drag time interval. This time interval reflects the time span of the pointer between two adjacent frames, i.e. the time cadence during pointer movement.

The calculated pointer drag time interval data has many applications in subsequent data processing and analysis. For example, when determining whether a pause or a slow motion situation exists in a user operation, the time interval data is an important criterion. If the time interval is long and the pointer movement amount is small, it may indicate that the user has a short stay or operation hesitation at a certain position.

In the specific implementation, after the pointer drag operation data of the user on the aerial operation device is obtained in real time in step 101, step 102 is performed, after drag release, reverse displacement of the pointer drag movement amount is performed on the data of each frame based on the pointer drag time interval and the pointer drag direction in the sequence or reverse sequence recorded in the frame in the pointer drag operation data.

In one embodiment, further comprising:

Determining whether the frame record data is a target frame or not according to each frame record data, wherein the pointer drag time interval corresponding to the target frame is larger than a first threshold value, modifying the pointer drag time interval of the target frame into a preset time interval, and replacing the target frame in the pointer drag operation data by the modified target frame to obtain modified pointer drag operation data;

Performing reverse displacement of the pointer drag movement amount for the data of each frame based on the pointer drag time interval and the pointer drag direction in the order of frame recording or reverse order in the pointer drag operation data, including:

And performing reverse displacement of the pointer drag movement amount on the data of each frame based on the pointer drag time interval and the pointer drag direction in the sequence or reverse sequence of frame records in the corrected pointer drag operation data.

In an embodiment, after the complete pointer drag operation data is obtained, a detailed analysis and judgment are performed on each frame record data to determine whether the frame record data belongs to the target frame. The condition for determining a frame as the target frame is that the corresponding pointer drag time interval is greater than the first threshold. For example, when the channel switching operation is performed by the air operation device (such as the remote controller controlling the smart television screen pointer), each time the pointer moves is recorded frame by frame, and the pointer corresponding to some frames stays at a certain position for a longer time, that is, the pointer drag time interval is relatively longer, and when the time interval exceeds the preset first threshold, the frame meets the target frame judging condition. The first threshold is a time standard preset according to the actual operation scene and consideration of the pointer operation characteristics, and is intended to screen out the operation frames with specific time characteristics.

Once it is determined that a frame is the target frame, its pointer drag interval is then modified to a preset interval. The preset time interval is also a fixed time value set in advance according to the overall requirement of the system and the operation fluency. After the pointer drag time interval of the target frames is modified, the modified target frames are used for replacing the corresponding target frames in the original pointer drag operation data, and the modified pointer drag operation data can be obtained through the replacing operation. In the whole data record of the whole pointer drag operation, the data which is originally in a time interval not conforming to the conventional process or can influence the subsequent process is optimized through the processing, and the corrected data can reflect the actual condition of the pointer operation more accurately.

When reverse displacement of the pointer drag movement amount is performed in the order of frame recording in the corrected pointer drag operation data, the operation is advanced stepwise backward from the first frame data. First, for the first frame data, the pointer drag time interval and the pointer drag direction information are acquired, and the direction of the reverse displacement is clarified according to the pointer drag direction. For example, the pointer drag direction is moved rightward in the horizontal direction, and then the reverse displacement direction is moved leftward in the horizontal direction. And setting corresponding waiting time according to the pointer dragging time interval recorded by the frame, and moving the pointer by corresponding pointer dragging movement amount according to the determined reverse displacement direction after the waiting time is ended. Then, each subsequent frame data of the second frame, the third frame, etc. is sequentially processed in the same steps. For example, in a scenario that a smart watch is used to control a mobile phone screen pointer to browse a picture set, if reverse displacement is performed in sequence, the pointer sequentially moves back from the last browsed picture position according to the previous browsed sequence, each frame precisely moves a corresponding distance according to a corresponding time interval and a reverse direction, so that a gradual backtracking effect is achieved, a process that the pointer reversely returns according to an operation sequence is simulated, an operation path is restored, and user requirements can be better met in some application scenarios with requirements on an operation track.

If the reverse displacement of the pointer drag movement amount is selected to be performed in the reverse order of the frame records in the corrected pointer drag operation data, that operation is sequentially processed from the last frame data onward. The key information such as the pointer drag time interval, the pointer drag direction and the like of the last frame is firstly obtained, and the reverse displacement direction is also determined according to the pointer drag direction, for example, the pointer drag direction moves downwards in the vertical direction, and the reverse displacement direction moves upwards in the vertical direction. And setting waiting time according to the pointer drag time interval of the last frame, and enabling the pointer to move by a corresponding pointer drag movement amount according to the reverse displacement direction as soon as the waiting time is reached. After the last frame is processed, the next to last frame is processed, and the steps of acquiring information, determining direction, waiting time and moving pointer are repeated, so that the process is circulated until the first frame data. For example, in a scene that a screen pointer is controlled to quickly browse long list contents (such as a news list) through a blank gesture (an aerial operation device) of a mobile phone, the pointer can quickly fall back from the end position to the beginning position of the list in a reverse sequence and reverse displacement mode, an intermediate operation process is skipped, the pointer returns to an initial state more efficiently, and in a scene that the operation efficiency is high and the specific operation path restoration is not concerned, the mode can reduce the waiting time of a user and improve the convenience and fluency of operation.

The reverse displacement operation of the pointer dragging movement amount is carried out based on the sequence or the reverse sequence, so that the pointer can reasonably return after the one-time dragging operation is completed, the operation of the whole touch screen pointer is more in line with the operation expectation of a user, the experience of the user when the user controls the touch screen pointer by using the air operation equipment is enhanced, and the whole pointer operation process is ensured to be in a better state logically and practically.

In the process of determining target frames and modifying the pointer dragging time interval and replacing data to obtain corrected pointer dragging operation data, for those target frames with small pointer movement amount and long time interval, the problem that the follow-up rebound calculation is too complicated and time-consuming due to long-time pause is avoided by modifying the time interval into a preset time interval, the speed and the operation efficiency of the control of the pointer of the touch screen are accelerated, the pointer can more timely carry out rebound operation, the waiting time of the user is reduced, the reverse displacement of the pointer dragging movement amount is carried out in the sequence of frame record in the corrected pointer dragging operation data or in the reverse sequence, the smooth experience similar to direct contact control is simulated, the satisfaction degree of the user for controlling the touch screen by using the air operation equipment under the non-touch screen scene is improved, and the user experience is remarkably improved.

In one embodiment, for each frame record data, determining whether the frame record data is a target frame includes:

determining whether a pointer drag time interval corresponding to the frame record data is larger than a first threshold value;

if yes, determining whether the pointer drag movement amount corresponding to the frame record data is smaller than or equal to a second threshold value;

If yes, determining the frame record data as a target frame.

In the embodiment, in a specific embodiment, for each acquired frame record data, whether it is a target frame is determined, and first, a judgment operation based on a pointer drag time interval needs to be performed.

And checking key information, namely a pointer dragging time interval, recorded in each frame of recorded data, and comparing the key information with a preset first threshold value. The recorded pointer drag time interval represents the length of time that the pointer spends in each operational phase in the process of staying or moving. If the pointer drag time interval in the record data of a certain frame is longer, for example, exceeds a set first threshold, wherein the first threshold is a time standard set according to the consideration of a longer pause time or a special time node in daily operation, when the first condition of the target frame is met, further subsequent judgment is needed.

When the pointer drag time interval corresponding to a certain frame of record data is larger than a first threshold value, whether the pointer drag moving amount corresponding to the frame of record data is smaller than or equal to a second threshold value is determined.

In this frame record data, the pointer drag movement amount reflects the magnitude of the position change of the pointer in the horizontal and vertical directions at the corresponding operation stage. If the pointer stays longer at a certain time (the condition that the time interval is larger than the first threshold value is satisfied) but the position of the pointer on the screen is also changed greatly (namely, the dragging movement amount of the pointer exceeds the second threshold value), the requirement of the target frame is not met. Only when the pointer is in a relatively small range, that is, less than or equal to the second threshold value, based on the time interval condition, the second threshold value is set according to the actual operation scene and the definition of the tiny movement condition of the pointer, and only if the frame record data is determined to accord with the characteristics of the target frame, the frame record data is further determined to be the target frame.

Through the mode of screening from the pointer dragging time interval and then carrying out secondary judgment by combining the pointer dragging movement amount, target frames which really meet specific requirements and have special operation characteristics can be accurately found out from a plurality of frame record data, a foundation is laid for follow-up data processing of the target frames and optimization of pointer operation of the whole touch screen, and the whole pointer control process is more suitable for various requirements in actual application scenes.

In the process of acquiring pointer drag operation data of a user on aerial operation equipment, determining whether the data recorded by the current frame is a target frame or not in real time;

Or after detecting that the user stops carrying out pointer drag operation on the aerial operation device, determining whether the frame record data is a target frame or not according to each frame record data.

In the above-described embodiment, in the process of acquiring pointer drag operation data performed by a user on an air operation device, the judgment operation of the target frame is performed immediately after each recording of one frame of data. And determining whether the current frame is a target frame by comparing the pointer drag movement amount of the frame with a preset first threshold value and comparing the pointer drag time interval of the frame with a preset second threshold value. And if the pointer drag movement amount is smaller than or equal to the first threshold value and the pointer drag time interval is larger than the second threshold value, judging that the frame is a target frame. For example, in a scenario where a mobile device drops a screen to a smart tv and controls a screen dropping pointer to browse a web page through a remote controller (air operation device), if a first threshold is set to a smaller moving distance value, a second threshold is set to a relatively longer time value, when a user slowly moves the pointer on the remote controller and the dwell time is longer, the recorded frame data may satisfy a target frame condition, and is determined as a target frame in real time during the recording process. The real-time judging mode can timely find out possible special frames and provide instant information for subsequent data processing.

Another timing for determining the target frame is after detecting that the user stops the pointer drag operation on the air operation device. At this time, the pointer drag operation data of each frame which has been completely recorded is sequentially judged. The pointer drag movement amount of each frame is compared with a first threshold value one by one, and the pointer drag time interval is compared with a second threshold value. Taking an example of drawing a graph on an electronic whiteboard by an electronic pen (aerial operation device) and completing drawing operation, target frame judgment is started for each frame of data recorded in the whole drawing process. If the pointer movement amount of a certain frame data is extremely small and the dwell time is long (the target frame condition is satisfied), it is determined as a target frame at this stage. The post judgment mode can more comprehensively analyze and determine the target frame from the whole angle after acquiring all operation data, avoid the misjudgment condition possibly caused by real-time judgment and ensure the accuracy of the judgment of the target frame.

In one embodiment, further comprising:

combining pointer dragging movement amounts in the recorded data of the continuous target frames to obtain combined frames;

Modifying the pointer drag time interval of the target frame to a preset time interval includes:

modifying the pointer drag time interval of the combined frame to a preset time interval;

replacing the target frame in the pointer drag operation data with the modified target frame to obtain modified pointer drag operation data, including:

and replacing the corresponding continuous target frames in the pointer drag operation data with the combined frames to obtain corrected pointer drag operation data.

In an embodiment, after the merged frame is obtained, a modification operation is required for its pointer drag time interval. Specifically, the pointer drag time interval of the merged frame is modified to a preset time interval. The preset time interval is a fixed time value preset according to various factors in the design process of the whole touch screen pointer control method. For example, in a car navigation system, when a driver performs operations such as destination searching or route adjustment by operating a navigation screen pointer through a car controller, if there are consecutive target frames (such as a plurality of short pauses and small movements of the pointer around a specific search result) that are combined into a combined frame through the previous steps, the respective pointer drag time intervals of the consecutive target frames may be irregular and complex. The pointer drag time interval of the combined frame is modified to be a preset time interval, so that the data can be more regular and unified, the follow-up analysis operation rhythm based on the data is facilitated, unified time reference is provided for related operations such as pointer rebound and the like, unnecessary trouble caused to follow-up data processing and system function realization due to the difference and complexity of the original time interval is avoided, and the whole data processing flow is smoother and efficient.

After the pointer drag time interval of the combined frame is modified, the data replacing operation is carried out next, that is, the modified combined frame is used for replacing the corresponding continuous target frame in the pointer drag operation data, so that the modified pointer drag operation data is obtained.

In the original pointer drag operation data, those successive target frames each record the operation of the pointer at different small phases, although there is a correlation, the data is relatively scattered and complex. And after the previous merging and time interval modification, replacing the merging frames by the merging frames is equivalent to integrating and optimizing the operation information carried by the scattered and internally associated target frames. If there are continuous target frames recording multiple small-amplitude movements and pauses of the pointer near a certain channel classification area, after the replacement of the merging frames, the corrected data can be more concise and clear when reflecting the pointer operation at this stage. The method can more accurately embody the integral operation characteristics of the pointer in the area, removes redundant information possibly caused by multi-frame data dispersion, enables the follow-up operations such as analysis of the operation track of the pointer or optimization of system response according to operation data to be carried out based on more accurate and effective data, further improves the smoothness and rationality of the operation process of the pointer of the whole touch screen, is more suitable for the operation expectation of a user in the actual use process, and enhances the operation experience of the user.

In one embodiment, fig. 5 is a specific schematic diagram of a control method of a touch screen pointer in an embodiment of the present invention, as shown in fig. 5, combining pointer dragging movement amounts in recorded data of consecutive target frames to obtain a combined frame, including:

step 501, according to the sequence from the first to the last of the time stamp of each frame record data, the target frame with the prior time stamp is obtained as the frame to be combined;

Step 502, determining whether the frame following the frame to be combined is a target frame;

step 503, if yes, combining the frame to be combined and the pointer dragging movement amount of the next frame to obtain intermediate data;

and step 504, replacing the frame to be combined with the intermediate data, repeating the steps of determining whether the next frame of the intermediate data is a target frame or not and combining until the next frame of the intermediate data is not the target frame or the intermediate data is larger than a third threshold value, and taking the obtained intermediate data as a combined frame.

In the embodiment, a large amount of frame record data is generated during the whole touch screen pointer operation, and for those data frames which meet specific conditions and are determined as target frames, a merging operation is required to be performed according to a certain rule to obtain a merged frame. First, the sorting and checking are performed in the order from the first to the last according to the time stamp of the data recorded for each frame. For example, in a scenario where a remote controller operates a screen pointer of an intelligent television to switch channels and browse related program information, each time the pointer moves, pauses, and other operations correspond to generating frame record data with a timestamp. From these data, the target frame with the earliest time stamp is found and determined as the frame to be merged. The frame to be combined becomes the basis for the start of the subsequent combining operation, and carries the operation characteristics of the pointer at a specific initial stage, such as the initial position of the pointer near a certain channel icon, the tiny movement condition and the like, so as to provide initial data reference for the subsequent combining operation with other continuous target frames.

After the frame to be merged is determined, it is checked whether the frame subsequent to the frame to be merged is also the target frame. This step is critical because it is only if the latter frame is also the target frame that the precondition for further merging the pointer drag movement amounts is provided. If the frame to be combined corresponds to a stage that the pointer stays briefly in a certain program detail page and the movement amount is small, then when looking at the next frame, it needs to be judged whether the pointer operation condition recorded by the frame meets the judgment standard of the target frame, that is, whether the related factors such as the pointer drag time interval and the pointer drag movement amount are in the preset range meeting the target frame. Only when the following frame also meets the target frame condition, the next merging operation can be carried out, the merging rationality and the data relevance are ensured, and the error merging of irrelevant frame data is avoided, so that the follow-up accurate analysis and processing of the whole condition of pointer operation are influenced.

Once it is determined that the frame subsequent to the frame to be merged is also the target frame, the merging of the pointer drag movement amounts of the two frames can be started to obtain intermediate data. For example, the pointer drag movement amount of the frame to be combined in the horizontal direction is a certain value, the corresponding movement amount is also provided in the vertical direction, and the corresponding movement amount values are provided in the horizontal and vertical directions of the next frame. At this time, the movement amounts of the two frames are added in the horizontal direction to obtain the combined horizontal direction pointer drag movement amount, and the same is true in the vertical direction, so that by such a calculation operation, a new intermediate data including the combined pointer drag movement amount is formed. The intermediate data integrates the condition of the two frames in the aspect of pointer position change, can more comprehensively embody the operation characteristics of the pointer in the continuous small stage, lays a foundation for the subsequent continuous merging operation and the formation of the final merging frame, and ensures that the integration of the continuous target frames is more consistent with the continuity and the integrity of the actual operation of the pointer.

After the intermediate data is obtained, the frame to be combined is replaced by the intermediate data, so that the frame to be combined is updated to the state represented by the intermediate data. Then, the previous step of determining whether the subsequent frame of the intermediate data is the target frame and the merging is repeated again. If the frame to be combined is updated after the combination operation is performed once, then checking the next frame of the frame corresponding to the new intermediate data, judging whether the frame is a target frame or not, if so, continuing to combine the pointer drag movement amount of the frame to be combined, continuously updating the intermediate data, and continuing the circulation process. However, the merging operation is not performed without limitation, but has a corresponding stopping condition, that is, the merging operation is stopped until the frame following the intermediate data is not the target frame, that is, a frame which does not meet the target frame judging condition is encountered, or when the condition of the pointer dragging movement amount corresponding to the intermediate data generated in the merging process exceeds a preset third threshold (the third threshold is a limit comprehensively determined according to factors such as reasonable pointer movement amplitude, operation logic and the like in the actual operation, and the phenomenon that the following judgment is influenced due to the unreasonable overlarge or undersize condition of the merged data is avoided).

In one embodiment, the reverse displacement of the pointer drag movement amount is performed on the data of each frame based on the pointer drag time interval and the pointer drag direction in the order of frame record or reverse order in the pointer drag operation data, including:

When the user stops the pointer drag operation on the aerial operation device, triggering an instruction for carrying out reverse displacement of the pointer drag movement amount on the data of each frame based on the pointer drag time interval and the pointer drag direction in order to correct the sequence or reverse sequence of frame records in the pointer drag operation data.

In an embodiment, when it is precisely detected that the user stops performing the pointer drag operation on the air operation device, this key event becomes a start signal for triggering a subsequent series of operations. For example, after a piece of content drag operation in document editing is completed by a stylus (air operation device) using a tablet computer, a signal of this stop operation is immediately captured at the moment the stylus is lifted. At this time, a preparation work for the pointer rebound operation is rapidly started, and a reverse displacement operation of the pointer drag movement amount is prepared for each frame of data based on the pointer drag time interval and the pointer drag direction according to the order or reverse order of the frame records in the corrected pointer drag operation data. The trigger mechanism ensures that the reverse displacement operation is timely and accurately started after the user completes one complete drag operation, and accords with the logic sequence and expected effect of the user operation.

In one embodiment, the order-based reverse displacement operation includes:

If the reverse shift operation is selected in the order of frame records in the corrected pointer drag operation data, the processing is started from the first frame data. First, the pointer drag time interval and the pointer drag direction information of a first frame and the pointer drag movement amount corresponding to the frame are acquired. The direction of the reverse displacement is determined according to the pointer drag direction, and if the pointer drag direction is rightward (assuming a horizontal direction), the reverse displacement direction is leftward. Then, a waiting time is set in accordance with the pointer drag time interval of the frame. After the waiting time is over, the pointer is moved by a corresponding pointer drag movement amount according to the reverse displacement direction. For example, in a scenario where a smart tv remote controller (air operation device) controls a screen pointer to browse a program list, if a first frame pointer is moved rightward by a certain distance, the pointer is moved leftward by the same distance after waiting for a corresponding time at the time of reverse displacement.

After the first frame is processed, the second frame is then processed in sequence. The steps of acquiring information, determining the reverse displacement direction, waiting time and moving the pointer are repeated. And so on, doing so for each frame of data until all frames are processed. In this process, the pointer is gradually traced back in the reverse order and the movement amount of the previous drag operation, and the reverse displacement from the end point to the start point is realized. The reverse displacement operation based on the sequence can intuitively simulate the rebound of the pointer according to the reverse process of the operation sequence, and is suitable for scenes with strict requirements on the operation sequence or the need of gradually restoring the operation path.

In one embodiment, the reverse displacement operation based on the reverse order includes:

When the reverse shift operation is selected in the reverse order of the frame records in the modified post-pointer drag operation data, the operation starts from the last frame data. First, the pointer drag time interval, the pointer drag direction and the pointer drag movement amount of the last frame are acquired. The reverse displacement direction is also determined according to the pointer drag direction, and if the pointer drag direction is upward (assuming a vertical direction), the reverse displacement direction is downward. Then, a waiting time is set at the pointer drag time interval of the frame, and after the waiting time is ended, the pointer is moved by a corresponding pointer drag movement amount in the reverse displacement direction. For example, in a scene where a picture is selected by controlling a screen pointer through an air gesture (air operation device) using a mobile phone, if the pointer of the last frame moves up by a certain distance, the pointer moves down by the same distance after waiting for a corresponding time at the time of reverse displacement.

After the last frame is processed, the penultimate frame is then processed. The steps of acquiring information, determining the reverse displacement direction, waiting time and moving the pointer are repeated, and each frame of data is operated in sequence from back to front. In this process, the pointer is gradually traced back from the last operation position in the direction opposite to the operation sequence, and is rapidly moved to the start position. The reverse displacement operation based on the reverse sequence can enable the pointer to return to the initial position more quickly under some conditions, is suitable for scenes with higher requirements on operation efficiency and less attention to details of operation paths, such as operation of returning to the top of a list after fast browsing a long list, and can quickly move the pointer back to the initial position through reverse sequence reverse displacement, so that waiting time of a user is reduced, and operation experience is improved.

Whether the reverse displacement operation is based on the sequence or the reverse sequence, the data of each frame needs to be processed in the whole process, the waiting and moving operation is carried out according to the pointer dragging time interval, the stable and accurate reverse displacement process of the pointer is ensured, and the reverse displacement operation can be well cooperated with other functions of the system (such as the processing of the user interrupt operation and the like) so as to realize the efficient and smooth pointer control effect and meet the requirements of the user in the scene of controlling the pointer of the touch screen by various air operation equipment.

In one embodiment, fig. 6 is a specific example diagram of a control method of a touch screen pointer according to an embodiment of the present invention, as shown in fig. 6, further including:

Step 601, in the process of carrying out reverse displacement of pointer drag movement amount on the data of each frame based on the pointer drag time interval, if the fact that the user carries out pointer drag operation again is detected, after the process of completing the reverse displacement is finished, acquiring pointer drag operation data which are carried out again by the user, wherein the starting record time of the pointer drag operation data which are carried out again is the process end time of completing the reverse displacement;

and 602, performing reverse displacement again based on the performed pointer drag operation data again.

In the embodiment, when the reverse displacement operation of the pointer drag movement amount is performed on the data of each frame based on the pointer drag time interval according to the predetermined rule, the whole process needs to keep monitoring of the user operation condition all the time. For example, after the tablet computer is used for controlling the screen pointer to carry out text movement operation in document editing through the stylus (serving as an air operation device), when the pointer starts to carry out reverse displacement according to the data recorded before, namely, moves back to the initial position, if the fact that the user carries out pointer dragging operation on the screen again through the stylus is detected at the moment, the current reverse displacement operation is not interrupted immediately, but is continued until the current whole reverse displacement process is finished.

This is done to ensure the integrity of each operation and the consistency of the data record. Only after the whole reverse displacement process is completed, the pointer dragging operation data which are carried out by the user again can be obtained. The data of the repeated pointer drag operation has an explicit requirement of the initial recording time, and the initial recording time is the time when the reverse displacement process is just finished. This means that from this specific moment on, the new user operation is regarded as an independent and complete operation phase, and all subsequent data records related to it will be spread around this starting point, so as to clearly and accurately capture the new operation intention of the user and various details in the operation process, such as the moving direction, moving amount of the pointer and the corresponding time interval of each operation, and provide an accurate data basis for subsequent further data processing and corresponding function implementation.

When the pointer drag operation data which is performed again by the user is successfully acquired, the reverse displacement operation is performed again according to the new data. The aim of the step is to enable the pointer to reasonably return according to the corresponding rule after the new operation stage is finished, so that the consistency of the whole pointer operation process in different stages is ensured, and the operation habit and the expectation of a user are met.

Based on the sorted pointer drag operation data, the reverse displacement operation is started to be actually executed. From the first frame of new data, the pointer drag time interval and the pointer drag direction information corresponding to the frame are acquired, and the reverse displacement direction is determined according to the pointer drag direction, for example, the pointer drag direction moves rightwards in the horizontal direction, and then the reverse displacement direction moves leftwards in the horizontal direction. And setting corresponding waiting time according to the pointer dragging time interval recorded by the frame, and after the waiting time is ended, enabling the pointer to move by a corresponding pointer dragging movement amount according to the determined reverse displacement direction.

And then, sequentially processing each subsequent frame of data such as a second frame, a third frame and the like according to the same steps, and gradually backtracking the pointer to realize the backward movement from the last operation position to the corresponding initial position. Through the complete reverse displacement operation again, no matter how many times the user performs the pointer drag operation at different stages, the pointer can be guaranteed to be reasonably returned finally, the whole control flow of the pointer of the touch screen is more flexible and reliable, the operation experience of the user when the user uses the air operation equipment to control the pointer of the touch screen is further improved, and the diversified practical application requirements are met.

In one embodiment, further comprising:

And in the process of carrying out reverse displacement of the pointer drag moving amount on the data of each frame based on the pointer drag time interval, if the user is detected to carry out the movement operation of the pointer, stopping the process of carrying out reverse displacement of the pointer drag moving amount based on the pointer drag time interval, and controlling the pointer to respond to the movement operation of the pointer.

In the embodiment, when performing the reverse displacement operation of the pointer drag movement amount on the data of each frame based on the pointer drag time interval according to the predetermined flow, the whole process needs to keep close attention and accurate monitoring on the user operation condition all the time.

When the pointer starts to reversely shift according to the previously recorded frame data, namely, in the process of moving back according to the corresponding time interval and the reverse moving direction, whether a new operation action is generated by the user is detected in real time. If it is detected that the user has performed a movement operation on the pointer during the backward displacement, for example, the user presses the corresponding key again through the remote controller and moves the pointer, the user intends to make the pointer go to a new position or change the original movement track, at this time, the user immediately responds to stop the backward displacement process of the pointer drag movement amount based on the pointer drag time interval.

This is because the movement operation of the pointer newly initiated by the user represents its current latest operation intention, and in order to ensure the immediacy of the operation and to meet the expectations of the user, priority is given to ensuring that the new operation can be effectively performed. Therefore, the state is quickly switched, and the pointer is controlled in response to the movement operation of the new pointer.

Specifically, according to specific instructions sent by a user through an aerial operation device, such as information of moving direction, moving speed and the like (the information is accurately captured through an interaction mechanism of the operation device and a screen), the position and moving state of a pointer on the screen are adjusted in real time, so that the pointer moves on the screen according to the direction and the rhythm expected by the user, as if the previous reverse displacement operation never happens, the pointer can move accurately and smoothly on the screen according to the new operation intention by taking the new operation instruction of the user as a leading part, thereby providing continuous and convenient operation experience for the user, meeting diversified requirements of the user on pointer control of the touch screen under different time and different requirements, and guaranteeing that the whole operation process of the pointer of the touch screen can flexibly cope with various conditions without interference of the previous unfinished operation.

A specific example is given below to illustrate a specific application of the method of the invention.

In a specific embodiment, an implementation scheme of an automatic pointer rebound mechanism based on relative coordinates is presented, and the scheme mainly comprises the following steps:

1. Fig. 7 is a specific example diagram of a sliding recording stage in a control method of a touch screen pointer according to an embodiment of the present invention, and as shown in fig. 7, the sliding recording stage involves the following operations:

1. And (3) starting operation detection, namely continuously monitoring the interactive operation between the air operation equipment (such as a touch pad, an air mouse and the like) and the touch screen by a user. When it is detected that the user starts the pointer drag operation on the touch screen through the air operation device, the point is determined as a starting point A. In the process, the whole track record of the pointer movement is started.

2. And the moving amount record is used for accurately recording the moving amounts deltax and deltay of the pointer in the horizontal direction (x axis) and the vertical direction (y axis) each time the pointer transmits a dragging instruction in the process of dragging the pointer from the starting point A. These shift amount data reflect the position change of the pointer on the screen, and are important basic data for subsequent calculation and operation.

3. The time interval is recorded, and at the same time, the time interval delta t between each instruction transmission is also recorded. The time interval data can embody the rhythm and the speed of the user operation, and has important significance for judging the behavior characteristics of the user operation and optimizing the subsequent rebound operation.

4. And continuously recording until the operation is finished, namely continuously executing the recording operation until the user releases the operation equipment at the end point B, and finishing the pointer dragging operation to ensure that all data in the whole operation process are completely recorded.

When the user uses the air operation device to drag the pointer on the touch screen, each drag instruction is sent to enable the pointer to move according to the direction and distance expected by the user. For example, when a pointer on a computer screen is controlled by a touch pad to move a file icon, each time a command corresponding to a drag action made by a finger on the touch pad is sent, an explicit operation intention is shown, and the purpose of the command is to change the position of the pointer. The "data recorded in each frame" is also the detailed disassembly and recording of the whole pointer drag operation process, and the operation process is recorded successively according to fixed time intervals or operation stage division. The frame data recorded each time corresponds to the operation of the user to generate corresponding change of the pointer through the air operation device at the moment, which is basically consistent with the action of sending a drag instruction to drive the pointer to move, and reflects an instant action and intention of the user in the operation process.

Each drag instruction is sent as a discrete operation point on the timeline that is arranged in turn to form the complete time series of the entire drag operation. For example, when the remote controller is used for operating the screen pointer of the intelligent television to browse the program list, the user presses a key of the remote controller and moves the pointer each time, or a drag instruction is sent, and the instructions are distributed on the time axis of operation in sequence. The data recorded in each frame is recorded according to time sequence, each recorded frame has a corresponding time stamp, and the time position of the recorded frame in the whole operation process is marked, so that the recorded frames are reflected on different time nodes in the time dimension, and have internal consistency.

Each time a drag command is sent, the corresponding state change of the pointer, such as the change of the position, the movement speed and the like of the pointer, is necessarily caused. The "record data per frame" includes information such as the amount of pointer drag movement, the direction of pointer drag, and the time interval of pointer drag, and these information are just detailed records of the state change of the pointer after each drag command is sent. For example, when a drag command is transmitted once to move the pointer from the left position to the right of the screen, "each frame of recording data" can record how much the pointer has moved in each small time interval (the amount of pointer drag movement), in which direction (the direction of pointer drag), and how long the adjacent two frames are spaced (the time interval of pointer drag), the effect produced by each transmission of the drag command can be completely restored by these recording data, both of which are closely corresponding in terms of recording the state of the pointer and reflecting the operation result.

2. Fig. 8 is a specific example diagram of a data processing stage and a rebound execution stage in a control method of a touch screen pointer according to an embodiment of the present invention, where, as shown in fig. 8, the data processing stage (including optimization processing) involves the following operations:

1. the frame-by-frame analysis and the stay determination are that the pointer drag time interval corresponding to the target frame in the embodiment is larger than a first threshold value and the corresponding pointer drag moving amount is smaller than or equal to a second threshold value, and the recorded data are subjected to frame-by-frame calculation analysis. If the moving distance of the pointer in both the horizontal direction (deltax) and the vertical direction (deltay) between the two recording points is smaller than a preset value and the time interval deltat between the two recording points exceeds the preset value (e.g., 50 ms), it is determined that the user stays there for a long time. To optimize data processing and subsequent rebound operations, the dwell time may be set to a fixed time T1 (e.g., 30 ms). The processing mode can reduce unnecessary time calculation, ensure that the stop state of the mouse is accurately judged, and ensure that acceleration influence on pointer movement caused by the stay state is avoided when the subsequent frame data is processed, so that the data processing is more accurate and efficient.

2. And (3) continuous stay processing, namely accumulating the time of continuous frames and classifying the time into one frame for processing if stay conditions of continuous time exist, namely the moving distance among a plurality of continuous recording points is smaller than a preset distance. The moving distance at this time is the cumulative distance of several consecutive frames, and the stay time is also set to T1. By the method, continuous stay conditions can be processed more reasonably, data redundancy is reduced, the data processing process is further optimized, and more accurate data support is provided for subsequent rebound operation.

3. And (3) optimizing the excessive drag time (extreme case), namely in the data processing process, if the excessive drag time of the user is detected, even the extreme case that the pointer is dragged back and forth between A and B occurs, corresponding measures are taken to reduce the rebound time. In this case, if the pointer is processed according to the conventional algorithm, the pointer will be moved back and forth for a long time, which seriously affects the operation efficiency and the user experience. Thus, the recorded data is analyzed and processed by a specific algorithm, for example, after all movement data is recorded, before moving backward, it is checked whether the pointer moves backward, and if so, the backward movement distance and the forward movement are offset from each other algorithmically. And then moving the cursor within a certain time range to enable the cursor to approximately return to the original position, avoiding unnecessary long-time reverse movement and improving the operation efficiency.

3. Fig. 8 is a specific example diagram of a data processing stage and a rebound execution stage in a control method of a touch screen pointer according to an embodiment of the present invention, where, as shown in fig. 8, the rebound execution stage involves the following operations:

1. Delay setting-a short delay T1 or more (e.g. 50 ms) is set after the user releases the operating device. The main purpose of this delay setting is to reduce the problem of inaccurate rebound caused by the cursor speed gain. Since the cursor may generate a certain gain effect due to the speed change during the pointer dragging process, if the rebound operation is performed immediately, the rebound position may be inaccurate. By appropriate delay, it is possible to make enough time to stabilize the cursor state, improving the accuracy of the rebound operation.

2. And executing reverse movement, namely sequentially executing reverse movement operation according to the original instruction interval according to the previously recorded instruction after the delay is finished. Namely, for the recorded horizontal direction movement amount deltax, a reverse movement-deltax is performed, and for the vertical direction movement amount deltay, a reverse movement-deltay is performed. In this way, the pointer can be gradually moved from the end point B in the direction opposite to the previous drag, bringing the pointer gradually closer to the start point a.

3. Rebound completion determination the pointer should be able to return substantially to the starting point a after execution of the reverse movement instruction is completed. The "approximate" here means that a certain error may exist in the processing process, but through the recording, processing and executing mechanisms, the pointer can be ensured to return to the vicinity of the starting position within a reasonable error range, so that the automatic rebound function of the pointer is realized, the user does not need to manually move the pointer back to the starting point, and the operation efficiency is improved.

4. Optimizing measures:

1. optimization (repetition) of too long drag time As described above, in the data processing stage, aiming at the situation that the drag time of the user is too long (including back and forth drag), the rebound time is reduced by the optimization algorithm, so that the pointer can return to the approximate initial position in a reasonable time, and the influence on the operation efficiency and the user experience due to too long rebound process is avoided.

2. User interrupt processing:

(1) New drag gesture processing, namely if the user is detected to start a new drag gesture in the rebound process, suspending the current rebound operation and treating the new drag operation as priority processing matters. And after the current rebound operation is completed, re-determining a target rebound operation mode and executing new pointer rebound operation according to the new pointer drag operation data and the sliding record, the data processing and rebound execution flow. Therefore, the new operation of the user can be timely responded, the conflict between the new operation and the old operation is avoided, and the continuity and the accuracy of the operation are ensured.

(2) Pointer movement processing, namely if the user is detected to move the pointer (instead of starting a new drag gesture) in the rebound process, immediately stopping the rebound process, and preferentially responding to pointer movement operation. At this time, the pointer position is updated according to the new pointer movement instruction of the user, and after the pointer movement operation is finished, whether the rebound operation needs to be re-executed or the processing is performed according to the new operation flow is judged according to the current pointer state and operation condition. The flexible interrupt processing mechanism can be better adapted to various changes of a user in the actual operation process, and the response speed and the user experience are improved.

In this embodiment, in the case of dragging the pointer, if the pointer slides back and forth for a long time for many times, the conventional rebound operation will cause the pointer to correspondingly move back and forth for many times during rebound, which inevitably reduces the operation efficiency, adversely affects the user experience, and therefore requires special algorithm optimization.

In the specific operation process, the whole process keeps complete record of pointer movement data, and all information from the initial position to each intermediate position and the final end position is recorded, wherein the information comprises horizontal and vertical movement change, corresponding time sequence and other data. And starting an optimization algorithm flow immediately before the pointer rebound operation is executed. First, a comprehensive check is developed on all the recorded movement data, with emphasis on determining whether the pointer has a reverse movement situation during dragging. The determination of the reverse movement is determined by comparing the change in the direction of movement of the pointer between adjacent data points based on an accurate analysis of the direction of movement.

Once the pointer is perceived to have backward movement, special processing is implemented at the algorithm level, and the backward movement distance and the forward movement distance are mutually counteracted. For example, if the pointer is moved 100 pixels prior to the positive x-axis direction and then 30 pixels in the negative x-axis direction, then in the optimization algorithm, the effective forward movement distance is recorded as 70 pixels (100-30). By the method, redundant movement calculation caused by back and forth sliding can be effectively reduced, and the data quantity required to be processed in the rebound process is reduced.

After the offset processing of the reverse moving distance and the forward moving distance is completed, the cursor is moved within a certain reasonable time range. The time range is set by comprehensively considering factors such as equipment performance, real-time requirement of user operation, stability of pointer movement and the like. The pointer movement is controlled in a relatively smooth and efficient manner based on the remaining effective movement data so as to approach toward the home position. Although the cursor can not return to the initial starting position accurately due to the interference of various factors in the offset processing and actual operation, the cursor can return to the vicinity of the initial position by the aid of the efficiency of the optimization algorithm, so that the requirement of a user for continuing to carry out subsequent operation is met, a long-time and complex reverse moving process caused by a traditional rebound mode is avoided, the operation efficiency and the user experience are remarkably improved, and the pointer control is intelligent and efficient under a complex dragging scene.

The specific embodiment has the following technical effects:

1. operational efficiency improvement

The automatic pointer springback mechanism of the present invention is implemented in relative coordinates, which feature allows a user to move a pointer back to the starting point without manual intervention after the pointer operation. For example, in a scenario where the pointer drag operation from the point a to the point B is frequently performed, such as when browsing a long video list on a smart tv, after a user selects one video to play, the pointer automatically rebounds to the initial position, and manual operation by the user is not required, so that repetitive operation is greatly reduced. The automated rebound process saves user operating time, significantly improves work efficiency, and enables the user to concentrate more on the operating task itself rather than spending time adjusting the pointer position.

2. User experience optimization

The design of the automatic rebound mechanism simulates the feel of touch operation. When an air operation device (such as a touch pad, an air mouse and the like) interacts with a touch screen, the automatic rebound behavior of the pointer is similar to the automatic reset feeling of an interface element after the finger leaves the screen in the direct touch operation. Taking vehicle-mounted information entertainment as an example, when a driver selects a navigation destination or switches music tracks through an air gesture control screen, the process that the pointer automatically rebounds to the starting position enables operation to be more visual and natural, and as if touch operation is directly performed on the screen, the sense of inadaptation caused by difference of operation modes is reduced, the operation convenience and comfort of the driver in the driving process are improved, and the satisfaction degree of the user to the whole interaction process is enhanced.

3. Cross-platform compatibility achievement

The implementation based on relative coordinates brings excellent cross-platform compatibility to the mechanism. The pointer automatic rebound mechanism can effectively operate no matter which operating system or different types of devices (such as a mobile phone, a tablet, a vehicle-mounted screen, a smart television, an education tablet, an augmented reality device, a virtual reality device and the like). On different platforms and devices, the relative coordinates focus on the relative movement relationship of the pointer within the screen, rather than relying on absolute position information of a particular device. The method ensures that developers do not need to conduct a large amount of customized development aiming at different operations and devices, reduces development cost and complexity, ensures that users can enjoy consistent pointer control experience on different devices and platforms, and improves the universality and expandability of the technical scheme.

4. Flexibility and controllability enhancement

By providing optimization measures such as threshold setting, speed control, etc., the rebound mechanism exhibits a high degree of flexibility and controllability. In terms of threshold setting, for example, thresholds of the pointer drag duration and distance may be set, and when the user operates within different threshold ranges, different rebound modes may be automatically selected. In the aspect of speed control, the speed of pointer rebound can be adjusted according to different application scenes and user operation habits. If on the education flat, the teacher may want the pointer rebound speed to be moderate when carrying out explanation demonstration, so that the student can clearly observe the pointer movement track, and if the user browses the long list content rapidly, for example, searching the program on the intelligent television, the pointer rebound speed can be properly improved to improve the operation efficiency. The flexibility enables the rebound mechanism to be better suitable for diversified user requirements and operation scenes, and intelligence and customizable performance of the mechanism are improved.

5. Wide adaptability

The pointer automatic rebound mechanism is particularly suitable for application scenes requiring frequent sliding operation. When browsing long lists, such as commodity lists or news information lists of an electronic commerce platform, a user continuously slides and browses, then the pointer automatically rebounds, so that the user can conveniently and quickly return to the top of the list to perform new browsing operation, in page turning operation, whether electronic book reading or document viewing is performed, the pointer rebounds to the initial position, so that the user can conveniently perform page turning operation of the next page, and in operations such as volume/brightness adjustment, for example, in a smart television or a vehicle, the user can automatically rebounds after adjusting the pointer, thereby avoiding interference to subsequent operations caused by pointer position change. The wide adaptability enables the technical scheme to play an important role in a plurality of common interaction scenes, and the convenience and fluency of user operation are improved.

6. Simulation of finger sliding effect implementation

The invention realizes the operation effect similar to direct touch control and effectively enhances the natural interaction experience of the user. During operation, the movement and rebound behavior of the pointer is similar to the feeling of a finger sliding on the screen. Taking augmented reality and virtual reality devices as examples, when a user controls a pointer through an aerial operation device in a virtual space, an automatic rebound mode of the pointer simulates operation logic of the finger on a virtual interface, so that the user can more easily understand and adapt to an interaction mode in the virtual environment, as if the user directly interacts with virtual elements through the finger, the immersion of the user in the virtual scene and the naturalness of the operation are improved, and the interaction distance between the user and the virtual interface is shortened.

The specific embodiment has the following innovation points:

1. relative coordinates to realize automatic rebound advantage

The use of relative coordinates to achieve automatic rebound is an important innovation of the present invention. The relative coordinates focus on the relative position change of the pointer in the screen coordinate system, and compared with the traditional absolute coordinates, the relative coordinates overcome various limitations. In absolute coordinates, the position of the pointer depends on the absolute physical location information of the device, which makes the control logic of the pointer complex and difficult to unify in different devices or in different operating environments. For example, in some mobile devices, changes in screen orientation or differences in device hardware may cause deviations in pointer position calculations in absolute coordinates, affecting the accuracy and consistency of user operation. The relative coordinates of the invention can stably realize the automatic rebound function of the pointer under different equipment and operation scenes by recording the movement amount (delta x and delta y) of the pointer relative to the starting point and the operation time interval (delta t), thereby ensuring the accuracy and reliability of the operation and providing more consistent and convenient pointer control experience for users.

2. Recording and replaying instructions to achieve accurate rebound

The invention achieves accurate rebound by recording and replaying movement instructions, which innovative way does not need to rely on absolute position information of the device. During the pointer operation, the movement amounts (Δx and Δy) and the instruction time interval (Δt) of each transmission of the drag instruction are accurately recorded. These recorded data become the key basis for pointer resilience. When the user releases the operation device, reverse movement (-deltax and-deltay) is sequentially executed according to the recorded instruction at the original instruction intervals, so that accurate rebound of the pointer from the end point to the starting point is realized. This approach avoids the problem of inaccurate rebound due to device hardware differences or differences in the understanding of absolute positions by the operation. For example, on mobile phones or flat panels of different models, although hardware parameters such as screen size, resolution and the like of the equipment may be different, a rebound mechanism based on recording and replaying movement instructions can ensure that a pointer can rebound to the vicinity of a starting position according to an expected track on various equipment, so that rebound accuracy and reliability are improved, and pointer control is more accurate and stable.

3. Delay mechanism for improving rebound accuracy

The introduction of a delay mechanism is another innovation point of the invention, and the problem of inaccurate rebound caused by the speed-following gain of the cursor is effectively reduced. In the process of pointer dragging, the speed change of a cursor may cause a speed following gain effect, so that the initial speed and the track of the cursor during rebound are difficult to accurately control, and the rebound accuracy is further affected. The present invention sets a short delay (e.g., T1 or greater than T1, e.g., 50 ms) after the user releases the operating device. During the delay period, the speed and state of the light can be stabilized, the influence of the speed-following gain is eliminated, and the preparation is made for the subsequent accurate rebound operation. Through the delay mechanism, the pointer rebound process can be started in a more stable and accurate state, the pointer is ensured to return to the initial position according to the preset path and speed, the accuracy and reliability of rebound operation are improved, the rebound behavior of the pointer can be expected more accurately by a user in the operation process, and the overall operation experience is improved.

4. Optimization measures to enhance rebound intelligence and adaptability

A series of optimization measures such as rebound threshold value, speed control and the like are designed, so that the intelligence and adaptability of a rebound mechanism are remarkably improved. The setting of the rebound threshold, for example, the threshold of the pointer drag duration and distance, can automatically select an appropriate rebound mode according to the characteristics of the user operation. When the user performs quick and small-range pointer operation, the quick and simple rebound mode can be selected through threshold judgment, and for long-distance and complex pointer operation, the corresponding more suitable rebound mode is selected according to the threshold. In the aspect of speed control, the rebound speed of the pointer can be dynamically adjusted according to different application scenes and user operation habits. In a scene requiring quick response, such as game operation or quick browsing of an interface, the rebound speed is increased, and in a scene requiring accurate operation, such as drawing or fine adjustment setting, the rebound speed is reduced. The optimization measures enable the rebound mechanism to be intelligently adapted to the requirements of different user operations and application scenes, more personalized and efficient pointer control service is provided, and the practicability and flexibility of the technical scheme are enhanced.

5. Cross-platform implementation ensures consistent experience

Providing a cross-platform implementation is one of the key innovation points of the present invention, ensuring a consistent experience across different operations and devices. Based on relative coordinates and unified pointer control logic, the invention can realize the same pointer automatic rebound function on different mobile phone operating systems and various devices such as mobile phones, flat plates, vehicle-mounted screens, intelligent televisions, education flat plates, augmented reality devices, virtual reality devices and the like. No matter what platform and equipment are, the user can feel similar operation modes and interaction experiences, and the learning cost of the user when switching between different equipment is reduced. For example, when a user uses the vehicle-mounted information entertainment and the mobile phone, the operation and resilience of the pointer are expected to be consistent, so that the acceptance and satisfaction of the user on the whole technical scheme are improved, the wide application of the technology in different fields and equipment is promoted, and the universality and compatibility of the technical scheme are enhanced.

The specific embodiment has the following potential application scenarios:

1. In-vehicle infotainment application

In vehicle-mounted infotainment, when a driver controls a screen through an air gesture, the pointer automatic rebound mechanism plays an important role. Because the driver needs to concentrate on road conditions during driving, the manual operation screen is inconvenient and unsafe. Through aerial operation equipment (such as a touch pad on a vehicle-mounted center console or a controller with gesture recognition function), a driver can conveniently control a pointer to move on a screen, select a navigation destination, switch music tracks, adjust a radio channel or operate other vehicle-mounted application functions. The pointer automatically rebounds to the starting position, so that a driver can quickly perform the next operation after completing one operation, the position of the pointer is not required to be adjusted by distraction, and the operation efficiency and the driving safety are improved. For example, in the navigation operation, the driver automatically rebounds the pointer after selecting the destination, which is convenient for the driver to immediately conduct route planning or view other navigation information, and when the music is played, the volume is adjusted or the rebounds of the pointer after switching songs are adjusted, so that the operation is smoother, the interference to the driving caused by inconvenient operation is reduced, and the interactive experience of the driver in the vehicle is improved.

2. Smart television or large screen device application

For smart televisions or large screen devices, users typically operate using a remote control or a space mouse. When browsing long list content (such as a video program list, an application program list or a set option list, etc.), the pointer automatically rebounds to the top or the starting position of the list after frequent sliding operation, so that the user can conveniently and quickly return to and make new selection. For example, when a user watches a network video platform, after browsing a plurality of video programs, the pointer rebounds to enable the user to easily return to the top of the list to select new classification or screening conditions, and when the intelligent television parameters are set, the pointer is automatically reset after the settings are adjusted, so that the next setting operation is facilitated. The automatic pointer rebound mechanism greatly improves the operation convenience of a user when the intelligent television or the large-screen device is used, so that the user can interact with the device more efficiently and enjoy rich multimedia content and convenient device control.

3. Education tablet application

In the educational field, educational tablets provide a teacher with the convenience of controlling screen content at a distance. The teacher can control the teaching courseware on the tablet to display, mark key content, switch pages or start different teaching application programs through the air operation equipment (such as a wireless stylus or a remote controller matched with the tablet) at different positions in the classroom. The automatic rebound of the pointer to the starting position helps the teacher maintain consistency and efficiency of the operation. For example, when teaching electronic courseware, a teacher marks one knowledge point and then the pointer rebounds, so that the teacher can immediately mark the next knowledge point or switch to the next courseware, and when teaching software is used for interactive teaching, the pointer is reset after mathematical formula derivation or graphic drawing is performed, so that the subsequent operation is facilitated. The teaching efficiency of teachers is improved, the teaching process is smoother, the understanding and the attention of students to teaching contents are enhanced, and a more efficient and convenient interaction mode is provided for modern education and teaching.

4. Augmented Reality (AR) and Virtual Reality (VR) device applications

In augmented reality and virtual reality devices, the pointer automatic rebound mechanism of the present invention provides a more natural virtual interface interaction approach. When a user interacts with the virtual interface in the virtual space through the head-mounted device and the matched controller (serving as an air operation device), the automatic rebound of the pointer simulates the feeling of the interaction of the finger and the object in the real world. In VR building design software, a designer uses the pointer to select building elements, adjusts the layout and then automatically resets the pointer, thereby facilitating the next round of design operation and improving the design efficiency and the naturalness of interaction. The pointer automatic rebound mechanism in the virtual environment is beneficial to shortening the distance between the user and the virtual interface, so that the user can be more naturally integrated into the virtual world, and the interaction experience and the working efficiency of the user in AR and VR applications are improved.

Of course, it is to be understood that other variations of the above detailed procedures are also possible, and all related variations should fall within the protection scope of the present invention.

As described above, the present invention focuses on solving the significant defect existing in the prior art, that is, in the situation that the user breaks away from the touch screen and controls the touch interface by means of the pointer, the operation of frequently manually recalling the pointer from the end point to the start point brings great inconvenience to the user. Based on this, the present invention establishes the following specific objects:

1. The operation efficiency of the user is remarkably improved, repeated actions are reduced to the greatest extent, and the automatic rebound function of the pointer is realized through an innovative technical means. For example, in an application scenario such as a smart television, when a user uses an air operation device (such as a remote controller) to frequently perform switching operation between different function menus, the pointer can automatically rebound to an initial position after each selection is completed, so that extra time and effort consumed by the user to manually adjust the pointer position are effectively avoided, the operation flow is greatly accelerated, the user can complete more operation tasks in unit time, and the overall operation efficiency is remarkably improved.

2. The user experience is deeply improved, and a use feeling which is highly similar to the direct touch experience is created. In the process of interaction between the air operation equipment and the touch screen, the movement characteristic and the rebound mechanism of the pointer are carefully designed, so that the pointer is similar to direct touch operation in terms of operation logic and visual feedback. For example, in vehicle-mounted information entertainment, when a driver performs navigation destination setting or music playing control by means of an air gesture control screen, the automatic rebound process of a pointer is smooth and natural, visual feeling like touch operation is directly performed on the screen is brought to the driver, uncomfortable feeling caused by operation mode difference is reduced, and satisfaction and comfort of the user when the air operation equipment is used are comprehensively improved.

3. Constructing a cross-platform and high-compatibility solution, namely carefully constructing a general technical framework based on relative coordinates, ensuring that the technical framework can be seamlessly adapted to various operating systems and diversified device types (including mobile phones, flat plates, vehicle-mounted screens, intelligent televisions, education flat plates, augmented reality devices, virtual reality devices and the like). By the mode, no matter what operation platform or equipment hardware configuration is, a user can enjoy consistent and stable pointer control and rebound experience, complicated customized setting or adjustment is not needed for different equipment or is carried out, development cost and maintenance difficulty are effectively reduced, and the application range and market potential of the invention are greatly expanded.

4. The finger sliding effect is accurately simulated, namely the operation effect similar to the direct sliding of the finger on the touch screen is achieved through fine optimization on the multiple aspects of the movement track, the speed change, the rebound mode and the like of the pointer. In the interaction scene of the augmented reality or virtual reality equipment, when a user operates the virtual interface by means of the matched controller, the pointer can move and rebound in a very natural mode which accords with the operation habit of a human body as if the finger of the user directly interacts with the virtual element, so that the naturalness and immersion of the operation are effectively enhanced, the user can be more deeply integrated into the virtual interaction environment, and the reality and fluency of the interaction experience are improved.

The embodiment of the invention also provides a control device of the touch screen pointer, which is expressed in the following embodiment. Because the principle of the device for solving the problem is similar to that of the control method of the touch screen pointer, the implementation of the device can refer to the implementation of the control method of the touch screen pointer, and the repetition is omitted.

The embodiment of the invention also provides a control device for a pointer of a touch screen, which is used for solving the problem of low efficiency of manually moving back the pointer, improving the control efficiency of the touch screen, improving the user experience and improving the cross-platform compatibility, and fig. 9 is a schematic structural diagram of the control device for the pointer of the touch screen in the embodiment of the invention, as shown in fig. 9, and the device comprises:

The pointer drag operation data acquisition module 901 is used for acquiring pointer drag operation data of a user on an aerial operation device in real time, wherein the pointer drag operation data comprise pointer drag movement quantity, pointer drag direction and pointer drag time interval of each frame in the drag operation process recorded frame by frame, wherein the pointer drag operation data is used for displaying a pointer which moves in response to the pointer drag operation in the touch screen;

And a backward displacement module 902, configured to perform backward displacement of the pointer drag movement amount on the data of each frame based on the pointer drag time interval and the pointer drag direction in the order of frame record in the pointer drag operation data or in the reverse order after drag release.

In one embodiment, further comprising:

If yes, determining the frame record data as a target frame.

In one embodiment, further comprising:

In one embodiment, merging pointer drag movement amounts in recorded data of consecutive target frames to obtain a merged frame includes:

Acquiring a target frame with a previous time stamp as a frame to be combined according to the sequence of the time stamp of each frame record data from the first to the last;

determining whether a frame subsequent to the frame to be combined is a target frame;

If yes, combining the frame to be combined and the pointer dragging movement amount of the next frame to obtain intermediate data;

And replacing the frame to be combined with the intermediate data, repeating the steps of determining whether the next frame of the intermediate data is a target frame or not and combining until the next frame of the intermediate data is not the target frame or the intermediate data is larger than a third threshold value, and taking the obtained intermediate data as a combined frame.

In one embodiment, further comprising:

In the process of carrying out reverse displacement of the pointer drag movement amount on the data of each frame based on the pointer drag time interval, if the fact that the user carries out the pointer drag operation again is detected, after the process of completing the reverse displacement is finished, acquiring pointer drag operation data which are carried out again by the user, wherein the starting recording time of the pointer drag operation data which are carried out again is the process end time of completing the reverse displacement;

based on the pointer drag operation data performed again, the reverse displacement is performed again.

In one embodiment, further comprising:

The embodiment of the invention provides a computer device for realizing all or part of the content in the control method of the touch screen pointer, which specifically comprises the following contents:

The system comprises a processor (processor), a memory (memory), a communication interface (Communications Interface) and a bus, wherein the processor, the memory and the communication interface are used for completing communication among the related devices through the bus, the communication interface is used for realizing information transmission among the related devices, and the computer device can be a desktop computer, a tablet computer, a mobile terminal and the like, and the embodiment is not limited to the above. In this embodiment, the computer device may be implemented with reference to an embodiment of the method for implementing a control method of a touch screen pointer and an embodiment of the device for implementing a control apparatus of a touch screen pointer, and the contents thereof are incorporated herein, and the repetition is omitted.

Fig. 10 is a schematic block diagram of a system configuration of a computer device 1000 according to an embodiment of the present application. As shown in fig. 10, the computer device 1000 may include a central processor 1001 and a memory 1002, the memory 1002 being coupled to the central processor 1001. It is noted that this fig. 10 is exemplary, and that other types of structures may be used in addition to or in place of the structures to implement telecommunications functions or other functions.

In one embodiment, the control function of the touch screen pointer may be integrated into the cpu 1001. The central processor 1001 may be configured to control, among other things, the following:

Determining whether the frame record data is a target frame or not according to each frame record data, wherein the pointer drag movement amount corresponding to the target frame is smaller than or equal to a first threshold value and the pointer drag time interval is larger than a second threshold value;

In another embodiment, the control device of the touch screen pointer may be configured separately from the cpu 1001, for example, the control device of the touch screen pointer may be configured as a chip connected to the cpu 1001, and the control function of the touch screen pointer is implemented by the control of the cpu.

As shown in fig. 10, the computer device 1000 may further include a communication module 1003, an input unit 1004, an audio processor 1005, a display 1006, a power supply 1007. It is noted that the computer device 1000 does not necessarily have to include all the components shown in fig. 10, and furthermore, the computer device 1000 may include components not shown in fig. 10, to which reference is made in the prior art.

As shown in fig. 10, the central processor 1001, sometimes also referred to as a controller or operational control, may include a microprocessor or other processor device and/or logic device, and the central processor 1001 receives input and controls the operation of the various components of the computer device 1000.

The memory 1002 may be, for example, one or more of a buffer, a flash memory, a hard drive, a removable media, a volatile memory, a non-volatile memory, or other suitable device. The above-described information about the device may be stored, and a program for executing the information may be stored. And the central processor 1001 can execute the program stored in the memory 1002 to realize information storage or processing, and the like.

The input unit 1004 provides input to the central processor 1001. The input unit 1004 is, for example, a key or a touch input device. The power supply 1007 is used to provide power to the computer device 1000. The display 1006 is used for displaying display objects such as images and characters. The display may be, for example, but not limited to, an LCD display.

The memory 1002 may be a solid state memory such as Read Only Memory (ROM), random Access Memory (RAM), SIM card, and the like. But also a memory which holds information even when powered down, can be selectively erased and provided with further data, an example of which is sometimes referred to as EPROM or the like. Memory 1002 may also be some other type of device. Memory 1002 includes a buffer memory 1021 (sometimes referred to as a buffer). The memory 1002 may include an application/function storage 1022, the application/function storage 1022 for storing application programs and function programs or for executing a flow of operations of the computer apparatus 1000 by the central processor 1001.

The memory 1002 may also include a data store 1023, the data store 1023 for storing data such as contacts, digital data, pictures, sounds, and/or any other data used by a computer device. The driver store 1024 of the memory 1002 can include various drivers for the computer device for communication functions and/or for performing other functions of the computer device (e.g., messaging applications, address book applications, etc.).

The communication module 1003 is a transmitter/receiver that transmits and receives signals via the antenna 1008. A communication module (transmitter/receiver) 1003 is coupled to the central processor 1001 to provide an input signal and receive an output signal, which may be the same as in the case of a conventional mobile communication terminal.

Based on different communication technologies, a plurality of communication modules 1003, such as a cellular network module, a bluetooth module, and/or a wireless lan module, etc., may be provided in the same computer device. The communication module (transmitter/receiver) 1003 is also coupled to a speaker 1009 and a microphone 1010 via an audio processor 1005 to provide audio output via the speaker 1009 and to receive audio input from the microphone 1010 to implement usual telecommunications functionality. The audio processor 1005 may include any suitable buffers, decoders, amplifiers and so forth. In addition, an audio processor 1005 is also coupled to the central processor 1001 so that sound can be recorded locally through the microphone 1010 and so that sound stored locally can be played through the speaker 1009.

It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

The foregoing description of the embodiments has been provided for the purpose of illustrating the general principles of the invention, and is not meant to limit the scope of the invention, but to limit the invention to the particular embodiments, and any modifications, equivalents, improvements, etc. that fall within the spirit and principles of the invention are intended to be included within the scope of the invention.

Claims

1. A method for controlling a pointer of a touch screen, comprising:

2. The method as recited in claim 1, further comprising:

3. The method of claim 2, wherein for each frame record data, determining whether the frame record data is a target frame comprises:

If yes, determining the frame record data as a target frame.

4. A method as claimed in claim 2 or 3, wherein for each frame of recorded data, determining whether the frame of recorded data is a target frame comprises:

5. A method as claimed in claim 2 or 3, further comprising:

6. The method of claim 5, wherein combining the pointer drag movement amounts in the recorded data of the successive target frames to obtain the combined frame comprises:

7. The method as recited in claim 1, further comprising:

8. The method as recited in claim 1, further comprising:

9. A control device for a touch screen pointer, comprising:

10. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the method of any of claims 1 to 8 when executing the computer program.

11. A computer readable storage medium, characterized in that the computer readable storage medium stores a computer program which, when executed by a processor, implements the method of any of claims 1 to 8.

12. A computer program product, characterized in that the computer program product comprises a computer program which, when executed by a processor, implements the method of any of claims 1 to 8.