CN102981620A - Terminal operation method and terminal - Google Patents

Abstract

The invention discloses a terminal operation method and a terminal. The method includes collecting a human eye picture of an operator in front of a display screen of the terminal, and operating the terminal according to the collected human eye picture. Through the above technical solutions, the present invention solves the problem of human eyes operating the terminal.

Description

A terminal operation method and terminal

technical field

The present invention relates to the field of electronic equipment, in particular to a terminal operation method and a terminal.

Background technique

With people's pursuit of quality of life, the requirements for terminal user experience are getting higher and higher, and the design of terminal operation schemes is also attracting more and more attention.

In the existing terminal operation solutions, there are touch technology, voice control technology, camera gestures, etc., although to a certain extent, they have achieved the effects of usability, ease of use, fun, security, etc., but they all use simple functions. Alternative technologies fail to achieve human-computer interaction in terms of visual effects.

Contents of the invention

The present invention provides a terminal operation method and a terminal to solve the problem of terminal operation by human eyes.

In order to solve the problems of the technologies described above, the present invention adopts the following technical solutions:

A terminal operation method, comprising: collecting a human eye picture of an operator in front of a terminal display screen; and operating the terminal according to the collected human eye picture.

Further, the method for collecting the human eye picture of the operator in front of the terminal display screen is specifically: collecting the imaging of the current interface displayed on the terminal display screen on the retina of at least one eye of the operator.

Further, the method for operating the terminal according to the collected human eye pictures is as follows: calculate the coordinate point on the current interface that the human eye is gazing at according to the collected human eye picture; calculate the duration of the human eye gazing at the same coordinate point, Operate the terminal while the human eye is gazing at the same coordinate point; and/or, calculate the change trajectory and rate of change of the coordinate point that the human eye is gazing at within a preset time, according to the change trajectory and change rate of the coordinate point that the human eye is gazing at rate to operate on the terminal.

Further, the method for operating the terminal according to the duration of the human eyes gazing at the same coordinate point is specifically: comparing the duration of the human eyes gazing at the same coordinate point with a preset threshold, and if the duration is greater than the preset threshold, perform the The operation corresponding to the coordinate point.

Further, the process of executing the operation corresponding to the coordinate point specifically includes: querying the preset operation instruction associated with the coordinate point according to the coordinate point, and performing the operation according to the queried operation instruction.

Further, the operation corresponding to the coordinate point includes: enlarging the display scale, reducing the display scale, returning to the previous page and/or entering the next page.

Further, the method for operating the terminal according to the change track and rate of change of the coordinate point that the human eye is looking at is specifically: adopting the same change track and the same rate of change as the coordinate point that the human eye is looking at, and controlling the terminal display screen. Adjust the current interface displayed; or query the preset operation instructions associated with the coordinate point of the human eye according to the change track and change rate, and operate according to the query operation instruction.

A terminal includes a collection module and an operation module, wherein the collection module is used to collect a human eye picture of an operator in front of a display screen of the terminal; the operation module is used to operate the terminal according to the human eye picture collected by the collection module.

Further, the collection module is used to collect the image of the current interface displayed on the terminal display screen on the retina of at least one eye of the operator.

Further, the operation module includes a coordinate point calculation module, a coordinate point analysis module and an operation sub-module, wherein the coordinate point calculation module is used to calculate the coordinate point on the current interface that the human eye is looking at according to the collected human eye picture; the coordinate point The point analysis module is used to calculate the duration of human eyes gazing at the same coordinate point, and/or calculate the change trajectory and rate of change of the coordinate point that human eyes are gazing at within a preset time; the operation sub-module is used to fix the same coordinate point according to the human eye duration, and/or operate the terminal according to the change trajectory and change rate of the coordinate point that the human eye is looking at.

Further, the operation sub-module includes a comparison module and a first execution module, wherein the comparison module is used to compare the duration of human eyes gazing at the same coordinate point with a preset threshold; the first execution module is used for comparison in the comparison module The result is that when the duration is greater than the preset threshold, the operation corresponding to the coordinate point is executed.

Further, the first execution module is configured to query a preset operation instruction associated with the coordinate point according to the coordinate point, and perform an operation according to the queried operation instruction.

Further, the operation sub-module includes a second execution module, and the second execution module is used to adjust the current interface displayed on the terminal display screen by adopting the same change trajectory and the same change rate as the coordinate point gazed by human eyes; or According to the change track and change rate of the coordinate point that the human eyes are looking at, the preset operation instructions associated with it are queried, and the operation is performed according to the queried operation instructions.

The terminal operation method and the terminal provided by the present invention collect the human eye picture of the operator in front of the terminal display screen, operate the terminal according to the collected human eye picture, realize the human-computer interaction on the visual effect, and make the terminal display screen The display page is adjusted according to the changes of human eyes, which greatly improves the user experience.

Description of drawings

FIG. 1 is a flowchart of a terminal operation method provided by an embodiment of the present invention;

Fig. 2 is a schematic diagram of a terminal provided by an embodiment of the present invention.

Detailed ways

The present invention will be further described in detail below through specific embodiments in conjunction with the accompanying drawings.

Figure 1 is a flowchart of a terminal operation method provided by an embodiment of the present invention, please refer to Figure 1:

S101. Collect the human eye picture of the operator in front of the display screen of the terminal. The human eye picture can reflect the gazing area of the human eye on the current interface of the display screen and the change of the gazing area of the human eye. The gaze area includes points and surfaces.

Specifically, after the user enables the dynamic eye tracking function on the terminal, the front camera starts to run, and collects the human eye picture of the operator in front of the terminal display through the front camera. Normal people blink 15 times per minute, which is 0.25HZ, while ordinary cameras, such as Casio EX-H35, can achieve high-speed auto-focus photography in 0.13 seconds, that is, they can take more than 7 high-definition photos of 16 million pixels per second , the photographing frequency is 7HZ, which is far greater than the normal blinking frequency. In addition, high-speed cameras can generally achieve 1000 to 10,000 frames per second of video recording, so the present invention only needs to use a common camera to complete the collection of human eye pictures.

In an embodiment, the method of collecting the human eye picture of the operator in front of the terminal display screen may specifically be: collecting the imaging of the current interface displayed on the terminal display screen on the retina of at least one eye of the operator. Usually, when the operator watches the display screen of the terminal, the current display interface will become an upside-down and shrunken real image on the retina of the human eye. In this embodiment, it is very convenient to use this point to collect human eye pictures without additional design. Since the image formed on the retina of the human eye is an upside-down reduced image, after the image is collected, preprocessing can be performed, such as: enlargement, flipping, etc.

S102. Operate the terminal according to the collected human eye picture.

In an embodiment, the method for operating the terminal according to the collected human eye pictures may specifically be:

Step A1. Calculate the coordinate point on the current interface where the human eye is looking at according to the collected human eye picture.

Step B1, calculating the duration of human eyes gazing at the same coordinate point.

Step C1. Operate the terminal according to the duration of the human eyes staring at the same coordinate point.

Preferably, the method for operating the terminal according to the duration of the human eyes gazing at the same coordinate point may specifically be: comparing the duration of the human eyes gazing at the same coordinate point with a preset threshold, and if the duration is greater than the preset threshold, execute the same method as The operation corresponding to the coordinate point, for example: enlarge the display scale, reduce the display scale, return to the previous page and/or enter the next page.

Preferably, executing the operation corresponding to the coordinate point may be specifically: establishing an association relationship between each coordinate point on the current interface and the operation instruction in advance, and the association relationship may be stored in the storage space of the terminal itself, or may be stored in a device capable of communicating with the terminal. The external device inquires the operation instructions associated with the coordinate point that the human eye continues to watch, and then operates according to the inquired operation instructions. For example: the current interface of the terminal display screen displays text information, and the preset operation command associated with a certain coordinate point on the current interface is to enlarge the display ratio. When the threshold is reached, execute the operation instruction, that is, enlarge and display the text on the coordinate point, which can greatly improve the user experience.

In another embodiment, the method for operating the terminal according to the collected human eye pictures may specifically be:

Step A2. Calculate the coordinate point on the current interface where the human eye is looking at according to the collected human eye picture.

Step B2, calculating the change trajectory and change rate of the coordinate points that the human eyes are looking at within a preset time.

Step C2: Operate the terminal according to the change track and change rate of the coordinate point that the human eyes are looking at.

Preferably, the method for calculating the change trajectory and rate of change of the coordinate points that the human eye is watching within the preset time can be: collecting the human eye pictures within the preset time, and analyzing the coordinate points of the human eyes watching within the preset time. Change, to obtain the change trajectory and the change rate, the change rate refers to the rate at which the position of the coordinate point changes per unit time.

Preferably, the method of operating the terminal according to the change track and rate of change of the coordinate point that the human eye is looking at can specifically be: adopt the same change track and the same rate of change as the coordinate point that the human eye is looking at, and control the terminal display screen The displayed current interface is adjusted, for example: when the operator's human eye gaze moves vertically downward at a rate of 0.2cm/ms from the top of the current interface, the same rate and change trajectory are adopted, that is, 0.2 The rate of cm/ms moves the displayed page vertically downward, which is equivalent to manually operating the scroll bar of the current interface. The display page of the terminal display screen is adjusted according to the changes of human eyes, which greatly improves the user experience.

Or, it is also possible to pre-establish the relationship between the change track and the rate of change of each coordinate point on the current interface and the operation command. The relationship can be stored in the storage space of the terminal itself, or can be stored in an external device that can communicate with the terminal. According to the steps The change track and change rate of the coordinate point that the human eye is looking at calculated in B2 query the operation instructions associated with it, and operate according to the query operation instructions. For example: when the operator's gaze point moves vertically downward at a rate of 0.2cm/ms from the top of the current interface, query the operation associated with it according to the parameters of 0.2cm/ms and vertical downward Instructions, if the queried operation instruction moves vertically downward at a rate of 0.5cm/ms, execute the operation instruction.

Fig. 2 is a schematic diagram of a terminal provided by an embodiment of the present invention, please refer to Fig. 2: the terminal includes a collection module 21 and an operation module 22, wherein,

The collection module 21 is used to collect the human eye picture of the operator in front of the terminal display screen.

The acquisition module 21 may specifically be a camera. After the user activates the function of dynamically tracking human eyes on the terminal, the front camera starts to run, and collects the human eye pictures of the operator in front of the terminal display through the front camera. The human eye picture can reflect the gazing area of the human eye on the current interface of the display screen and the change of the gazing area of the human eye. The gaze area includes points and surfaces.

In an embodiment, the collection module 21 may be used to collect the image of the current interface displayed on the display screen of the terminal on the retina of at least one eye of the operator. Usually, when the operator watches the display screen of the terminal, the current display interface will become an upside-down and shrunken real image on the retina of the human eye. In this embodiment, it is very convenient to use this point to collect human eye pictures without additional design. Since the image formed on the retina of the human eye is an upside-down shrunken image, after the image is collected by the acquisition module 21 , preprocessing can be performed on the image of the human eye, such as zooming in and flipping.

The operation module 22 is used to operate the terminal according to the human eye pictures collected by the collection module 21 .

In one embodiment, the operation module 22 may include a coordinate point calculation module 221, a coordinate point analysis module 222 and an operation sub-module 223, wherein the coordinate point calculation module 221 is used to calculate the current interface according to the human eye pictures collected by the collection module 21 The coordinate point that the human eye is watching; the coordinate point analysis module 222 is used to calculate the duration that the human eye is watching the same coordinate point, and/or calculate the change track and rate of change of the coordinate point that the human eye is watching in the preset time; operation The sub-module 223 is configured to operate the terminal according to the duration of human eyes gazing at the same coordinate point, and/or according to the change trajectory and rate of change of the human eyes gazing at the coordinate point.

The operation sub-module 223 may further include a comparison module 223a and a first execution module 223b, wherein the comparison module 223a is used to compare the duration of people's gaze on the same coordinate point with a preset threshold; When the comparison result in 223a is that the duration is greater than the preset threshold, the operation corresponding to the coordinate point is executed. For example: enlarge the display scale, reduce the display scale, return to the previous page and/or enter the next page. Preferably, the first execution module 223b can specifically be used to query the preset operation instructions associated with the coordinate point that the human eye continues to fixate on when the comparison result of the comparison module 223a is that the duration is greater than the preset threshold, and then according to the query Operate the given operating instructions.

The operation sub-module 223 may further include a second execution module 223c, and the second execution module 223c is used to adopt the same change trajectory and the same change rate as the coordinate point that the human eye is gazing at, and execute the current interface displayed on the terminal display screen. Adjustment; or query the preset operation instructions associated with the coordinate point of the human eye according to the change track and change rate, and operate according to the query operation instruction.

The terminal operation method and the terminal provided by the present invention collect the human eye picture of the operator in front of the terminal display screen, operate the terminal according to the collected human eye picture, realize the human-computer interaction on the visual effect, and make the terminal display screen The display page is adjusted according to the changes of human eyes, which greatly improves the user experience.

The above content is a further detailed description of the present invention in conjunction with specific embodiments, and it cannot be assumed that the specific implementation of the present invention is limited to these descriptions. For those of ordinary skill in the technical field of the present invention, without departing from the concept of the present invention, some simple deduction or replacement can be made, which should be regarded as belonging to the protection scope of the present invention.

Claims (13)

1. A terminal operation method, characterized in that, comprising: Collect the human eye pictures of the operator in front of the terminal display screen; Operate the terminal according to the collected human eye pictures. 2. The terminal operation method according to claim 1, wherein the method of collecting the human eye picture of the operator in front of the terminal display screen is specifically: collecting the current interface displayed on the terminal display screen at least one eye of the operator. Imaging on the retina. 3. The terminal operation method according to claim 1, wherein the method for operating the terminal according to the collected human eye pictures is specifically: Calculate the coordinate points that the human eyes are looking at on the current interface according to the collected human eye pictures; Calculate the duration of human eyes gazing at the same coordinate point, and operate the terminal according to the duration of human eyes gazing at the same coordinate point; The change track and change rate of the coordinate point that human eyes are looking at operate the terminal. 4. The terminal operation method according to claim 3, characterized in that, the method of operating the terminal according to the duration of human eyes gazing at the same coordinate point is specifically: comparing the duration of human eyes gazing at the same coordinate point with a preset threshold For comparison, if the duration is greater than the preset threshold, the operation corresponding to the coordinate point is executed. 5. The terminal operation method according to claim 4, wherein the process of executing the operation corresponding to the coordinate point is specifically: querying the preset operation instruction associated with the coordinate point according to the coordinate point, and according to the query operation instructions to operate. 6. The terminal operation method according to claim 4 or 5, wherein the operation corresponding to the coordinate point comprises: zooming in on the display scale, reducing the display scale, returning to the previous page and/or entering the next page. 7. The terminal operation method according to claim 3, characterized in that, the method of operating the terminal according to the change trajectory and rate of change of the coordinate point that the human eye is gazing at is as follows: adopting the same method as the coordinate point that the human eye is gazing at Adjust the current interface displayed on the terminal display screen at the same change trajectory and the same rate of change; or query the preset operating instructions associated with it according to the change trajectory and change rate of the coordinate point that the human eye is looking at, and according to the queried Operation instructions to operate. 8. A terminal, characterized in that it includes an acquisition module and an operation module, wherein, The collection module is used to collect the human eye picture of the operator in front of the terminal display screen; The operation module is used to operate the terminal according to the human eye pictures collected by the collection module. 9. The terminal according to claim 8, wherein the acquisition module is configured to acquire an image of the current interface displayed on the display screen of the terminal on the retina of at least one eye of the operator. 10. The terminal according to claim 8, wherein the operation module comprises a coordinate point calculation module, a coordinate point analysis module and an operation submodule, wherein, The coordinate point calculation module is used to calculate the coordinate points that the human eyes are looking at on the current interface according to the collected human eye pictures; The coordinate point analysis module is used to calculate the duration of human eyes gazing at the same coordinate point, and/or calculate the change trajectory and rate of change of the coordinate point that human eyes are gazing at within a preset time; The operation sub-module is used to operate the terminal according to the duration of human eyes gazing at the same coordinate point, and/or according to the change track and rate of change of the human eyes gazing at the coordinate point. 11. The terminal according to claim 10, wherein the operation sub-module comprises a comparison module and a first execution module, wherein, The comparison module is used to compare the duration of human eyes staring at the same coordinate point with a preset threshold; The first execution module is configured to execute an operation corresponding to the coordinate point when the comparison result of the comparison module is that the duration is greater than a preset threshold. 12 . The terminal according to claim 11 , wherein the first execution module is configured to query a preset operation instruction associated with the coordinate point according to the coordinate point, and perform an operation according to the queried operation instruction. 13 . 13. The terminal according to claim 10, wherein the operation sub-module includes a second execution module, the second execution module is used to take the same change trajectory and the same change rate as the coordinate point that the human eye is looking at, for Adjust the current interface displayed on the terminal display screen; or query the preset operation instructions associated with the coordinate point according to the change track and change rate of the coordinate point that the human eye is looking at, and operate according to the queried operation instruction.

Images