CN111198644A

CN111198644A - Method and system for identifying screen operation of intelligent terminal

Info

Publication number: CN111198644A
Application number: CN201811367066.9A
Authority: CN
Inventors: 冯文康
Original assignee: Xian Yep Telecommunication Technology Co Ltd
Current assignee: Xian Yep Telecommunication Technology Co Ltd
Priority date: 2018-11-16
Filing date: 2018-11-16
Publication date: 2020-05-26
Anticipated expiration: 2038-11-16
Also published as: CN111198644B

Abstract

The invention discloses a method and a system for identifying screen operation of an intelligent terminal, wherein a plurality of distance detection modules are arranged below a screen of the intelligent terminal, and the identification method comprises the following steps: acquiring the corresponding relation between each distance detection module and each coordinate information on the screen; when at least one target object is subjected to the air separation operation, different target objects are detected through different distance detection modules, and distance information between each target object and a screen is respectively acquired; target coordinate information of the target object on the screen, which is detected by the distance detection module, corresponds to the coordinate information corresponding to the distance detection module one by one; and identifying the operation behavior of the space operation of each target object on the screen according to the target coordinate information and the distance information. According to the invention, the screen can be directly operated in the vertical direction and the horizontal direction in the air-separating operation scene, gesture actions do not need to be preset, the operation is simple, the practicability is strong, and the user experience is improved.

Description

Method and system for identifying screen operation of intelligent terminal

Technical Field

The invention relates to the technical field of mobile terminals, in particular to a method and a system for identifying screen operation of an intelligent terminal.

Background

At present, the screen operation can be performed in a mode of touch operation and in a mode of space gesture operation. However, the existing method for operating the screen through the air gesture has the following defects: 1) generally, only the gesture motion which is parallel to the screen and is operated up and down, left and right can be recognized, so that the operability is simple and limited for users, and the practicability is not strong; 2) even if the opening, closing, uninstalling, installing and the like of the application program can be realized, the operation requires the user to preset gesture actions in advance, for example, the preset gesture "O" corresponds to an opening operation, the preset gesture "C" corresponds to a closing operation and the like, so that the operation is complex and the user experience is poor for the user.

Disclosure of Invention

The invention aims to solve the technical problems that in the prior art, operation corresponding to gesture actions in screen operation through an air-isolated gesture needs to be preset, and the defects of complex operation, low practicability, poor user experience and the like exist, and the method and the system for identifying the screen operation of the intelligent terminal are provided.

The invention solves the technical problems through the following technical scheme:

the invention provides a method for identifying screen operation of an intelligent terminal, wherein a plurality of distance detection modules are arranged below a screen of the intelligent terminal, and the method comprises the following steps:

acquiring the corresponding relation between each distance detection module and each coordinate information on the screen;

when at least one target object is subjected to the air separation operation, different target objects are detected through different distance detection modules, and distance information between each target object and the screen is respectively acquired;

target coordinate information of the target object on the screen, which is detected by the distance detection module, corresponds to the coordinate information corresponding to the distance detection module one by one;

and identifying the operation behavior of the air-spaced operation of each target object on the screen according to the target coordinate information and the distance information.

Optionally, the distance information includes vertical distance information between each of the target objects and the screen;

the step of identifying the operation behavior of the space operation of each target object on the screen according to the target coordinate information and the distance information comprises the following steps:

and when the target coordinate information is kept unchanged within a first set time length and the vertical distance information is changed within the first set time length for at least one time, wherein the change is increased after being decreased, determining that the operation behavior comprises the click operation.

Optionally, when the target coordinate information is kept unchanged within the first set time length and the vertical distance information is changed within the first set time length, wherein the change is decreased first and then increased first, it is determined that the operation behavior comprises a single-click operation;

and when the target coordinate information is kept unchanged within the first set time length and the vertical distance information is changed twice within the first set time length, wherein the change is increased after being reduced, determining that the operation behavior comprises double-click operation.

the step of identifying the operation line of the space operation on the screen of each target object according to the target coordinate information and the distance information comprises the following steps:

and when the target coordinate information is kept unchanged in a second set time length, the vertical distance information is changed in a reduced mode in the second set time length, and the reduced distance is kept unchanged in a third set time length, determining that the operation behavior comprises long-press operation.

and when the change range of the vertical distance information is within the set range and the target coordinate information is changed within a fourth set time length, determining that the operation behavior comprises sliding operation.

Optionally, the step of identifying the operation behavior of the space operation of each target object on the screen according to the target coordinate information and the distance information includes:

judging whether the target coordinate information and the distance information meet identification conditions or not, and if so, executing the step of identifying the operation behavior of the air-break operation of each target object on the screen according to the target coordinate information and the distance information;

wherein the identification condition includes that the target coordinate information and the distance information are kept unchanged within a fifth set time length.

The invention also provides a system for identifying the screen operation of the intelligent terminal, wherein a plurality of distance detection modules are arranged below the screen of the intelligent terminal, and the identification system comprises a corresponding relation acquisition module, an information acquisition module and an identification module;

the corresponding relation acquisition module is used for acquiring the corresponding relation between each distance detection module and each coordinate information on the screen;

the information acquisition module is used for detecting different target objects through different distance detection modules and acquiring distance information between each target object and the screen when at least one target object performs the air separation operation;

the identification module is used for identifying the operation behavior of the air-spaced operation of each target object on the screen according to the target coordinate information and the distance information.

the identification module is used for determining that the operation behavior comprises the click operation when the target coordinate information is kept unchanged within a first set time length and the vertical distance information is changed within the first set time length for at least one time, wherein the change is decreased firstly and then increased secondly.

Optionally, the identification module is configured to determine that the operation behavior includes a single-click operation when the target coordinate information remains unchanged for the first set time period and the vertical distance information changes from decreasing to increasing once for the first set time period;

the identification module is further configured to determine that the operation behavior includes a double-click operation when the target coordinate information remains unchanged for the first set time period and the vertical distance information changes twice, namely, decreases and then increases within the first set time period.

the identification module is used for determining that the operation behavior comprises long press operation when the target coordinate information is kept unchanged in a second set time length, the vertical distance information is changed in a reduced mode in the second set time length, and the reduced distance is protected and unchanged in a third set time length.

the identification module is used for determining that the operation behavior comprises sliding operation when the change range of the vertical distance information is within the set range and the target coordinate information changes within a fourth set duration.

Optionally, the identification system further comprises a judging module;

the judging module is used for judging whether the target coordinate information and the distance information meet identification conditions or not, and if so, calling the identification module;

The positive progress effects of the invention are as follows:

according to the invention, a plurality of distance detection modules are arranged below a screen of the intelligent terminal, and different target objects are respectively detected and corresponding distance information is obtained through the corresponding detection modules when the air separation operation is carried out by presetting the corresponding relation between each detection module and each coordinate information on the screen; the target coordinate information of the target object on the screen is coordinate information corresponding to a detection module for detecting the target object; and finally, identifying the operation behavior of the air-separating operation of each target object on the screen according to the target coordinate information and the distance information, so that the screen is directly operated in the vertical direction and the horizontal direction (including clicking, long pressing, sliding and the like) in an air-separating operation scene, the gesture action does not need to be preset, the operation is simple, the practicability is high, and the user experience is improved.

Drawings

Fig. 1 is a flowchart of a method for identifying a screen operation of an intelligent terminal according to embodiment 1 of the present invention.

Fig. 2 is a screen display diagram illustrating a method for identifying a screen operation of an intelligent terminal according to embodiment 1 of the present invention.

Fig. 3 is a flowchart of a method for identifying a screen operation of an intelligent terminal according to embodiment 2 of the present invention.

Fig. 4 is a first screen display diagram of a method for identifying screen operations of an intelligent terminal according to embodiment 2 of the present invention.

Fig. 5 is a second screen display diagram illustrating a method for identifying a screen operation of an intelligent terminal according to embodiment 2 of the present invention.

Fig. 6 is a third screen display diagram illustrating a method for identifying a screen operation of an intelligent terminal according to embodiment 2 of the present invention.

Fig. 7 is a fourth screen display diagram illustrating a method for identifying a screen operation of an intelligent terminal according to embodiment 2 of the present invention.

Fig. 8 is a fifth screen display diagram illustrating a method for identifying a screen operation of an intelligent terminal according to embodiment 2 of the present invention.

Fig. 9 is a sixth screen display diagram illustrating a method for identifying a screen operation of an intelligent terminal according to embodiment 2 of the present invention.

Fig. 10 is a seventh screen display diagram illustrating a method for identifying a screen operation of an intelligent terminal according to embodiment 2 of the present invention.

Fig. 11 is a flowchart of a method for identifying a screen operation of an intelligent terminal according to embodiment 3 of the present invention.

Fig. 12 is a flowchart of a method for identifying a screen operation of an intelligent terminal according to embodiment 4 of the present invention.

Fig. 13 is a first screen display diagram illustrating a method for identifying a screen operation of an intelligent terminal according to embodiment 4 of the present invention.

Fig. 14 is a second screen display diagram illustrating a method for identifying a screen operation of an intelligent terminal according to embodiment 4 of the present invention.

Fig. 15 is a third screen display diagram illustrating a method for identifying a screen operation of an intelligent terminal according to embodiment 4 of the present invention.

Fig. 16 is a fourth screen display diagram illustrating a method for identifying a screen operation of an intelligent terminal according to embodiment 4 of the present invention.

Fig. 17 is a fifth screen display diagram illustrating a method for identifying a screen operation of an intelligent terminal according to embodiment 4 of the present invention.

Fig. 18 is a sixth screen display diagram illustrating a method for identifying a screen operation of an intelligent terminal according to embodiment 4 of the present invention.

Fig. 19 is a seventh screen display diagram illustrating a method for identifying a screen operation of an intelligent terminal according to embodiment 4 of the present invention.

Fig. 20 is an eighth screen display schematic diagram of a method for identifying a screen operation of an intelligent terminal according to embodiment 4 of the present invention.

Fig. 21 is a first screen display diagram illustrating a method for identifying a screen operation of an intelligent terminal according to embodiment 5 of the present invention.

Fig. 22 is a second screen display diagram illustrating a method for identifying a screen operation of an intelligent terminal according to embodiment 5 of the present invention.

Fig. 23 is a schematic block diagram of a screen operation recognition system of an intelligent terminal according to embodiment 6 of the present invention.

Fig. 24 is a block diagram of a screen operation recognition system of an intelligent terminal according to embodiment 7 of the present invention.

Detailed Description

The invention is further illustrated by the following examples, which are not intended to limit the scope of the invention.

Example 1

In this embodiment, a plurality of distance detection modules are arranged below the screen of the intelligent terminal.

According to actual requirements, a plurality of distance detection modules are arranged below the whole screen in a full-screen mode, so that the accuracy of the spaced gesture operation for identifying target objects (such as fingers) is guaranteed.

The distance detection module includes a distance sensor, but is not limited to a distance sensor, and may also include any other device that can acquire distance information between each target object and the screen.

The intelligent terminal comprises a mobile phone, a tablet computer and other equipment.

As shown in fig. 1, the method for identifying screen operations of an intelligent terminal of the present embodiment includes:

s101, acquiring a corresponding relation between each distance detection module and each coordinate information on a screen;

specifically, as shown in fig. 2, according to a plurality of distance detection modules arranged below the whole screen, the screen is divided into regions corresponding to each distance detection module, each region corresponds to one piece of coordinate information, and if the upper left corner of the screen is taken as the origin coordinate information (x0, y0), the corresponding coordinate information (such as application programs APP1, APP2, APP3, APP4) at any position in the screen can also be determined.

S102, when at least one target object is subjected to the air separation operation, different target objects are detected through different distance detection modules, and distance information between each target object and a screen is respectively acquired;

target coordinate information of a target object on a screen, which is detected by the distance detection module, corresponds to coordinate information corresponding to the distance detection module one to one;

specifically, when a single finger or a plurality of fingers of a user are in idle operation with respect to the screen, different distance detection modules detect different fingers and acquire distance information between each finger and the screen, and since the distance detection modules acquiring the distance information between the fingers and the screen have a one-to-one correspondence with each coordinate information on the screen, a specific position (projection point) of the finger projected on the screen can be marked by lighting (such as a bright spot of a set pattern, a cross mark and the like) the position of the coordinate information corresponding to the distance detection module for detecting the finger on the screen, so that the user can know the projection position of the finger on the screen conveniently, and the operability of the user on the mobile phone is improved.

S103, identifying the operation behavior of the air-spaced operation of each target object on the screen according to the target coordinate information and the distance information.

In the embodiment, a plurality of distance detection modules are arranged below a screen of the intelligent terminal, and different target objects are respectively detected and corresponding distance information is acquired through the corresponding detection modules when the air separation operation is performed by presetting the corresponding relation between each detection module and each coordinate information on the screen; the target coordinate information of the target object on the screen is coordinate information corresponding to a detection module for detecting the target object; and finally, the operation behavior of the air-separating operation of each target object on the screen is identified according to the target coordinate information and the distance information, so that the screen is directly operated in the vertical direction and the horizontal direction in the air-separating operation scene, the gesture action does not need to be preset, the operation is simple, the practicability is high, and the user experience is improved.

Example 2

As shown in fig. 3, the method for identifying screen operations of the intelligent terminal of this embodiment is a further improvement of embodiment 1, and specifically:

the distance information includes vertical distance information between each target object and the screen.

After step S102 and before step S103, the method further includes:

s1030, judging whether the target coordinate information and the distance information meet the identification condition, and if so, executing the step S103;

and the identification condition comprises that the target coordinate information and the distance information are kept unchanged within a fifth set time length.

When the target coordinate information and the distance information do not satisfy the recognition condition, it indicates that the operation of the target object at this time is a general movement, and there is no actual operation meaning.

Step S103 includes:

and S1031, when the target coordinate information is kept unchanged in the first set time length and the vertical distance information is changed for at least one time in the first set time length, wherein the change is decreased firstly and then increased, determining that the operation behavior comprises click operation.

Specifically, when the target coordinate information is kept unchanged within a first set time length and the vertical distance information is changed within the first set time length, wherein the change is decreased firstly and then increased, the operation behavior is determined to comprise a single-click operation;

and when the target coordinate information is kept unchanged for the first set time length and the vertical distance information is changed twice within the first set time length, wherein the change is increased after being reduced, the operation behavior is determined to comprise double-click operation.

For single click operation:

when the target coordinate information of the finger on the screen and the vertical distance information between the finger and the screen are kept unchanged within 0.2s (or more than 0.2s), marking operation behavior is started; and then, if the corresponding target coordinate information of the finger is not changed within 0.1s, the vertical distance information is changed by firstly reducing 2cm (or more than 2cm) and then increasing 2cm (or more than 2cm), the marking operation behavior is finished, and the whole operation process of the finger is defined as single click operation.

The setting of the time and the distance presented in the present application is only one possible setting manner, and the present embodiment is not limited to the above time and distance values, and may also be set according to the system performance and the user requirement.

Taking a single-finger click operation to open the application APP3 as an example, the specific identification process is as follows:

as shown in fig. 4, when the user prepares to start the clear operation with a single finger, the distance detection module disposed below the screen detects the finger, and the "cross" cursor a1 represents the corresponding projected point of the finger on the screen at this time.

As shown in fig. 5, when the user's finger starts the space operation, the distance detection module disposed below the screen detects the finger, and the "cross" cursor a2 indicates that the finger moves onto the icon of the application APP 3; if the target coordinate information of the finger of the user on the screen and the vertical distance information between the finger of the user and the screen are kept unchanged within 0.2s, marking the operation behavior to start; and then, the distance detection module continuously detects the change of the position of the finger, if the target coordinate information corresponding to the finger in 0.1s is unchanged, the vertical distance information is reduced by 2cm once, and then the change of the vertical distance information within 0.1s is increased by 2cm, the distance detection module sends the change information of the finger to a conversion module in the intelligent equipment, and the change information is converted into (0.2s, -2cm, 0.1s, +2cm, x, y), wherein (x, y) represents the target coordinate information corresponding to a single finger.

The conversion module judges that an event corresponding to the finger operation is a single-finger clicking event according to the converted change information, and sends the single-finger clicking event and the target coordinate information to a processing module of the intelligent terminal, and the processing module controls to open an application program APP3 (as shown in FIG. 6, a display interface of a homepage of an application program APP 3) according to the clicking event and the target coordinate information, and meanwhile, the clicking operation is finished; and the distance detection module continuously monitors the change condition of the target coordinate information of the finger on the screen and the distance information between the finger and the screen, and waits for the next spacing operation.

Taking a double-finger single-click operation to wake up the mobile phone as an example, the specific identification process is as follows:

as shown in fig. 7, when the mobile phone is in the sleep mode (the area a in fig. 7 represents the state of the screen at this time), when the user starts the idle operation with two fingers, the distance detection module disposed below the screen detects the fingers, the target coordinate information of the two fingers of the user on the screen and the vertical distance information between the two fingers and the screen are both kept unchanged within 0.2s, and the marking operation starts at this time; then, the distance detection module continuously detects the position change of the two fingers, if the target coordinate information corresponding to the two fingers is not changed within 0.1s, the vertical distance information is reduced by 2cm once, and then the vertical distance information is increased by 2cm within 0.1s, the distance detection module sends the change information of the two fingers to a conversion module in the intelligent device, and the change information is converted into (0.2s, -2cm, +2cm, x1, y1, x2, y2), wherein (x1, y1) and (x2, y2) respectively represent the target coordinate information corresponding to the two fingers.

The conversion module judges that an event corresponding to the finger operation is a double-finger clicking event according to the converted change information, and sends the double-finger clicking event and the target coordinate information to a processing module of the intelligent terminal, the processing module controls to wake up and light a mobile phone screen (as shown in fig. 8) according to the double-finger clicking event and the target coordinate information, and meanwhile, the clicking operation is finished; and the distance detection module continuously monitors the change condition of the target coordinate information of the finger on the screen and the distance information between the finger and the screen, and waits for the next spacing operation.

Taking the three-finger clicking operation to quickly open the camera function of the mobile phone as an example, the specific identification process is as follows:

when a user starts the space operation by using three fingers, a distance detection module arranged below a screen detects the fingers, the target coordinate information of the three fingers of the user on the screen and the distance information between the three fingers and the screen are kept unchanged within 0.2s, and the marking operation behavior starts at the moment; then, the distance detection module continuously detects the position change of the three fingers, if the target coordinate information corresponding to the three fingers is not changed within 0.1s, the vertical distance information is reduced by 2cm once, and then the vertical distance information within 0.1s is increased by 2cm, the distance detection module sends the change information of the fingers to a conversion module in the intelligent device, and the change information is converted into (0.2s, -2cm, 0.1s, +2cm, x1, y1, x2, y2, x3, y3), wherein (x1, y1), (x2, y2) and (x3, y3) respectively represent the target coordinate information corresponding to the three fingers.

The conversion module judges that an event corresponding to the finger operation is a three-finger clicking event according to the converted change information, and sends the three-finger clicking event and the target coordinate information to a processing module of the intelligent terminal, the processing module controls to open a mobile phone camera according to the three-finger clicking event and the target coordinate information, and meanwhile, the clicking operation is finished; and the distance detection module continuously monitors the change condition of the target coordinate information of the finger on the screen and the distance information between the finger and the screen, and waits for the next spacing operation.

Taking a four-finger click operation to quickly open a Wi-Fi (wireless local area network) function as an example, the specific identification process is as follows:

when a user starts the space operation by adopting four fingers, a distance detection module arranged below a screen detects the fingers, the target coordinate information of the four fingers of the user on the screen and the distance information between the four fingers and the screen are kept unchanged within 0.2s, and the marking operation behavior starts at the moment; then, the distance detection module continuously detects the position change of the four fingers, if the target coordinate information corresponding to the four fingers is not changed within 0.1s, the vertical distance information is reduced by 2cm once, and then the vertical distance information within 0.1s is increased by 2cm, the distance detection module sends the change information of the fingers to a conversion module in the intelligent device, and the change information is converted into (0.2s, -2cm, 0.1s, +2cm, x1, y1, x2, y2, x3, y3, x4, y4), wherein (x1, y1), (x2, y2), (x3, y3) and (x4, y4) respectively represent the target coordinate information corresponding to the four fingers.

The conversion module judges that an event corresponding to the finger operation is a four-finger clicking event according to the converted change information, and sends the four-finger clicking event and the target coordinate information to a processing module of the intelligent terminal, the processing module controls the opening of a Wi-Fi function according to the four-finger clicking event and the target coordinate information, and meanwhile, the clicking operation is finished; and the distance detection module continuously monitors the change condition of the target coordinate information of the finger on the screen and the distance information between the finger and the screen, and waits for the next spacing operation.

Taking the example of quickly opening the bluetooth function by five-finger clicking operation, the specific identification process is as follows:

when a user starts the spaced operation by using five fingers, a distance detection module arranged below a screen detects the fingers, the target coordinate information of the five fingers of the user on the screen and the distance information between the five fingers and the screen are kept unchanged within 0.2s, and the marking operation behavior starts at the moment; then, the distance detection module continuously detects the change of the position of the five fingers, if the target coordinate information corresponding to the five fingers is not changed within 0.1s, the vertical distance information is reduced by 2cm once, and then the vertical distance information within 0.1s is increased by 2cm, the distance detection module sends the change information of the fingers to a conversion module in the intelligent device, and the change information is converted into (0.2s, -2cm, 0.1s, +2cm, x1, y1, x2, y2, x3, y3, x4, y4, x5, and y5), wherein (x1, y1), (x2, y2), (x3, y3), (x4, y4) and (x5, y5) respectively represent the target coordinate information corresponding to the five fingers.

The conversion module judges that an event corresponding to the finger operation is a five-finger clicking event according to the converted change information, and sends the five-finger clicking event and the target coordinate information to a processing module of the intelligent terminal, the processing module controls the Bluetooth to be opened according to the five-finger clicking event and the target coordinate information, and meanwhile, the clicking operation is finished; and the distance detection module continuously monitors the change condition of the target coordinate information of the finger on the screen and the distance information between the finger and the screen, and waits for the next spacing operation.

For a double click operation:

when the target coordinate information of the finger on the screen and the vertical distance information between the finger and the screen are kept unchanged within 0.2s (or more than 0.2s), marking operation behavior is started; and then, if the corresponding target coordinate information of the finger is not changed within 0.1s, the vertical distance information completes the change of firstly reducing 2cm (or more than 2cm) and then increasing 2cm (or more than 2cm) once, and then completes the change of firstly reducing 2cm (or more than 2cm) and then increasing 2cm (or more than 2cm) once within 0.1s, at the moment, the marking operation behavior is ended, and the whole operation process of the finger is defined as double-click operation.

Taking a single-finger double-click quick-delete application APP1 as an example, the specific identification process is as follows:

as shown in fig. 9, when the user prepares to start the clear operation with a single finger, the distance detection module disposed below the screen detects the finger, and the "cross" cursor b1 indicates that the single finger is moved onto the icon of the application APP 1; when the target coordinate information of a single finger of a user on the screen and the distance information between the single finger of the user and the screen are kept unchanged within 0.2s, marking operation behavior is started; then, the distance detection module continuously detects the change of the position of the finger, and if the corresponding target coordinate information of the finger is not changed within 0.1s, the vertical distance information is changed by firstly reducing 2cm and then increasing 2cm, and then the change of firstly reducing 2cm and then increasing 2cm is completed within 0.1s, the distance detection module sends the change information of the finger to a conversion module in the intelligent device, and the change information is converted into (0.2s, -2cm, 0.1s, +2cm, 0.1s, -2cm, +2cm, x1, y1), wherein (x1, y1) represents the target coordinate information corresponding to a single finger.

The conversion module judges that an event corresponding to the finger operation is a single-finger double-click event according to the converted change information, and sends the single-finger double-click event and the target coordinate information to a processing module of the intelligent terminal, and the processing module controls and deletes an application program APP1 according to the single-click event and the target coordinate information; as shown in fig. 10, for a popup interface appearing after double-clicking on the application APP1, the user is given a second determination whether to delete the application APP1, and the click operation is ended; and the distance detection module continuously monitors the change condition of the target coordinate information of the finger on the screen and the distance information between the finger and the screen, and waits for the next spacing operation.

Taking a double-finger double-click quick-entry telephone dialing interface as an example, the specific identification process is as follows:

when a user adopts double fingers to prepare for starting the spaced operation, a distance detection module arranged below a screen detects the fingers, and when the target coordinate information of the double fingers of the user on the screen and the distance information between the double fingers and the screen are kept unchanged within 0.2s, the marking operation behavior starts; then, the distance detection module continuously detects the change of the position of the finger, and if the target coordinate information corresponding to the finger is not changed within 0.1s, the vertical distance information is changed by firstly decreasing by 2cm and then increasing by 2cm, and then the change by firstly decreasing by 2cm and then increasing by 2cm is completed within 0.1s, the distance detection module sends the change information of the finger to a conversion module in the intelligent device, and the change information is converted into (0.2s, -2cm, 0.1s, +2cm, 0.1s, -2cm, +2cm, x1, y1, x2, y2), wherein (x1, y1) and (x2, y2) respectively represent the target coordinate information corresponding to the two fingers.

The conversion module judges whether an event corresponding to finger operation is a double-finger double-click event according to the converted change information, and sends the double-finger double-click event and target coordinate information to a processing module of the intelligent terminal, and the processing module controls the access to a telephone dialing interface according to the single-click event and the target coordinate information, so that a user can conveniently dial quickly; at the same time, the click operation is finished; and the distance detection module continuously monitors the change condition of the target coordinate information of the finger on the screen and the distance information between the finger and the screen, and waits for the next spacing operation.

Taking the three-finger double click to realize the quick screen capture function as an example, the specific identification process is as follows:

when a user adopts three fingers to prepare for starting the spaced operation, a distance detection module arranged below a screen detects the fingers, the target coordinate information of the three fingers of the user on the screen and the distance information between the three fingers and the screen are kept unchanged within 0.2s, and the marking operation behavior starts at the moment; then, the distance detection module continuously detects the change of the position of the finger, and if the target coordinate information corresponding to the finger is not changed within 0.1s, the vertical distance information is changed by firstly decreasing by 2cm and then increasing by 2cm, and then the change by firstly decreasing by 2cm and then increasing by 2cm is completed within 0.1s, the distance detection module sends the change information of the finger to a conversion module in the intelligent device, and the change information is converted into (0.2s, -2cm, 0.1s, +2cm, 0.1s, -2cm, +2cm, x1, y1, x2, y2, x3, y3), wherein (x1, y1), (x2, y2) and (x3, y3) respectively represent the target coordinate information corresponding to three fingers.

The conversion module judges that an event corresponding to the finger operation is a three-finger double-click event according to the converted change information, and sends the three-finger double-click event and the target coordinate information to a processing module of the intelligent terminal, and the processing module controls to perform screen capture operation according to the single-click event and the target coordinate information; at the same time, the click operation is finished; and the distance detection module continuously monitors the change condition of the target coordinate information of the finger on the screen and the distance information between the finger and the screen, and waits for the next spacing operation.

Taking the example of realizing the function of rapidly opening the recording by double clicking with four fingers, the specific identification process is as follows:

when a user adopts four fingers to prepare to start the spaced operation, a distance detection module arranged below a screen detects the fingers, and when target coordinate information of the four fingers of the user on the screen and distance information between the four fingers and the screen are kept unchanged within 0.2s, the marking operation behavior starts; then, the distance detection module continuously detects the change of the position of the finger, and if the target coordinate information corresponding to the finger is not changed within 0.1s, the vertical distance information is changed by firstly decreasing by 2cm and then increasing by 2cm, and then the change by firstly decreasing by 2cm and then increasing by 2cm is completed within 0.1s, the distance detection module sends the change information of the finger to a conversion module in the intelligent device, and the change information is converted into (0.2s, -2cm, 0.1s, +2cm, 0.1s, -2cm, +2cm, x1, y1, x2, y2, x3, y3, x4, y4), wherein (x1, y1), (x2, y2), (x3, y3) and (x4, y4) respectively represent the target coordinate information corresponding to four fingers.

The conversion module judges that an event corresponding to the finger operation is a four-finger double-click event according to the converted change information, and sends the four-finger double-click event and the target coordinate information to the processing module of the intelligent terminal, and the processing module controls to open a recording function interface according to the single-click event and the target coordinate information, so that a user can conveniently perform recording operation; at the same time, the click operation is finished; and the distance detection module continuously monitors the change condition of the target coordinate information of the finger on the screen and the distance information between the finger and the screen, and waits for the next spacing operation.

Taking the function of quickly entering the mobile phone setting interface by double-clicking five fingers as an example, the specific identification process is as follows:

when a user adopts five fingers to prepare to start the spaced operation, a distance detection module arranged below a screen detects the fingers, and when target coordinate information of the five fingers of the user on the screen and distance information between the five fingers and the screen are kept unchanged within 0.2s, the marking operation behavior starts; then, the distance detection module continuously detects the change of the position of the finger, and if the target coordinate information corresponding to the finger is not changed within 0.1s, the vertical distance information is changed by first decreasing by 2cm and then increasing by 2cm, and then the change by first decreasing by 2cm and then increasing by 2cm is completed within 0.1s, the distance detection module sends the change information of the finger to a conversion module in the smart device, and converts the change information into (0.2s, -2cm, 0.1s, +2cm, 0.1s, -2cm, +2cm, x1, y1, x2, y2, x3, y3, x4, y4, x5, y5), wherein (x1, y1), (x2, y2), (x3, y3), (x4, y4) and (x5, y5) respectively represent the target coordinate information corresponding to the five fingers.

The conversion module judges whether the event corresponding to the finger operation is a five-finger double-click event according to the converted change information, and sends the five-finger double-click event and the target coordinate information to a processing module of the intelligent terminal, and the processing module controls the mobile phone to enter a mobile phone setting interface according to the single-click event and the target coordinate information, so that a user can conveniently set functions of the mobile phone; at the same time, the click operation is finished; and the distance detection module continuously monitors the change condition of the target coordinate information of the finger on the screen and the distance information between the finger and the screen, and waits for the next spacing operation.

In the embodiment, a plurality of distance detection modules are arranged below a screen of the intelligent terminal, and different target objects are respectively detected and corresponding distance information is acquired through the corresponding detection modules when the air separation operation is performed by presetting the corresponding relation between each detection module and each coordinate information on the screen; the target coordinate information of the target object on the screen is coordinate information corresponding to a detection module for detecting the target object; and finally, the clicking operation of the space operation of each target object on the screen is recognized according to the change conditions of the target coordinate information and the distance information within the first set time length, so that the clicking operation of the screen in the space operation scene is realized, the screen is directly clicked in the vertical direction and the horizontal direction, a mobile phone application program and the like are quickly called, gesture actions do not need to be preset, the operation is simple, the practicability is high, and the user experience is improved.

Example 3

As shown in fig. 11, the method for identifying a screen operation of an intelligent terminal of this embodiment is a further improvement of embodiment 1, and specifically includes:

the distance information includes vertical distance information between each target object and the screen;

after step S102 and before step S103, the method further includes:

s1030, judging whether the target coordinate information and the distance information meet the identification conditions, and if so, identifying the operation behavior of the space operation of each target object on the screen;

Step S103 includes:

s1032, when the target coordinate information is kept unchanged in the second set time length, the vertical distance information is changed in a reducing mode in the second set time length, and the reduced distance is kept unchanged in the third set time length, the operation behavior is determined to comprise long-press operation.

When the target coordinate information of the finger on the screen and the distance information between the finger and the screen are kept unchanged within 0.2s (or more than 0.2s), marking operation behavior is started; subsequently, if the vertical distance information between the finger and the screen is reduced by 2cm (or more than 2cm) and the reduced distance is maintained for 1s (or more than 1s), the entire operation process of the finger is defined as a long press operation.

After the blank operation of the finger is recognized as the long-press operation, if the vertical distance information between the finger and the screen changes or the target coordinate information, the long-press operation is finished.

Taking the example of starting multiple content selection functions by single finger length pressing operation, the specific identification process is as follows:

when a user adopts a single finger to prepare for starting the spaced operation, a distance detection module arranged below a screen detects the finger, when a projection point corresponding to the finger is displayed on the screen by a cross cursor, and when target coordinate information of the single finger of the user on the screen and distance information between the single finger of the user and the screen are kept unchanged within 0.2s, the marking operation behavior starts at the moment; then, the distance detection module continuously detects the change of the position of the finger, and if the vertical distance information between the finger and the screen is reduced by 2cm and the reduced distance is kept unchanged within 1s, the distance detection module sends the change information of the finger to a conversion module in the intelligent device, and converts the change information into (0.2s, -2cm,1s, x, y), wherein (x, y) represents the target coordinate information corresponding to a single finger.

The conversion module judges that an event corresponding to the finger operation is a single-finger long-time pressing event according to the converted change information, and sends the single-finger long-time pressing event and the target coordinate information to the processing module of the intelligent terminal, and the processing module controls and starts a multi-item content selection function according to the single-click event and the target coordinate information, so that a user can conveniently perform multi-item selection; meanwhile, the single finger long pressing operation is finished; and the distance detection module continuously monitors the change condition of the target coordinate information of the finger on the screen and the distance information between the finger and the screen, and waits for the next spacing operation.

By analogy, for other multi-finger long-press operations such as a two-finger long-press operation, a three-finger long-press operation, and the like, each long-press operation corresponds to one execution operation, and the specific identification process is the identification process of the single-finger long-press operation as described above, and therefore, the description is omitted here.

In the embodiment, a plurality of distance detection modules are arranged below a screen of the intelligent terminal, and different target objects are respectively detected and corresponding distance information is acquired through the corresponding detection modules when the air separation operation is performed by presetting the corresponding relation between each detection module and each coordinate information on the screen; the target coordinate information of the target object on the screen is coordinate information corresponding to a detection module for detecting the target object; and finally, the long-press operation of the space-between operation of each target object on the screen is recognized according to the change conditions of the target coordinate information and the distance information in the second set duration, so that the long-press operation of the screen in the space-between operation scene is realized, the screen is directly operated in the vertical direction and the horizontal direction, the gesture action does not need to be preset, the operation is simple, the practicability is high, and the user experience is improved.

Example 4

As shown in fig. 12, the method for identifying screen operations of the intelligent terminal of the present embodiment is a further improvement of embodiment 1, and specifically:

after step S102 and before step S103, the method further includes:

Step S103 includes:

and S1033, when the change range of the vertical distance information is within the set range and the target coordinate information is changed in the fourth set time length, determining that the operation behavior comprises a sliding operation.

When the target coordinate information of the finger on the screen and the distance information between the finger and the screen are kept unchanged within 0.2s (or more than 0.2s), marking operation behavior is started; subsequently, if the vertical distance information between the finger and the screen is changed in the range of ± -0.5cm and the target coordinate information (i.e., the projection point of the finger on the screen) is changed, the whole operation process of the finger is defined as a sliding operation.

After the blank operation of the finger is identified as the sliding operation, if the finger moves out of the screen (i.e. there is no corresponding projection point on the screen), or the finger stays for more than 0.2s at a certain position after moving for a period of time, or the vertical distance information between the finger and the screen changes, the sliding operation is finished.

When the sliding operation is performed by using the two fingers, the operation of enlarging the screen display by using the two fingers and the operation of reducing the screen display by using the two fingers are specifically as follows:

when the target coordinate information of the finger on the screen and the distance information between the finger and the screen are kept unchanged within 0.2s (or more than 0.2s), marking operation behavior is started; subsequently, if the distance between the two fingers is enlarged, the operation of the fingers is positioned as a two-finger enlarging operation;

after the operation of the finger is positioned as the double-finger enlarging operation, if the stay time of the double fingers at the current position exceeds 0.2s or the vertical distance information between the finger and the screen changes, the double-finger enlarging operation is finished.

When the target coordinate information of the finger on the screen and the distance information between the finger and the screen are kept unchanged within 0.2s (or more than 0.2s), marking operation behavior is started; then, the distance between the two fingers is reduced, and the operation of the fingers is positioned as the double-finger reduction operation;

after the operation of the finger is positioned as the double-finger zoom-out operation, if the stay time of the double fingers at the current position exceeds 0.2s or the vertical distance information between the finger and the screen changes, the double-finger zoom-out operation is finished.

Taking the single-finger sliding operation to realize the mobile phone unlocking function as an example, the specific identification process is as follows:

as shown in fig. 13, when the mobile phone enters the unlocking interface, an unlocking operation needs to be performed according to a gesture password preset by the user. When a user adopts a single finger to prepare for starting the spaced operation, a distance detection module arranged below a screen detects the finger, and a projected point position corresponding to the finger is indicated by a cross cursor c1 on the screen; when the target coordinate information of a single finger of a user on the screen and the vertical distance information between the single finger of the user and the screen are kept unchanged within 0.2s, marking operation behavior is started; subsequently, the finger moves parallel to the mobile phone screen to complete a screen unlocking gesture graph preset by the user, such as "L" (as shown in fig. 14), the distance detection module sends change information of the finger to a conversion module in the smart device, and converts the change information into (0.2s, x, y, x, y + m, x + n, y + m), where (x, y) represents target coordinate information corresponding to "cross" cursor c1, (x, y + m) represents m point in fig. 14, and (x + n, y + m) represents target coordinate information corresponding to "cross" cursor c 2.

The conversion module judges that an event corresponding to the finger operation is a single-finger sliding event according to the converted change information, and sends the single-finger sliding event and the target coordinate information to a processing module of the intelligent terminal, and the processing module controls sliding unlocking according to the single-click event and the target coordinate information.

When the finger slides to a point 'cross' cursor c2 at the end, the vertical distance information between the finger and the screen changes, or the stay time at the point exceeds 0.2s, the single-finger sliding operation is considered to be ended; and the distance detection module continuously monitors the change condition of the target coordinate information of the finger on the screen and the distance information between the finger and the screen, and waits for the next spacing operation.

Taking the example of implementing image amplification by double-finger expansion sliding operation, the specific identification process is as follows:

as shown in fig. 15, when the user uses two fingers to prepare for starting the blank operation, the distance detection module disposed below the screen detects the fingers, the positions of the projection points corresponding to the two fingers are respectively represented by "cross" cursors d1 and d2 on the screen, wherein the target coordinate information corresponding to d1 and d2 is (x1, y1) and (x2, y2), and the target coordinate information of the two fingers of the user on the screen and the vertical distance information between the two fingers and the screen are kept unchanged within 0.2s, at which time the marking operation starts; subsequently, when the two fingers move back and forth parallel to the mobile phone screen, the distance detection module sends the change information of the fingers to the conversion module in the intelligent device, and converts the change information into (0.2s, x1-n1, y1+ m1, x2+ n2, y2-m2), wherein (x1-n1, y1+ m1) represents the corresponding target coordinate information after d1 moves, and (x2+ n2, y2-m2) represents the corresponding target coordinate information after d2 moves, as shown in fig. 16.

The conversion module judges that the event corresponding to the finger operation is a double-finger expansion sliding event according to the converted change information, and sends the double-finger expansion sliding event and the target coordinate information to the processing module of the intelligent terminal, and the processing module controls the image to be amplified according to the single-click event and the target coordinate information. When the vertical distance information between the finger and the screen changes or the stay time at a certain point exceeds 0.2s, the double-finger expansion sliding operation is considered to be finished; and the distance detection module continuously monitors the change condition of the target coordinate information of the finger on the screen and the distance information between the finger and the screen, and waits for the next spacing operation.

Taking the two-finger zoom-out sliding operation to realize image zoom-out as an example, the specific identification process is as follows:

as shown in fig. 17, when the user uses two fingers to prepare for starting the blank operation, the distance detection module disposed below the screen detects the fingers, the positions of the projection points corresponding to the two fingers are respectively represented by "cross" cursors e1 and e2 on the screen, wherein the target coordinate information corresponding to e1 and e2 is (x1, y1) and (x2, y2), and the target coordinate information of the two fingers of the user on the screen and the vertical distance information between the two fingers and the screen are kept unchanged within 0.2s, at which time the marking operation starts; subsequently, when the two fingers move back and forth parallel to the mobile phone screen, the distance detection module sends the change information of the fingers to the conversion module in the intelligent device, and converts the change information into (0.2s, x1-n1, y1-m1, x2+ n2, y2+ m2), wherein as shown in fig. 18, (x1-n1, y1-m1) represents the corresponding target coordinate information after the e1 moves, (x2+ n2, y2+ m2)) represents the corresponding target coordinate information after the e2 moves.

The conversion module judges that the event corresponding to the finger operation is a double-finger zooming-out sliding event according to the converted change information, and sends the double-finger zooming-out sliding event and the target coordinate information to a processing module of the intelligent terminal, and the processing module controls zooming-out images according to the clicking event and the target coordinate information. When the vertical distance information between the finger and the screen changes or the stay time at a certain point exceeds 0.2s, the double-finger expansion sliding operation is considered to be finished; and the distance detection module continuously monitors the change condition of the target coordinate information of the finger on the screen and the distance information between the finger and the screen, and waits for the next spacing operation.

Taking the three-finger sliding operation to realize the volume adjustment of the mobile phone as an example, the specific identification process is as follows:

as shown in fig. 19, when the user prepares to start the blank operation with three fingers, the distance detection module disposed below the screen detects the fingers, and the positions of the projection points corresponding to the two fingers are respectively represented by "cross" cursors f1, f2, f3 on the screen, where the target coordinate information corresponding to f1, f2, f3 are (x1, y1), (x2, y2) and (x3, y3), respectively. When the target coordinate information of a single finger of a user on the screen and the distance information between the single finger of the user and the screen are kept unchanged within 0.2s, marking operation behavior is started; subsequently, when the finger moves parallel to the mobile phone screen, the distance detection module sends the change information of the finger to the conversion module in the smart device, and converts the change information into (0.2s, x1+ n1, y1+ m1, x2+ n2, y2+ m2, x3+ n3, y3+ m3), wherein, as shown in fig. 20, (x1+ n1, y1+ m1) represents the corresponding target coordinate information after f1 moves, (x2+ n2, y2+ m2) represents the corresponding target coordinate information after f2 moves, and (x3+ n3, y3+ m3) represents the corresponding target coordinate information after f3 moves.

The conversion module judges that an event corresponding to finger operation is a three-finger sliding event according to the converted change information, and sends the three-finger sliding event and the target coordinate information to a processing module of the intelligent terminal, and the processing module controls and adjusts the volume of the mobile phone according to the clicking event and the target coordinate information.

By analogy, for other multi-finger sliding operations such as four-finger sliding operation, five-finger sliding operation, and the like, each sliding operation corresponds to one execution operation, and the specific identification process is the identification process of the single-finger, double-finger, and three-finger long press operation as described above, and therefore, the description is omitted here.

In the embodiment, a plurality of distance detection modules are arranged below a screen of the intelligent terminal, and different target objects are respectively detected and corresponding distance information is acquired through the corresponding detection modules when the air separation operation is performed by presetting the corresponding relation between each detection module and each coordinate information on the screen; the target coordinate information of the target object on the screen is coordinate information corresponding to a detection module for detecting the target object; and finally, the sliding operation of the air-separating operation of each target object on the screen is identified according to the change conditions of the target coordinate information and the distance information in the fourth set duration, so that the screen is directly operated (including clicking, sliding and the like) in the vertical direction and the horizontal direction in the air-separating operation scene, the mobile phone application program and the like are quickly called, gesture actions do not need to be preset, the operation is simple, the practicability is high, and the user experience is improved.

Example 5

The method for identifying the screen operation of the intelligent terminal is a further improvement of the embodiment 1, and specifically comprises the following steps:

the identification method for the screen operation of the intelligent terminal in the embodiment is not only used for identifying one operation behavior, but also used for identifying a plurality of combined operation behaviors (such as any combination of clicking, long pressing and sliding operation) so as to facilitate a user to complete more operations.

Taking the deletion of the APP1 as an example, the specific identification process is as follows:

as shown in fig. 9, when the user starts the clear operation with a single finger, the distance detection module disposed below the screen detects the finger, and the "cross" cursor b1 indicates that the single finger moves onto an icon of the application APP 1; then, as shown in fig. 21, recognizing the operation of pressing the application APP1 by a single finger length, and displaying a recycle bin icon g on the display interface; subsequently, as shown in fig. 22, the single-finger sliding operation is recognized, and the application APP1 is dragged to the recycle bin icon g, so that the operation of deleting the application APP1 is completed, that is, the operation process includes both the long-press operation and the sliding operation.

In the embodiment, the identification of the spaced gesture operation in the identification method for the screen operation of the intelligent terminal is not only the identification of one operation behavior, but also the identification of various combined operation behaviors, so that the gesture operation in a spaced scene is enriched, a user can conveniently complete more operations, the practicability is high, and the user experience is improved.

Example 6

As shown in fig. 23, the identification system of screen operation of the intelligent terminal of the present embodiment includes a correspondence obtaining module 1, an information obtaining module 2, and an identification module 3.

The corresponding relation obtaining module 1 is used for obtaining the corresponding relation between each distance detection module and each coordinate information on the screen;

The information acquisition module 2 is used for detecting different target objects through different distance detection modules and acquiring distance information between each target object and a screen when at least one target object performs an air separation operation;

The identification module 3 is used for identifying the operation behavior of the space operation of each target object on the screen according to the target coordinate information and the distance information.

Example 7

As shown in fig. 24, the recognition system of the screen operation of the intelligent terminal of the present embodiment is a further improvement of embodiment 6, specifically:

the identification system further comprises a decision module 4.

The judging module 4 is used for judging whether the target coordinate information and the distance information meet the identification conditions, and if so, calling the identification module 3 to identify the operation behavior of the space operation of each target object on the screen;

The identification module 3 is configured to determine that the operation behavior includes a click operation when the target coordinate information remains unchanged for a first set time period and the vertical distance information changes at least once in the first set time period, wherein the changes are decreased and then increased.

Specifically, the identification module 3 is configured to determine that the operation behavior includes a single-click operation when the target coordinate information is kept unchanged within a first set time period and the vertical distance information changes from decreasing to increasing once within the first set time period;

the identification module 3 is further configured to determine that the operation behavior includes a double-click operation when the target coordinate information remains unchanged for a first set time period and the vertical distance information changes twice, namely, decreases and then increases within the first set time period.

For single click operation:

when the target coordinate information of the finger on the screen and the vertical distance information between the finger and the screen are kept unchanged within 0.2s (or more than 0.2s), marking operation behavior is started; and then, if the corresponding target coordinate information of the finger is not changed within 0.1s, the vertical distance information is changed by firstly reducing 2cm (or more than 2cm) and then increasing 2cm (or more than 2cm), the marking operation behavior is finished, and the whole operation process of the finger is defined as single click operation. Taking a single-finger click operation to open the application APP3 as an example, the specific identification process is as follows:

as shown in fig. 7, when the mobile phone is in the sleep mode and the user starts the spaced operation with two fingers, the distance detection module disposed below the screen detects the fingers, the target coordinate information of the two fingers of the user on the screen and the vertical distance information between the two fingers and the screen are kept unchanged within 0.2s, and at this time, the operation behavior is marked to start; then, the distance detection module continuously detects the position change of the two fingers, if the target coordinate information corresponding to the two fingers is not changed within 0.1s, the vertical distance information is reduced by 2cm once, and then the vertical distance information is increased by 2cm within 0.1s, the distance detection module sends the change information of the two fingers to a conversion module in the intelligent device, and the change information is converted into (0.2s, -2cm, +2cm, x1, y1, x2, y2), wherein (x1, y1) and (x2, y2) respectively represent the target coordinate information corresponding to the two fingers.

For a double click operation:

Example 8

the identification system further comprises a decision module 4.

The identification module 3 is configured to determine that the operation behavior includes a long press operation when the target coordinate information is kept unchanged within a second set time period, the vertical distance information is changed in a reduced manner within the second set time period, and the reduced distance is protected and unchanged within a third set time period. When the target coordinate information of the finger on the screen and the distance information between the finger and the screen are kept unchanged within 0.2s (or more than 0.2s), marking operation behavior is started; subsequently, if the vertical distance information between the finger and the screen is reduced by 2cm (or more than 2cm) and the reduced distance is maintained for 1s (or more than 1s), the entire operation process of the finger is defined as a long press operation.

In the embodiment, a plurality of distance detection modules are arranged below a screen of the intelligent terminal, and different target objects are respectively detected and corresponding distance information is acquired through the corresponding detection modules when the air separation operation is performed by presetting the corresponding relation between each detection module and each coordinate information on the screen; the target coordinate information of the target object on the screen is coordinate information corresponding to a detection module for detecting the target object; and finally, the long-press operation of the space-between operation of each target object on the screen is recognized according to the change conditions of the target coordinate information and the distance information in the second set time length, so that the long-press operation of the screen in the space-between operation scene is realized, the long-press operation is directly performed on the screen in the vertical direction and the horizontal direction, the mobile phone application program and the like are quickly called, gesture actions do not need to be preset, the operation is simple, the practicability is high, and the user experience is improved.

Example 9

the identification system further comprises a decision module 4.

The identification module 3 is configured to determine that the operation behavior includes a sliding operation when the variation range of the vertical distance information is within the set range and the target coordinate information is changed within the fourth set duration.

after the operation of the finger is positioned as the double-finger zoom-out operation, if the stay time of the double fingers at the current position exceeds 0.2s or the vertical distance information between the finger and the screen changes, the double-finger zoom-out operation is finished. Taking the single-finger sliding operation to realize the mobile phone unlocking function as an example, the specific identification process is as follows:

as shown in fig. 17, when the user uses two fingers to prepare for starting the blank operation, the distance detection module disposed below the screen detects the fingers, the positions of the projection points corresponding to the two fingers are respectively represented by "cross" cursors e1 and e2 on the screen, wherein the target coordinate information corresponding to e1 and e2 is (x1, y1) and (x2, y2), and the target coordinate information of the two fingers of the user on the screen and the vertical distance information between the two fingers and the screen are kept unchanged within 0.2s, at which time the marking operation starts; subsequently, when the two fingers move back and forth parallel to the mobile phone screen, the distance detection module sends the change information of the fingers to the conversion module in the intelligent device, and converts the change information into (0.2s, x1-n1, y1-m1, x2+ n2, y2+ m2), wherein as shown in fig. 17, (x1-n1, y1-m1) represents the corresponding target coordinate information after the e1 moves, (x2+ n2, y2+ m2)) represents the corresponding target coordinate information after the e2 moves.

Example 10

The identification system of the screen operation of the intelligent terminal of the embodiment is a further improvement of the embodiment 6, specifically:

the space gesture operation recognition in the recognition system for screen operation of the intelligent terminal in the embodiment is not only recognition of one operation behavior, but also recognition of various combined operation behaviors (such as any combination of clicking, long pressing and sliding operation) so as to facilitate a user to complete more operations.

While specific embodiments of the invention have been described above, it will be appreciated by those skilled in the art that these are by way of example only, and that the scope of the invention is defined by the appended claims. Various changes and modifications to these embodiments may be made by those skilled in the art without departing from the spirit and scope of the invention, and these changes and modifications are within the scope of the invention.

Claims

1. The method for identifying the screen operation of the intelligent terminal is characterized in that a plurality of distance detection modules are arranged below a screen of the intelligent terminal, and the method for identifying the screen operation of the intelligent terminal comprises the following steps:

2. The recognition method of a screen operation of an intelligent terminal according to claim 1, wherein the distance information includes vertical distance information between each of the target objects and the screen;

3. The method for identifying screen operations of an intelligent terminal according to claim 2, wherein when the target coordinate information is kept constant for the first set time period and the vertical distance information is changed in a first decreasing and then increasing manner within the first set time period, it is determined that the operation behavior includes a single-click operation;

4. The recognition method of a screen operation of an intelligent terminal according to claim 1, wherein the distance information includes vertical distance information between each of the target objects and the screen;

5. The recognition method of a screen operation of an intelligent terminal according to claim 1, wherein the distance information includes vertical distance information between each of the target objects and the screen;

6. The method for identifying screen operations of an intelligent terminal according to claim 1, wherein the step of identifying the operation behavior of the idle operation of each target object on the screen according to the target coordinate information and the distance information comprises:

7. The system for identifying the screen operation of the intelligent terminal is characterized in that a plurality of distance detection modules are arranged below a screen of the intelligent terminal, and the identification system comprises a corresponding relation acquisition module, an information acquisition module and an identification module;

8. The recognition system of screen operation of an intelligent terminal according to claim 7, wherein the distance information includes vertical distance information between each of the target objects and the screen;

9. The system for identifying screen operations of an intelligent terminal according to claim 8, wherein the identification module is configured to determine that the operation behavior includes a single-click operation when the target coordinate information remains unchanged for the first set time period and the vertical distance information changes in a first decreasing and then increasing manner within the first set time period;

10. The recognition system of screen operation of an intelligent terminal according to claim 7, wherein the distance information includes vertical distance information between each of the target objects and the screen;

11. The recognition system of screen operation of an intelligent terminal according to claim 7, wherein the distance information includes vertical distance information between each of the target objects and the screen;

12. The screen-operated recognition system of an intelligent terminal according to claim 7, wherein the recognition system further comprises a judgment module;