CN104462437B - The method and system of search are identified based on the multiple touch control operation of terminal interface - Google Patents

Abstract

The invention discloses a method and system for identifying and searching based on multiple touch operations on a terminal interface, and relates to the field of search technology. The degree of coincidence between operations Select the interface area to be searched on the mask, identify the elements contained in the interface area to be searched, and search based on the identified elements, which not only omits the cumbersome input of keywords into the search box by the user The word process makes the search more convenient, avoids redundant elements contained in the interface area to be searched, and improves the search accuracy.

Description

Method and system for identifying and searching based on multiple touch operations on terminal interface

technical field

The present invention relates to the field of search technology, in particular to a method and system for identifying and searching based on multiple touch operations on a terminal interface.

Background technique

The search service on the terminal (for example: smartphone) is based on the input of the search box. The user needs to input keywords into the search box cumbersomely, so that the search engine can display the search results related to the keyword on the terminal display interface. , but the efficiency of this search method is very low, and the search experience is very poor.

Especially in the process of using the terminal, when the user has an immediate search demand for elements such as characters and images on the terminal display interface, he needs to open the search application first, and after the application pops up the search box, he can then search in the search box. Entering keywords is very inconvenient.

Contents of the invention

In view of the above problems, the present invention is proposed to provide a method for identifying and searching based on multiple touch operations on a terminal interface and a corresponding method based on multiple touch operations on a terminal interface that overcomes the above problems or at least partially solves the above problems. Operate systems that conduct identification searches.

According to one aspect of the present invention, a method for identifying and searching based on multiple touch operations on a terminal interface is provided, the method comprising:

When the user triggers an interface touch search request, a mask covering the terminal interface display area is generated in the current interface display area of the terminal;

receiving the user's multiple touch operations on the mask, and selecting an interface area to be searched on the mask according to the overlap between the multiple touch operations;

Identify the elements contained in the interface area to be searched, and search according to the identified elements.

Optionally, the receiving multiple touch operations of the user on the mask, and selecting an interface area to be searched on the mask according to the touch overlap between the multiple touch operations, further includes :

Detecting multiple touch trajectories generated based on multiple touch operations of the user on the mask, displaying multiple interface areas to be searched respectively corresponding to the multiple touch trajectories on the mask, the to-be-searched The search interface area is displayed differently from other areas on said mask;

If the coincidence degree between the two interface areas to be searched is not less than the coincidence threshold, then the two interface areas to be searched are merged into one interface area to be searched, otherwise the two interface areas to be searched are regarded as two independent areas to be searched interface area.

Optionally, the identifying elements contained in the interface area to be searched includes:

Select the smallest rectangular frame where each interface area to be searched is located on the mask, and identify the elements contained in the smallest rectangular frame.

Optionally, when the user triggers the interface touch search request, generating a mask covering the terminal interface display area in the current interface display area of the terminal further includes:

When the user triggers an interface touch search request, take a screenshot of the current interface display area of the terminal to obtain a screenshot of the current interface display area of the terminal, and use the screenshot as a mask covering the terminal interface display area.

Optionally, using the screenshot as a mask covering the display area of the terminal interface further includes:

The screenshot is used as a mask in a translucent state covering the display area of the terminal interface.

Optionally, the display mode of the interface area to be searched is highlighted or fully transparent;

and / or,

Other areas on the mask are displayed in grayscale, blurred, or translucent.

Optionally, the searching according to the identified elements further includes:

removing the mask, and pushing the element to a search box located in the display area of the terminal interface;

If the user clicks on the first icon set in the display area of the terminal interface, the search is performed according to the elements.

Optionally, before searching according to the identified elements, it also includes:

If the user clicks on the second icon set in the mask, it is triggered to search for webpages related to the information according to the identified information.

Optionally, after searching according to the identified elements, it further includes:

Displaying search result items related to the element in the display area of the terminal interface.

Optionally, displaying search result items related to the element in the display area of the terminal interface further includes:

The search result items related to the elements are displayed in the display area of the terminal interface in the form of application cards, text entries, and a combination of graphics and text.

Optionally, the elements include: at least one of text, pictures and symbols.

According to one aspect of the present invention, a system for identifying and searching based on multiple touch operations on a terminal interface is provided, and the system includes:

The mask generator is adapted to generate a mask covering the terminal interface display area in the current interface display area of the terminal when the user triggers an interface touch search request;

The touch receiver is adapted to receive the user's multiple touch operations on the mask, and select an interface area to be searched on the mask according to the overlap between the multiple touch operations;

The identification searcher is adapted to identify the elements contained in the interface area to be searched, and search according to the identified elements.

Optionally, the touch receiver is also adapted to detect multiple touch trajectories generated based on the user's multiple touch operations on the mask, and display on the mask A plurality of interface areas to be searched corresponding to the trajectories, the display mode of the interface areas to be searched is different from other areas on the mask; if the coincidence degree between the two interface areas to be searched is not less than the coincidence threshold, then The two interface areas to be searched are merged into one interface area to be searched, otherwise the two interface areas to be searched are regarded as two independent interface areas to be searched.

Optionally, the identifying searcher is further adapted to select the smallest rectangular frame where each interface area to be searched is located on the mask, and identify elements contained in the smallest rectangular frame.

Optionally, the mask generator is further adapted to take a screenshot of the current interface display area of the terminal when the user triggers an interface touch search request, so as to obtain a screenshot of the current interface display area of the terminal, and take the screenshot As a mask covering the display area of the terminal interface.

Optionally, the mask generator is further adapted to use the screenshot as a mask in a translucent state covering the display area of the terminal interface.

Optionally, the display mode of the interface area to be searched is highlighted or fully transparent;

and / or,

Other areas on the mask are displayed in grayscale, blurred, or translucent.

Optionally, the identification searcher is further adapted to remove the mask, and push the element to a search box located in the display area of the terminal interface;

When the user clicks on the first icon set in the display area of the terminal interface, the identification searcher is further adapted to search according to the element.

Optionally, when the user clicks on the second icon provided in the mask, the identification searcher is further adapted to trigger the execution of searching for webpages related to the information according to the identified information.

Optionally, the system also includes:

A search displayer is adapted to display search result items related to the element in the display area of the terminal interface.

Optionally, the search displayer is further adapted to display search result items related to the element in the form of application cards, text entries, and a combination of graphics and text in the display area of the terminal interface.

Optionally, the elements include: at least one of text, pictures and symbols.

The present invention selects the interface area to be searched on the mask according to the coincidence degree between the multiple touch operations by receiving the user's multiple touch operations on the mask, and identifies that the interface area to be searched contains The elements are searched according to the identified elements, which not only omits the cumbersome process of inputting keywords into the search box by the user, making the search more convenient, but also avoids the inclusion of redundant elements in the area of the interface to be searched, and improves the search efficiency. accuracy.

Description of drawings

Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiment. The drawings are only for the purpose of illustrating a preferred embodiment and are not to be considered as limiting the invention. Also throughout the drawings, the same reference numerals are used to designate the same components. In the attached picture:

FIG. 1 is a flow chart of steps of a method for identifying and searching based on multiple touch operations on a terminal interface according to an embodiment of the present invention;

FIG. 2 is a flow chart of steps of a method for identifying and searching based on multiple touch operations on a terminal interface according to an embodiment of the present invention;

Fig. 3 is an exemplary diagram of selecting an interface area to be searched on the mask according to an embodiment of the present invention;

Fig. 4 is an exemplary diagram of selecting an interface area to be searched on the mask according to an embodiment of the present invention;

Fig. 5 is an exemplary diagram of selecting an interface area to be searched on the mask according to an embodiment of the present invention;

FIG. 6 is a flow chart of steps of a method for identifying and searching based on multiple touch operations on a terminal interface according to an embodiment of the present invention;

FIG. 7 is a structural block diagram of a system for identifying and searching based on multiple touch operations on a terminal interface according to an embodiment of the present invention;

FIG. 8 is a structural block diagram of a system for identifying and searching based on multiple touch operations on a terminal interface according to an embodiment of the present invention.

Detailed ways

The specific implementation manners of the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

Fig. 1 is a flow chart of the steps of a method for identifying and searching based on multiple touch operations on a terminal interface according to an embodiment of the present invention; referring to Fig. 1 , the method includes:

S101: When the user triggers an interface touch search request, generate a mask covering the terminal interface display area in the current interface display area of the terminal;

It should be noted that the user can trigger the interface touch search request in a variety of ways, for example, the user can trigger the search floating window on the screen, or trigger the interface touch search request through some shortcuts such as buttons or gestures. The example does not limit this.

S102: Receive the user's multiple touch operations on the mask, and select an interface area to be searched on the mask according to the overlap between the multiple touch operations;

The user's touch operation on the mask, such as the user touching the touch point controlled by the screen with a finger or a sensor pen, can be selected by sliding the finger/stylus on the screen, or circle, etc. The interface area to be searched and so on.

S103: Identify elements included in the interface area to be searched, and search according to the identified elements.

It should be noted that the interface area to be searched usually only contains one type of element, such as a piece of text or a picture, and of course may also contain multiple types of elements at the same time, for example, both text and The picture is not limited in this embodiment of the present invention, and the types of the elements include but not limited to: text, pictures and symbols.

It can be understood that when the user performs a touch operation on the mask, multiple touch operations of the user on the mask are received, and an interface to be searched is selected on the mask according to the multiple touch operations area, identifying the elements included in the interface area to be searched. There are many ways of recognition. For example, text elements can be recognized through OCR optical character recognition technology, or text elements can be recognized by calling controls supported by the operating system, such as Uiautomater controls, and face recognition technology can also be used to identify text elements to be searched. Whether the elements contained in the interface area are images and faces, and similar image recognition technology can also be used to identify other sources of pictures on the Internet. Of course, other recognition technologies can also be used, and multiple recognition technologies can be combined to improve recognition accuracy. Spend. In this way, you can directly jump to the search interface according to the identified elements, thereby omitting the cumbersome process of users inputting keywords into the search box, or users need to copy the content of the page first, and then open the search client and paste it into the search box. Make the whole search interaction more convenient.

Of course, when selecting the interface area to be searched on the mask according to the multiple touch operations, the general processing method is to take the multiple touch operations on the mask as a whole, but this selection method will The interface area to be searched contains redundant elements, which affects the accuracy of the search. In this embodiment, multiple touch operations by the user on the mask are received, and according to the multiple touch operations, Select the interface area to be searched on the mask, identify the elements contained in the interface area to be searched, and search according to the identified elements, avoiding redundant elements contained in the interface area to be searched, Improved search accuracy.

Fig. 2 is a flow chart of the steps of a method for identifying and searching based on multiple touch operations on a terminal interface according to an embodiment of the present invention; referring to Fig. 2 , the method includes:

S201: When the user triggers an interface touch search request, take a screenshot of the current interface display area of the terminal to obtain a screenshot of the current interface display area of the terminal, and use the screenshot as a mask covering the terminal interface display area;

It is understandable that firstly taking a screenshot of the display area of the current interface of the terminal, and then using the screenshot as a mask covering the display area of the terminal interface, so that the applications or programs running in the background of the terminal will not be affected, and after the entire search interaction ends, the screen will be displayed again. It can return to the original browsing interface, thereby improving the user experience to a certain extent.

Of course, if the screenshot is only used as a pure transparent mask covering the display area of the terminal interface, since the mask is consistent with the content displayed in the display area of the terminal interface, the user may mistakenly believe that the terminal has not responded. In order to avoid this problem, in this embodiment, the mask is set to a translucent state, which can effectively prompt the user.

S202: Detect multiple touch trajectories generated based on the user's multiple touch operations on the mask, and display a plurality of interface areas to be searched respectively corresponding to the multiple touch trajectories on the mask. The display mode of the interface area to be searched is different from other areas on the mask; if the overlap between the two interface areas to be searched is not less than the coincidence threshold, the two interface areas to be searched are merged into one to be searched Search the interface area, otherwise the two interface areas to be searched are regarded as two independent interface areas to be searched;

It should be noted that, when selecting an interface area to be searched on the mask, if the display mode of the interface area to be searched is not distinguished from other areas on the mask, it will make it difficult for the user to determine the interface area to be searched. Elements in the interface area, in order to avoid this problem, in this embodiment, the display mode of the interface area to be searched is different from other areas on the mask.

Of course, in order to realize the difference in display methods, many display methods can be used. In this embodiment, the display method of the interface area to be searched can be displayed in a highlighted or fully transparent form, while other display methods on the mask The area can still be displayed in a semi-transparent form, or it can be displayed in a grayscale or blurred form. Other areas on the mask are displayed in a grayscale form, and the interface area to be searched is fully transparent. For presentation, of course, other ways may also be used to realize the difference in presentation manners, which is not limited in this embodiment of the present invention.

It can be understood that when selecting the interface area to be searched on the mask according to the degree of overlap between the multiple touch operations, various methods can be used, and in this embodiment, two interface areas to be searched are judged Whether the coincidence degree between is less than the coincidence threshold, if not less than, the two interface areas to be searched are merged into one interface area to be searched, otherwise the two interface areas to be searched are regarded as two independent interface areas to be searched, It further prevents redundant elements from being included in the interface area to be searched, and improves search accuracy.

Therefore, step S202 is essentially a preferred implementation of step S102 in the embodiment shown in FIG. 1 .

In this embodiment, the value of the coincidence threshold is 30%. Referring to FIG. The coincidence degree is lower than 30%, so the interface areas to be searched A and B are regarded as two independent interface areas to be searched;

Referring to Figures 4 and 5, the interface areas A and B to be searched corresponding to the user's touch track are displayed on the mask. Since the overlap between the interface areas A and B to be searched is higher than 30%, the interface area to be searched A and B are merged into one interface area to be searched, and it can be understood that the interface area to be searched after the merger is the same as the interface area B to be searched.

Of course, the coincidence threshold can be adjusted accordingly and set to other values as required.

S203: Identify the elements contained in the interface area to be searched, remove the mask, and push the elements to the search box located in the display area of the terminal interface; if the user clicks to set the search box in the display area of the terminal interface the first icon, search based on that element.

It should be noted that after identifying the elements contained in the interface area to be searched, the elements need to be pushed to the search box located in the display area of the terminal interface. If the mask is still retained, it is difficult to use Seeing the elements pushed into the search box, the mask needs to be removed after identifying the elements included in the interface area to be searched.

Therefore, step S203 is essentially a preferred implementation of step S103 in the embodiment shown in FIG. 1 .

It can be understood that, since the element needs to be pushed to the search box located in the display area of the terminal interface, the search box needs to be popped up in the display area of the terminal interface first.

In order to facilitate the user to search after confirming the element, after the user clicks the first icon that triggers the search, the search is implemented based on the element.

It should be noted that since there may be merged interface areas to be searched in the interface area to be searched after selecting according to the coincidence degree in step S202, in order to avoid identifying redundant elements as much as possible, in this embodiment, the Mongolian A method of selecting the smallest rectangular frame on the board where each interface area to be searched is located, and identifying the elements contained in the smallest rectangular frame.

When identifying the elements contained in the minimum rectangular frame, the boundary of the minimum rectangular frame can be determined first, and then the four coordinate points of the left, right, upper and lower sides of the minimum rectangular frame can be determined, and finally the elements determined by the four coordinate points can be identified. The elements contained within the smallest rectangular box.

Fig. 6 is a flow chart of the steps of a method for identifying and searching based on multiple touch operations on the terminal interface according to an embodiment of the present invention; the method of this embodiment includes:

S301: When the user triggers an interface touch search request, generate a mask covering the terminal interface display area in the current interface display area of the terminal;

S302: Receive multiple touch operations by the user on the mask, and select an interface area to be searched on the mask according to the overlap between the multiple touch operations;

S303: Identify elements included in the interface area to be searched, and search according to the identified elements;

S304: Display search result items related to the element in the display area of the terminal interface.

It should be noted that steps S301 to S303 are basically the same as steps S101 to S103 in the embodiment shown in FIG. 1 , so they will not be repeated here.

It should be noted that the search result items can be displayed in various forms. In this embodiment, in the display area of the terminal interface, information related to the element is displayed in the form of application cards, text entries, and a combination of graphics and text. The search result items, of course, may also be displayed in other forms, which are not limited in this embodiment of the present invention.

It can be understood that, in order to facilitate the user to confirm the interface area to be searched, and then identify the elements contained in the interface area to be searched, optionally, on the basis of the embodiments shown in Figures 1, 2, and 6, the Before identifying the elements included in the interface area to be searched, it further includes: if the user clicks on the second icon set in the mask, triggering the identification of the elements included in the interface area to be searched.

Of course, other icons can also be set in the mask, such as a repaint icon for reselecting the interface area to be searched, a return icon for exit, etc., which can be added as needed. This is not limited.

It can be understood that the execution subjects of the methods in the embodiments shown in Figures 1, 2, and 6 are all terminals, and the terminals can be devices such as desktop computers, notebooks, mobile phones, and tablet computers with touch screens. There are no restrictions on this.

For the method embodiment, for the sake of simple description, it is expressed as a series of action combinations, but those skilled in the art should know that the embodiment of the present invention is not limited by the described action order, because according to the embodiment of the present invention , certain steps may be performed in other order or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification belong to preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

Fig. 7 is a structural block diagram of a system for identifying and searching based on multiple touch operations on the terminal interface according to an embodiment of the present invention; referring to Fig. 7, the system includes:

The mask generator is adapted to generate a mask covering the terminal interface display area in the current interface display area of the terminal when the user triggers an interface touch search request;

The touch receiver is adapted to receive the user's multiple touch operations on the mask, and select an interface area to be searched on the mask according to the overlap between the multiple touch operations;

The identification searcher is adapted to identify the elements contained in the interface area to be searched, and search according to the identified elements.

In an optional embodiment of the present invention, the touch receiver is further adapted to detect multiple touch trajectories generated based on the user's multiple touch operations on the mask. A plurality of interface areas to be searched respectively corresponding to the plurality of touch tracks are displayed on the screen, and the display mode of the interface areas to be searched is different from other areas on the mask; If the coincidence degree is not less than the coincidence threshold, the two interface areas to be searched are merged into one interface area to be searched; otherwise, the two interface areas to be searched are regarded as two independent interface areas to be searched.

In an optional embodiment of the present invention, the identification searcher is further adapted to select the smallest rectangular frame where each interface area to be searched is located on the mask, and identify the elements contained in the smallest rectangular frame .

In an optional embodiment of the present invention, the mask generator is further adapted to take a screenshot of the current interface display area of the terminal when the user triggers an interface touch search request, so as to obtain the current interface display area of the terminal screenshot, and use the screenshot as a mask covering the display area of the terminal interface.

In an optional embodiment of the present invention, the mask generator is further adapted to use the screenshot as a mask in a translucent state covering the display area of the terminal interface.

In an optional embodiment of the present invention, the display mode of the interface area to be searched is highlighted or fully transparent;

and / or,

Other areas on the mask are displayed in grayscale, blurred, or translucent.

In an optional embodiment of the present invention, the identification searcher is further adapted to remove the mask, and push the element to a search box located in the display area of the terminal interface;

When the user clicks on the first icon set in the display area of the terminal interface, the identification searcher is further adapted to search according to the element.

In an optional embodiment of the present invention, when the user clicks on the second icon set in the mask, the identification searcher is further adapted to trigger the execution of searching for web pages related to the information according to the identified information .

Fig. 8 is a structural block diagram of a system for identifying and searching based on multiple touch operations on a terminal interface according to an embodiment of the present invention; referring to Fig. 8, the system includes:

The mask generator is adapted to generate a mask covering the terminal interface display area in the current interface display area of the terminal when the user triggers an interface touch search request;

The touch receiver is adapted to receive the user's multiple touch operations on the mask, and select an interface area to be searched on the mask according to the overlap between the multiple touch operations;

The identification searcher is adapted to identify the elements contained in the interface area to be searched, and search according to the identified elements.

A search displayer is adapted to display search result items related to the element in the display area of the terminal interface.

In an optional embodiment of the present invention, the search displayer is further adapted to display search results related to the element in the form of application cards, text entries, and a combination of graphics and text in the display area of the terminal interface item.

In an optional embodiment of the present invention, the elements include: at least one of text, pictures and symbols.

As for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and for the related parts, please refer to the part of the description of the method embodiment.

It should be noted that among the various components of the system of the present invention, the components are logically divided according to the functions to be realized, but the present invention is not limited thereto, and each component can be re-divided as required Or combined, for example, some components can be combined into a single component, or some components can be further broken down into more subcomponents.

The various component embodiments of the present invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art should understand that a microprocessor or a digital signal processor (DSP) may be used in practice to implement some or all functions of some or all components in the system according to the embodiments of the present invention. The present invention can also be implemented as an apparatus or an apparatus program (for example, a computer program and a computer program product) for performing a part or all of the methods described herein. Such a program for realizing the present invention may be stored on a computer-readable medium, or may be in the form of one or more signals. Such a signal may be downloaded from an Internet site, or provided on a carrier signal, or provided in any other form.

It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means can be embodied by one and the same item of hardware. The use of the words first, second, and third, etc. does not indicate any order. These words can be interpreted as names.

The above embodiments are only used to illustrate the present invention, but not to limit the present invention. Those of ordinary skill in the relevant technical field can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, all Equivalent technical solutions also belong to the category of the present invention, and the scope of patent protection of the present invention should be defined by the claims.

Claims (22)

1. A method for identifying and searching based on multiple touch operations on a terminal interface, characterized in that the method comprises: When the user triggers an interface touch search request, a mask covering the terminal interface display area is generated in the current interface display area of the terminal; receiving the user's multiple touch operations on the mask, and selecting an interface area to be searched on the mask according to the overlap between the multiple touch operations; Identify the elements contained in the interface area to be searched, and search according to the identified elements; wherein, The elements included in the interface area to be searched are identified by one or a combination of OCR optical character recognition technology, calling controls supported by the operating system, face recognition technology, and similar image recognition technology. 2. The method according to claim 1, characterized in that, the receiving user's multiple touch operations on the mask, and according to the touch overlap between the multiple touch operations in the Select the interface area to be searched on the mask, which further includes: Detecting multiple touch trajectories generated based on multiple touch operations of the user on the mask, displaying multiple interface areas to be searched respectively corresponding to the multiple touch trajectories on the mask, the to-be-searched The search interface area is displayed differently from other areas on said mask; If the coincidence degree between the two interface areas to be searched is not less than the coincidence threshold, then the two interface areas to be searched are merged into one interface area to be searched, otherwise the two interface areas to be searched are regarded as two independent areas to be searched interface area. 3. The method according to claim 1, wherein the identifying elements contained in the interface area to be searched comprises: Select the smallest rectangular frame where each interface area to be searched is located on the mask, and identify the elements contained in the smallest rectangular frame. 4. The method according to claim 1, wherein when the user triggers the interface touch search request, generating a mask covering the terminal interface display area in the current interface display area of the terminal, further comprising: When the user triggers an interface touch search request, take a screenshot of the current interface display area of the terminal to obtain a screenshot of the current interface display area of the terminal, and use the screenshot as a mask covering the terminal interface display area. 5. The method according to claim 4, wherein said taking the screenshot as a mask covering the display area of the terminal interface further comprises: The screenshot is used as a mask in a translucent state covering the display area of the terminal interface. 6. The method according to claim 1, characterized in that, the display mode of the interface area to be searched is highlighted or fully transparent; and / or, Other areas on the mask are displayed in grayscale, blurred, or translucent. 7. The method according to claim 1, wherein said searching according to the identified elements further comprises: removing the mask, and pushing the element to a search box located in the display area of the terminal interface; If the user clicks on the first icon set in the display area of the terminal interface, the search is performed according to the elements. 8. The method according to claim 1, further comprising: before searching according to the identified elements: If the user clicks on the second icon set in the mask, it is triggered to search for webpages related to the information according to the identified information. 9. The method according to claim 1, further comprising: after searching according to the identified elements: Displaying search result items related to the element in the display area of the terminal interface. 10. The method according to claim 9, wherein the displaying search result items related to the element in the display area of the terminal interface further comprises: The search result items related to the elements are displayed in the display area of the terminal interface in the form of application cards, text entries, and a combination of graphics and text. 11. The method according to any one of claims 1-10, wherein the elements include: at least one of text, pictures and symbols. 12. A system for identifying and searching based on multiple touch operations on a terminal interface, characterized in that the system includes: The mask generator is adapted to generate a mask covering the terminal interface display area in the current interface display area of the terminal when the user triggers an interface touch search request; The touch receiver is adapted to receive the user's multiple touch operations on the mask, and select an interface area to be searched on the mask according to the overlap between the multiple touch operations; An identification searcher, adapted to identify the elements contained in the interface area to be searched, and search according to the identified elements; wherein, The identification searcher is specifically used to identify the elements contained in the interface area to be searched by one or a combination of OCR optical character recognition technology, invoking controls supported by the operating system, face recognition technology, and similar image recognition technology. 13. The system according to claim 12, wherein the touch receiver is further adapted to detect multiple touch trajectories generated based on the user's multiple touch operations on the mask. A plurality of interface areas to be searched corresponding to the plurality of touch tracks are displayed on the mask, and the display mode of the interface areas to be searched is different from other areas on the mask; if two interface areas to be searched If the coincidence degree between them is not less than the coincidence threshold, the two interface areas to be searched are merged into one interface area to be searched, otherwise the two interface areas to be searched are regarded as two independent interface areas to be searched. 14. The system according to claim 12, wherein the identification searcher is further adapted to select the smallest rectangular frame where each interface area to be searched is located on the mask, and identify the smallest rectangular frame in the smallest rectangular frame. contained elements. 15. The system according to claim 12, wherein the mask generator is further adapted to take a screen shot of the terminal's current interface display area when the user triggers an interface touch search request, so as to obtain the terminal's current A screenshot of the interface display area, and using the screenshot as a mask covering the terminal interface display area. 16. The system according to claim 15, wherein the mask generator is further adapted to use the screenshot as a mask in a translucent state covering the display area of the terminal interface. 17. The system according to claim 12, wherein the display mode of the interface area to be searched is highlighted or fully transparent; and / or, Other areas on the mask are displayed in grayscale, blurred, or translucent. 18. The system according to claim 12, wherein the recognition searcher is further adapted to remove the mask, and push the element to the search box located in the display area of the terminal interface; When the user clicks on the first icon set in the display area of the terminal interface, the identification searcher is further adapted to search according to the element. 19. The system according to claim 12, wherein when the user clicks on the second icon set in the mask, the identification searcher is further adapted to trigger the execution of a search based on the information identified and the information related pages. 20. The system of claim 12, further comprising: The search displayer is adapted to display search result items related to the element in the display area of the terminal interface. 21. The system according to claim 20, wherein the search displayer is further adapted to display information related to the element in the form of application cards, text entries, and a combination of graphics and text in the display area of the terminal interface. search result items for . 22. The system according to any one of claims 12-21, wherein the elements include: at least one of text, pictures and symbols.