CN104536995B - The method and system of searching for based on terminal interface touch control operation - Google Patents

Abstract

The invention discloses a method and system for searching based on terminal interface touch operation, and relates to the field of search technology. The method includes: when a user triggers an interface touch search request, screenshots are taken of the display area of the current interface of the terminal to obtain A screenshot of the display area of the current interface of the terminal, and using the screenshot as a mask covering the display area of the terminal interface; receiving a user's touch operation on the mask, and according to the touch operation in the Select the interface area to be searched on the mask; identify the elements contained in the interface area to be searched, and search according to the identified elements. The present invention receives the user's touch operation on the mask, selects an interface area to be searched on the mask according to the touch operation, identifies the elements contained in the interface area to be searched, and performs a search based on the identified elements. Search, thereby omitting the cumbersome process of users entering keywords into the search box, making the search more convenient.

Description

Method and system for searching based on terminal interface touch operation

technical field

The present invention relates to the field of search technologies, in particular to a method and system for searching based on terminal interface touch operations.

Background technique

The search service on the terminal (for example: smartphone) is based on the input of the search box. The user needs to input keywords into the search box cumbersomely, so that the search engine can display the search results related to the keyword on the terminal display interface. , but the efficiency of this search method is very low, and the search experience is very poor.

Especially in the process of using the terminal, when the user has an immediate search demand for elements such as characters and images on the terminal display interface, he needs to open the search application first, and after the application pops up the search box, he can then search in the search box. Entering keywords is very inconvenient.

Contents of the invention

In view of the above problems, the present invention is proposed to provide a method for searching based on terminal interface touch operations and a corresponding system for searching based on terminal interface touch operations that overcome the above problems or at least partially solve the above problems .

According to one aspect of the present invention, there is provided a method for searching based on a terminal interface touch operation, the method comprising:

When the user triggers the interface touch search request, taking a screenshot of the current interface display area of the terminal to obtain a screenshot of the current interface display area of the terminal, and using the screenshot as a mask covering the display area of the terminal interface;

receiving a user's touch operation on the mask, and selecting an interface area to be searched on the mask according to the touch operation;

Identify the elements contained in the interface area to be searched, and search according to the identified elements.

Optionally, using the screenshot as a mask covering the display area of the terminal interface further includes:

The screenshot is used as a mask in a translucent state covering the display area of the terminal interface.

Optionally, receiving a user's touch operation on the mask, and selecting an interface area to be searched on the mask according to the touch operation further includes:

Detecting the touch track generated based on the user's touch operation on the mask, displaying the interface area to be searched corresponding to the touch track on the mask, the display mode of the interface area to be searched is different from that of Other areas on the mask.

Optionally, the display mode of the interface area to be searched is highlighted or fully transparent;

and / or,

Other areas on the mask are displayed in grayscale, blurred, or translucent.

Optionally, the searching according to the identified elements further includes:

removing the mask, and pushing the element to the search box located in the display area of the terminal interface;

If the user clicks on the first icon set in the display area of the terminal interface, the search is performed according to the elements.

Optionally, before identifying the elements contained in the interface area to be searched, further includes:

If the user clicks on the second icon set in the mask, it triggers to identify the elements contained in the interface area to be searched.

Optionally, after searching according to the identified elements, it further includes:

Displaying search result items related to the element in the display area of the terminal interface.

Optionally, displaying search result items related to the element in the display area of the terminal interface further includes:

The search result items related to the elements are displayed in the display area of the terminal interface in the form of application cards, text entries, and a combination of graphics and text.

Optionally, the elements include: at least one of text, pictures and symbols.

According to another aspect of the present invention, a system for searching based on terminal interface touch operation is provided, and the system includes:

The mask generator is adapted to take a screenshot of the current interface display area of the terminal when the user triggers an interface touch search request, so as to obtain a screenshot of the current interface display area of the terminal, and display the screenshot as covering the terminal interface area mask;

A touch receiver, adapted to receive a user's touch operation on the mask, and select an interface area to be searched on the mask according to the touch operation;

The identification searcher is adapted to identify the elements contained in the interface area to be searched, and search according to the identified elements.

Optionally, the mask generator is further adapted to: use the screenshot as a mask in a translucent state covering the display area of the terminal interface.

Optionally, the touch receiver is further adapted to detect a touch track generated based on the user's touch operation on the mask, and display an interface to be searched corresponding to the touch track on the mask area, the display mode of the interface area to be searched is different from other areas on the mask.

Optionally, the display mode of the interface area to be searched is highlighted or fully transparent;

and / or,

Other areas on the mask are displayed in grayscale, blurred, or translucent.

Optionally, the identification searcher is further adapted to remove the mask, and push the element to a search box located in the display area of the terminal interface;

When the user clicks on the first icon provided in the display area of the terminal interface, the identification searcher is further adapted to search according to the element.

Optionally, when the user clicks on the second icon provided in the mask, the identifying searcher is further adapted to trigger identifying elements contained in the interface area to be searched.

Optionally, the system also includes:

A search displayer is adapted to display search result items related to the element in the display area of the terminal interface.

Optionally, the search displayer is further adapted to display search result items related to the element in the form of application cards, text entries, and a combination of graphics and text in the display area of the terminal interface.

Optionally, the elements include: at least one of text, pictures and symbols.

The present invention receives the user's touch operation on the mask, selects an interface area to be searched on the mask according to the touch operation, identifies the elements contained in the interface area to be searched, and performs a search based on the identified elements. Search, thereby omitting the cumbersome process of users entering keywords into the search box, making the search more convenient.

Description of drawings

Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiment. The drawings are only for the purpose of illustrating a preferred embodiment and are not to be considered as limiting the invention. Also throughout the drawings, the same reference numerals are used to designate the same components. In the attached picture:

FIG. 1 is a flow chart of steps of a method for searching based on terminal interface touch operations according to an embodiment of the present invention;

FIG. 2 is a flow chart of steps of a method for searching based on terminal interface touch operations according to an embodiment of the present invention;

Fig. 3 is an example diagram of an interface area to be searched in an embodiment of the present invention;

Fig. 4 is an example diagram of a terminal interface display area after elements are pushed according to an embodiment of the present invention;

FIG. 5 is a flow chart of steps of a method for searching based on terminal interface touch operations according to an embodiment of the present invention;

Fig. 6 is a display example diagram of a search result item according to an embodiment of the present invention;

FIG. 7 is a structural block diagram of a system for searching based on terminal interface touch operations according to an embodiment of the present invention;

FIG. 8 is a structural block diagram of a system for searching based on terminal interface touch operations according to an embodiment of the present invention.

detailed description

The specific implementation manners of the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

Fig. 1 is a flow chart of steps of a method for searching based on terminal interface touch operation according to an embodiment of the present invention; referring to Fig. 1 , the method includes:

S101: When the user triggers an interface touch search request, take a screenshot of the current interface display area of the terminal to obtain a screenshot of the current interface display area of the terminal, and use the screenshot as a mask covering the terminal interface display area;

It should be noted that the user can trigger the interface touch search request in a variety of ways, for example, the user can trigger the search floating window on the screen, or trigger the interface touch search request through some shortcuts such as buttons or gestures. The example does not limit this.

It is understandable that firstly taking a screenshot of the display area of the current interface of the terminal, and then using the screenshot as a mask covering the display area of the terminal interface, so that the applications or programs running in the background of the terminal will not be affected, and after the entire search interaction ends, the screen will be displayed again. It can return to the original browsing interface, thereby improving the user experience to a certain extent.

Of course, if the screenshot is only used as a pure transparent mask covering the display area of the terminal interface, since the mask is consistent with the content displayed in the display area of the terminal interface, the user may mistakenly believe that the terminal has not responded. In order to avoid this problem, in this embodiment, the mask is set to a translucent state, which can effectively prompt the user.

S102: Receive a user's touch operation on the mask, and select an interface area to be searched on the mask according to the touch operation;

The user's touch operation on the mask, such as touching the touch point controlled by the screen with the user's finger or stylus, can be selected by sliding the finger/stylus on the screen, or circle, etc., so as to select the The interface area to be searched and so on.

S103: Identify elements included in the interface area to be searched, and search according to the identified elements.

It should be noted that the interface area to be searched usually only contains one type of element, such as a piece of text or a picture, and of course may also contain multiple types of elements at the same time, for example, both text and The picture is not limited in this embodiment of the present invention, and the types of the elements include but not limited to: text, pictures and symbols.

In this embodiment, by receiving a user's touch operation on the mask, an interface area to be searched is selected on the mask according to the touch operation, and elements contained in the interface area to be searched are identified. There are many ways of recognition. For example, text elements can be recognized through OCR optical character recognition technology, or text elements can be recognized by calling controls supported by the operating system, such as Uiautomater controls, and face recognition technology can also be used to identify text elements to be searched. Whether the elements contained in the interface area are images and faces, and similar image recognition technology can also be used to identify other sources of pictures on the Internet. Of course, other recognition technologies can also be used, and multiple recognition technologies can be combined to improve recognition accuracy. Spend. In this way, you can directly jump to the search interface according to the identified elements, thereby omitting the cumbersome process of users inputting keywords into the search box, or users need to copy the content of the page first, and then open the search client and paste it into the search box. Make the whole search interaction more convenient.

Fig. 2 is a flow chart of steps of a method for searching based on terminal interface touch operation according to an embodiment of the present invention; the method of this embodiment includes:

S201: When the user triggers an interface touch search request, take a screenshot of the current interface display area of the terminal to obtain a screenshot of the current interface display area of the terminal, and use the screenshot as a mask covering the terminal interface display area;

S202: Detect the touch track generated based on the user's touch operation on the mask, and display the interface area to be searched corresponding to the touch track on the mask, and the display modes of the interface area to be searched are different other areas on the mask;

It should be noted that, when selecting an interface area to be searched on the mask, if the display mode of the interface area to be searched is not distinguished from other areas on the mask, it will make it difficult for the user to determine the interface area to be searched. Elements in the interface area, in order to avoid this problem, in this embodiment, the display mode of the interface area to be searched is different from other areas on the mask.

Therefore, step S202 is essentially a preferred implementation of step S102 in the embodiment shown in FIG. 1 .

It can be understood that, in order to achieve the difference in display methods, many display methods can be adopted. In this embodiment, the display method of the interface area to be searched can be highlighted or fully transparent, and the mask Other areas on the mask can still be displayed in a semi-transparent form, or in a grayscale form. Referring to FIG. The search interface area B is displayed in a fully transparent manner. Of course, other methods may also be used to realize the difference in display methods, which is not limited in this embodiment of the present invention.

S203: Identify the elements contained in the interface area to be searched, remove the mask, and push the elements to the search box C located in the display area of the terminal interface with reference to Figure 4; The first icon D in the display area of the terminal interface is searched according to the element.

It should be noted that after identifying the elements contained in the interface area to be searched, the elements need to be pushed to the search box located in the display area of the terminal interface. If the mask is still retained, it is difficult to use Seeing the elements pushed into the search box, the mask needs to be removed after identifying the elements included in the interface area to be searched.

Therefore, step S203 is essentially a preferred implementation of step S103 in the embodiment shown in FIG. 1 .

It can be understood that, since the element needs to be pushed to the search box located in the display area of the terminal interface, the search box needs to be popped up in the display area of the terminal interface first.

In order to facilitate the user to search after confirming the element, after the user clicks the first icon that triggers the search, the search is implemented based on the element.

Fig. 5 is a flow chart of steps of a method for searching based on terminal interface touch operation according to an embodiment of the present invention; the method of this embodiment includes:

S501: When the user triggers an interface touch search request, take a screenshot of the current interface display area of the terminal to obtain a screenshot of the current interface display area of the terminal, and use the screenshot as a mask covering the terminal interface display area;

S502: Receive a user's touch operation on the mask, and select an interface area to be searched on the mask according to the touch operation;

S503: Identify elements included in the interface area to be searched, and search according to the identified elements.

S504: Display search result items related to the element in the display area of the terminal interface.

It should be noted that steps S501 to S503 are basically the same as steps S101 to S103 in the embodiment shown in FIG. 1 , so they will not be repeated here.

It should be noted that the search result items can be displayed in various forms. In this embodiment, in the display area of the terminal interface, information related to the element is displayed in the form of application cards, text entries, and a combination of graphics and text. As for the search result item, FIG. 6 is a display manner of the search result item. Of course, other forms may also be used for display, which is not limited in this embodiment of the present invention.

It can be understood that, in order to facilitate the user to confirm the interface area to be searched, and then identify the elements contained in the interface area to be searched, optionally, on the basis of the embodiments shown in Figures 1, 2, and 5, the Before identifying the elements contained in the interface area to be searched, it also includes: if the user clicks on the second icon (i.e. "E" in Fig. 3 ) set in the mask, triggering the identification of the interface to be searched The elements contained within the region.

Of course, other icons can also be set in the mask, such as a repaint icon (ie, "F" in Fig. "G") in 3, etc., can be added as needed, which is not limited in this embodiment of the present invention.

It can be understood that the execution subjects of the methods in the embodiments shown in Figures 1, 2, and 5 are all terminals, and the terminals can be devices such as desktop computers, notebooks, mobile phones, and tablet computers with touch screens. There is no restriction on this.

For the method embodiment, for the sake of simple description, it is expressed as a series of action combinations, but those skilled in the art should know that the embodiment of the present invention is not limited by the described action order, because according to the embodiment of the present invention , certain steps may be performed in other order or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification belong to preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

Fig. 7 is a structural block diagram of a system for searching based on terminal interface touch operation according to an embodiment of the present invention; referring to Fig. 7, the system includes:

The mask generator 701 is adapted to take a screenshot of the current interface display area of the terminal when the user triggers an interface touch search request, so as to obtain a screenshot of the current interface display area of the terminal, and use the screenshot as a method to cover the terminal interface mask of the display area;

The touch receiver 702 is adapted to receive a user's touch operation on the mask, and select an interface area to be searched on the mask according to the touch operation;

The identification searcher 703 is adapted to identify the elements contained in the interface area to be searched, and search according to the identified elements.

In an optional embodiment of the present invention, the mask generator is further adapted to: use the screenshot as a mask in a translucent state covering the display area of the terminal interface.

In an optional embodiment of the present invention, the display mode of the interface area to be searched is highlighted or fully transparent;

and / or,

Other areas on the mask are displayed in grayscale, blurred, or translucent.

In an optional embodiment of the present invention, the touch receiver is further adapted to detect a touch track generated based on the user's touch operation on the mask, and display on the mask the corresponding The interface area to be searched corresponding to the touch track, the display mode of the interface area to be searched is different from other areas on the mask.

In an optional embodiment of the present invention, the identification searcher is further adapted to remove the mask, and push the element to a search box located in the display area of the terminal interface;

When the user clicks on the first icon provided in the display area of the terminal interface, the identification searcher is further adapted to search according to the element.

Fig. 8 is a structural block diagram of a system for searching based on terminal interface touch operation according to an embodiment of the present invention; referring to Fig. 8, the system includes:

The mask generator 801 is adapted to take a screenshot of the current interface display area of the terminal when the user triggers an interface touch search request, so as to obtain a screenshot of the current interface display area of the terminal, and use the screenshot as a method to cover the terminal interface mask of the display area;

The touch receiver 802 is adapted to receive a user's touch operation on the mask, and select an interface area to be searched on the mask according to the touch operation;

The identification searcher 803 is adapted to identify the elements contained in the interface area to be searched, and search according to the identified elements.

The search displayer 804 is adapted to display search result items related to the element in the display area of the terminal interface.

In an optional embodiment of the present invention, the search displayer 804 is further adapted to display the search related to the element in the form of application cards, text entries, and a combination of graphics and text in the display area of the terminal interface. result item.

In an optional embodiment of the present invention, it is characterized in that the elements include: at least one of text, pictures and symbols.

In an optional embodiment of the present invention, when the user clicks on the second icon provided in the mask, the identifying searcher is further adapted to trigger identifying elements contained in the interface area to be searched.

As for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and for the related parts, please refer to the part of the description of the method embodiment.

It should be noted that among the various components of the system of the present invention, the components are logically divided according to the functions to be realized, but the present invention is not limited thereto, and each component can be re-divided as required Or combined, for example, some components can be combined into a single component, or some components can be further broken down into more subcomponents.

The various component embodiments of the present invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art should understand that a microprocessor or a digital signal processor (DSP) may be used in practice to implement some or all functions of some or all components in the system according to the embodiments of the present invention. The present invention can also be implemented as an apparatus or an apparatus program (for example, a computer program and a computer program product) for performing a part or all of the methods described herein. Such a program for realizing the present invention may be stored on a computer-readable medium, or may be in the form of one or more signals. Such a signal may be downloaded from an Internet site, or provided on a carrier signal, or provided in any other form.

It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means can be embodied by one and the same item of hardware. The use of the words first, second, and third, etc. does not indicate any order. These words can be interpreted as names.

The above embodiments are only suitable for illustrating the present invention, rather than limiting the present invention. Those of ordinary skill in the relevant technical fields can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, all Equivalent technical solutions also belong to the category of the present invention, and the scope of patent protection of the present invention should be defined by the claims.

Claims (16)

1. A method for searching based on a touch track generated by a terminal interface touch operation, characterized in that the method comprises: When the user triggers the interface touch search request, taking a screenshot of the current interface display area of the terminal to obtain a screenshot of the current interface display area of the terminal, and using the screenshot as a mask covering the display area of the terminal interface; Detecting the touch track generated based on the user's touch operation on the mask, displaying the interface area to be searched corresponding to the touch track on the mask, the display mode of the interface area to be searched is different from that of other areas on said mask; Identify the elements contained in the interface area to be searched, and search according to the identified elements. 2. The method according to claim 1, wherein said taking the screenshot as a mask covering the display area of the terminal interface further comprises: The screenshot is used as a mask in a translucent state covering the display area of the terminal interface. 3. The method according to claim 2, wherein the display mode of the interface area to be searched is highlighted or fully transparent; and / or, Other areas on the mask are displayed in grayscale, blurred, or translucent. 4. The method according to claim 1, wherein said searching according to the identified elements further comprises: removing the mask, and pushing the element to the search box located in the display area of the terminal interface; If the user clicks on the first icon set in the display area of the terminal interface, the search is performed according to the elements. 5. The method according to any one of claims 1-4, characterized in that before identifying the elements contained in the interface area to be searched, further comprising: If the user clicks on the second icon set in the mask, it triggers to identify the elements contained in the interface area to be searched. 6. The method according to any one of claims 1 to 4, characterized in that, after searching according to the identified elements, further comprising: Displaying search result items related to the element in the display area of the terminal interface. 7. The method according to claim 6, wherein the displaying search result items related to the element in the display area of the terminal interface further comprises: The search result items related to the elements are displayed in the display area of the terminal interface in the form of application cards, text entries, and a combination of graphics and text. 8. The method according to any one of claims 1-4, wherein the elements include: at least one of text, pictures and symbols. 9. A system for searching based on touch traces generated by terminal interface touch operations, characterized in that the system includes: The mask generator is adapted to take a screenshot of the current interface display area of the terminal when the user triggers the interface touch search request, so as to obtain a screenshot of the current interface display area of the terminal, and display the screenshot as covering the terminal interface area mask; The touch receiver is adapted to detect the touch track generated based on the user's touch operation on the mask, and display the area of the interface to be searched corresponding to the touch track on the mask, and the interface to be searched the way the region is displayed differently from other regions on said mask; The identification searcher is adapted to identify the elements contained in the interface area to be searched, and search according to the identified elements. 10. The system according to claim 9, wherein the mask generator is further adapted to: use the screenshot as a mask in a translucent state covering the display area of the terminal interface. 11. The system according to claim 10, characterized in that, the display mode of the interface area to be searched is highlighted or fully transparent; and / or, Other areas on the mask are displayed in grayscale, blurred, or translucent. 12. The system according to claim 9, wherein the identification searcher is further adapted to remove the mask, and push the element to the search box located in the display area of the terminal interface; When the user clicks on the first icon provided in the display area of the terminal interface, the identification searcher is further adapted to search according to the element. 13. The system according to any one of claims 9-12, wherein when the user clicks on the second icon set in the mask, the recognition searcher is further adapted to trigger the recognition of the The elements contained within the interface area to be searched. 14. The system according to any one of claims 9 to 12, further comprising: A search displayer is adapted to display search result items related to the element in the display area of the terminal interface. 15. The system according to claim 14, wherein the search displayer is further adapted to display information related to the element in the form of application cards, text entries, and a combination of graphics and text in the display area of the terminal interface. search result items for . 16. The system according to any one of claims 9-12, wherein the elements include: at least one of text, pictures and symbols.