CN106354777B - A method and device for searching questions applied to an electronic terminal - Google Patents

Abstract

The invention is applicable to the technical field of electronic terminals and provides a question searching method and device applied to an electronic terminal. The method comprises the following steps: when receiving a question searching trigger instruction, starting a rear camera of the electronic terminal; displaying a preview picture shot by the rear camera on a screen of the electronic terminal; starting a front camera of the electronic terminal, and capturing the motion trail of the pupil of the user through the front camera; mapping the captured motion track to a corresponding position of a screen, and capturing a preview picture in the corresponding position of the screen; uploading the intercepted preview picture to a server for identification and test question matching; and receiving and displaying the problem solving result which is fed back by the server and corresponds to the matched test problem. The invention can reduce the steps of searching the questions and improve the searching efficiency.

Description

A method and device for searching questions applied to an electronic terminal

technical field

The invention belongs to the technical field of electronic terminals, and in particular relates to a method and device for searching questions applied to electronic terminals.

Background technique

At present, many electronic education products on the market can realize the function of searching for questions by taking photos, searching for corresponding definitions by taking photos of words or phrases, or searching for corresponding answers by taking photos of questions. Most of the existing photo search functions need to go through the following steps: start the camera - click to take a photo - intercept the search object in the image - convert the search object into text through OCR technology - search and match the search result by the server - feed the search result to the server electronic terminal.

As can be seen from the above, the existing methods for taking photos and searching for questions have many steps, cumbersome operations and a single method (such as manual clicking, manual frame selection of test questions). Therefore, it is necessary to propose a new technical solution to solve the above technical problems.

Contents of the invention

In view of this, the embodiments of the present invention provide a method and device for searching questions applied to electronic terminals, so as to control the electronic terminal to search for test questions through eye movement sensing, reduce the steps of searching for questions, and improve the efficiency of searching for questions.

In the first aspect, an embodiment of the present invention provides a method for searching a question applied to an electronic terminal, and the method for searching a question includes:

When receiving the search trigger command, start the rear camera of the electronic terminal;

displaying the preview picture captured by the rear camera on the screen of the electronic terminal;

Start the front camera of the electronic terminal, and capture the movement track of the user's pupil through the front camera;

Mapping the captured motion track to a corresponding position on the screen, and intercepting a preview image in the corresponding position of the screen;

Upload the captured preview screen to the server for identification and test question matching;

Receiving and displaying the problem-solving results corresponding to the matched test questions fed back by the server.

In a second aspect, an embodiment of the present invention provides a question searching device applied to an electronic terminal, and the question searching device includes:

The starting module is used to start the rear camera of the electronic terminal when receiving the trigger command of the search question;

A first display module, configured to display a preview image captured by the rear camera on the screen of the electronic terminal;

The capture module is used to start the front camera of the electronic terminal, and capture the movement track of the user's pupil through the front camera;

An intercepting module, configured to map the captured motion trajectory to a corresponding position on the screen, and intercept a preview image in the corresponding position of the screen;

The upload module is used to upload the intercepted preview screen to the server for identification and test item matching;

The second display module is used to receive and display the problem-solving results corresponding to the matched test questions fed back by the server.

Compared with the prior art, the embodiment of the present invention has the following beneficial effects: after the camera is started, the embodiment of the present invention captures the movement trajectory of the user's pupils and directly performs a screenshot operation on the preview interface, without waiting for the picture to be taken before intercepting the test questions. It simplifies the operation process and improves the efficiency of searching questions. Moreover, the use of eye movement sensing to control the electronic terminal to search for test questions solves the problem of a single way to search for existing questions, meets the individual needs of users, and has strong ease of use and practicability.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the accompanying drawings that need to be used in the descriptions of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only of the present invention. For some embodiments, those skilled in the art can also obtain other drawings according to these drawings without paying creative efforts.

FIG. 1 is a schematic diagram of an implementation flow of a method for searching a question applied to an electronic terminal provided by an embodiment of the present invention;

Fig. 2 is a schematic diagram of the composition and structure of a question searching device applied to an electronic terminal provided by an embodiment of the present invention.

Detailed ways

In the following description, specific details such as specific system structures and technologies are presented for the purpose of illustration rather than limitation, so as to thoroughly understand the embodiments of the present invention. It will be apparent, however, to one skilled in the art that the invention may be practiced in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present invention with unnecessary detail.

In order to illustrate the technical solutions of the present invention, specific examples are used below to illustrate.

Please refer to Fig. 1, the implementation flow of the search question method applied to the electronic terminal provided by the embodiment of the present invention, which mainly includes the following steps:

Step S101, when receiving a topic search trigger command, start the rear camera of the electronic terminal.

In the embodiment of the present invention, the electronic terminal is configured with an input interface of "triggering instruction for searching questions", so that the user can input the above-mentioned triggering instruction for searching questions through the input interface. Exemplarily, the input interface can be represented as a physical key, so that the above-mentioned search trigger command can be input through the physical key; or the input interface can be represented as a specific touch operation, for example, when the user's touch action on the screen is detected After a two-point touch and the sliding trajectory is a vertical relative slide, it is judged whether the displacement of the two touch points relative to the vertical slide is greater than the preset first threshold at the same time, and whether the distance difference between the final landing points of the two touch points is smaller than the preset second threshold. Threshold, and whether the sliding speed of the two touch points is greater than the preset third threshold at the same time, if so, it is determined as a search question trigger command. Alternatively, the input interface can also be represented as a start-up interface of an application program (such as a question-searching App), and when the user starts the question-searching App, it is equivalent to inputting the above-mentioned question-searching trigger command. The embodiment of the present invention does not limit the specific input method of the above-mentioned search question trigger command.

In the embodiment of the present invention, when the above-mentioned search question trigger command is received, the rear camera of the electronic terminal is started (the rear camera can be the built-in camera of the above-mentioned electronic terminal, and can also be the external camera of the above-mentioned electronic terminal), Further, after the above-mentioned rear camera is started, the preview interface of the above-mentioned rear camera can be loaded on the screen of the above-mentioned electronic terminal, so that the user can browse the viewfinder range of the above-mentioned rear camera through the preview interface, and control the viewing of the viewfinder. Shooting within the search object. Wherein, the search objects include but not limited to words, phrases, test questions and the like.

Step S102, displaying the preview image captured by the rear camera on the screen of the electronic terminal.

Step S103, start the front camera of the electronic terminal, and use the front camera to capture the movement track of the user's pupil.

In the embodiment of the present invention, the front camera may be a built-in camera of the above-mentioned electronic terminal, or may be an external camera of the above-mentioned electronic terminal. Optionally, the front camera may also use an eye tracker to capture the movement track of the user's pupils through the eye tracker.

Step S104, mapping the captured motion trajectory to a corresponding position on the screen, and intercepting a preview image in the corresponding position on the screen.

For example, when the motion trajectory is a circle or an ellipse, the trajectory mapped to the corresponding position on the screen is also a circle or an ellipse.

Preferably, mapping the captured motion track to a corresponding position on the screen, and intercepting the preview image in the corresponding position on the screen includes:

Mapping the captured motion trajectory to a corresponding position on the screen, and when the trajectory mapped to the corresponding position on the screen forms a closed loop, intercepting a preview image in the closed loop.

Preferably, the mapping the captured motion trajectory to the corresponding position of the screen, and intercepting the preview picture in the corresponding position of the screen may also include:

Mapping the captured motion trajectory to a corresponding position on the screen, and intercepting a preview image in the corresponding position on the screen when a photographing instruction is received. Wherein, the photographing instruction may be specifically triggering the photographing instruction when the front-facing camera captures the user's blinking action (for example, blinking more than twice within a predetermined time period). Exemplarily, it may be to take a picture of the current preview screen after receiving the picture taking instruction, and then capture the picture in the corresponding position of the screen after taking the picture.

Step S105, uploading the intercepted preview screen to the server for identification and test item matching.

In the embodiment of the present invention, the character recognition technology includes but not limited to Optical Character Recognition (OCR, Optical Character Recognition) technology, Scene Text Recognition (STR, Scene Text Recognition) technology, and the like. After character recognition is performed on the intercepted preview screen by the character recognition technology, the recognized characters are put into the question bank for search and matching, and the matched results are fed back to the electronic terminal.

Step S106, receiving and displaying the problem-solving results corresponding to the matched test questions fed back by the server.

For example, when the number of matched test questions is 1, one or more results corresponding to the matched test questions are received and displayed. Wherein, the problem-solving result may include: a problem-solving process and an answer.

Preferably, because traditional question-searching apps use a single presentation method to directly display the problem-solving results, it is easy for students to form dependent thinking, directly search for questions and plagiarize answers, lack process thinking and sense of participation, and it is difficult to stimulate interest in problem-solving. Therefore, optionally, the embodiment of the present invention uses a non-transparent mask to block the problem-solving result while displaying the problem-solving result, and the corresponding problem-solving can only be viewed after the non-transparent mask is eliminated by inputting a mask removal command As a result, students are given more time to think.

Wherein, the displaying the problem-solving results fed back by the server and corresponding to the matched test questions this time may include:

Displaying the problem-solving process fed back by the server corresponding to the matched test question this time and using the preset first non-transparent mask to block the problem-solving process;

Displaying the answers fed back by the server corresponding to the matched test questions this time and using a preset second non-transparent mask to block the answers;

After displaying the problem-solving results fed back by the server and corresponding to the matched test questions this time, it may also include:

When a first mask removal instruction is received, remove the first opaque mask under the instruction of the first mask removal instruction;

When a second mask removal instruction is received, the second non-transparent mask layer is removed under the instruction of the second mask removal instruction.

Specifically, the user can respectively input the first mask removal instruction and the first mask removal instruction through different preset physical buttons, or can also input the first mask removal instruction and the second mask removal instruction through touch operation. Mask removal command. For the above-mentioned touch operation, it can be set that when it is detected that the user touches the area where the above-mentioned first non-transparent mask layer (or the above-mentioned second non-transparent layer) is located, it is determined that the above-mentioned first mask layer removal instruction (or the above-mentioned second non-transparent layer) two mask removal instructions), and can eliminate the above-mentioned first non-transparent mask layer (or the above-mentioned first non-transparent mask layer) under the instruction of the above-mentioned first mask layer removal instruction (or the above-mentioned second mask layer removal instruction). The entire area or the touched area (the touched area that eliminates the above-mentioned first non-transparent mask layer (or the above-mentioned second non-transparent mask layer) is similar to the form of a scratch prize).

In the embodiment of the present invention, the rear camera captures a preview image, and the front camera captures the motion trajectory of the user's pupil, and maps the captured motion trajectory to a corresponding position on the screen, so as to intercept the preview image in the corresponding position. The embodiment of the present invention does not need to wait for the picture to be taken before intercepting the test questions, which simplifies the operation process and improves the efficiency of searching questions. Moreover, the use of eye movement sensing to control the electronic terminal to search for test questions solves the problem of a single way of searching for existing questions and meets the individual needs of users.

Please refer to FIG. 2 , which is a schematic structural diagram of a question searching device applied to an electronic terminal provided by an embodiment of the present invention. For ease of description, only parts related to the embodiments of the present invention are shown.

The question searching device applied to an electronic terminal can be a software unit, a hardware unit, or a combination of software and hardware built into an electronic terminal (such as a mobile phone, a tablet computer, etc.), or it can be integrated into the electronic terminal as an independent pendant. in the terminal.

The question searching device applied to the electronic terminal includes:

The starting module 21 is used to start the rear camera of the electronic terminal when receiving the trigger command of the search question;

The first display module 22 is configured to display the preview picture captured by the rear camera on the screen of the electronic terminal;

The capture module 23 is used to start the front camera of the electronic terminal, and capture the movement track of the user's pupil through the front camera;

An intercepting module 24, configured to map the captured motion trajectory to a corresponding position on the screen, and intercept a preview image in the corresponding position of the screen;

The upload module 25 is used to upload the captured preview screen to the server for identification and test item matching;

The second display module 26 is configured to receive and display the problem-solving results fed back by the server and corresponding to the matched test questions this time.

Optionally, the interception module 24 is specifically used for:

Mapping the captured motion trajectory to a corresponding position on the screen, and when the trajectory mapped to the corresponding position on the screen forms a closed loop, intercepting a preview image in the closed loop.

Optionally, the interception module 24 is specifically used for:

Mapping the captured motion trajectory to a corresponding position on the screen, and intercepting a preview image in the corresponding position on the screen when a photographing instruction is received.

Optionally, the interception module 24 is specifically used for:

Map the captured motion trajectory to the corresponding position of the screen, and when the front camera captures the user's blinking action, trigger a camera instruction, and intercept the preview image in the corresponding position of the screen.

Optionally, the problem-solving results include: problem-solving process and answers;

The second display module 26 is specifically configured to display the problem-solving process fed back by the server and corresponding to the matched test question this time and use the preset first non-transparent mask to cover the problem-solving process, and display the problem-solving process. The answer corresponding to the matched test question fed back by the server and using the preset second non-transparent mask to block the answer;

The device also includes:

The elimination module 27 is configured to, after displaying the problem-solving results fed back by the server and corresponding to the matched test questions this time, when receiving the first mask removal instruction, under the instruction of the first mask removal instruction Eliminate the first non-transparent mask layer; when a second mask layer removal instruction is received, remove the second non-transparent mask layer under the instruction of the second mask layer removal instruction.

To sum up, in the embodiment of the present invention, the rear camera captures the preview image, and the front camera captures the movement trajectory of the user's pupils, and maps the captured movement trajectory to the corresponding position of the screen, so as to intercept the corresponding position. preview screen. The embodiment of the present invention does not need to wait for the picture to be taken before intercepting the test questions, which simplifies the operation process and improves the efficiency of searching questions. Moreover, the use of eye movement sensing to control the electronic terminal to search for test questions solves the problem of a single way to search for existing questions, meets the individual needs of users, and has strong ease of use and practicability.

Those skilled in the art can clearly understand that for the convenience and brevity of description, only the division of the above-mentioned functional modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules according to needs. That is, the internal structure of the device is divided into different functional units or modules, so as to complete all or part of the functions described above. Each functional module in the embodiment can be integrated into one processing unit, or each unit can exist separately physically, or two or more units can be integrated into one unit, and the above-mentioned integrated units can be implemented in the form of hardware , can also be implemented in the form of software functional units. In addition, the specific names of the functional modules are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the present application. For the specific working process of the modules in the above-mentioned device, reference may be made to the corresponding process in the aforementioned method embodiments, which will not be repeated here.

Those skilled in the art can appreciate that the modules and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed by hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be regarded as exceeding the scope of the present invention.

In the embodiments provided in the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be Incorporation may either be integrated into another system, or some features may be omitted, or not implemented. In another point, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

If the integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage In the medium, several instructions are included to make a computer device (which may be a personal computer, server, or network device, etc.) or a processor (processor) execute all or part of the steps of the methods described in the various embodiments of the embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program codes. .

The above-described embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: it can still carry out the foregoing embodiments The technical solutions described in the examples are modified, or some of the technical features are equivalently replaced; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims (8)

1. A method for searching a question applied to an electronic terminal, characterized in that, the method for searching a question comprises: When receiving the search trigger command, start the rear camera of the electronic terminal; displaying the preview picture captured by the rear camera on the screen of the electronic terminal; Start the front camera of the electronic terminal, and capture the movement track of the user's pupil through the front camera; Mapping the captured motion trajectory to a corresponding position on the screen, and intercepting a preview image in the corresponding position of the screen, wherein the shape of the trajectory mapped to the corresponding position on the screen is the same as the captured motion trajectory of the user's pupil; Upload the captured preview screen to the server for identification and test question matching; receiving and displaying the problem-solving results fed back by the server and corresponding to the matched test questions; The problem-solving result includes: a problem-solving process and an answer; The displaying the problem-solving results corresponding to the matched test questions this time as fed back by the server includes: Displaying the problem-solving process fed back by the server corresponding to the matched test question this time and using the preset first non-transparent mask to block the problem-solving process; Displaying the answers fed back by the server corresponding to the matched test questions this time and using a preset second non-transparent mask to block the answers; After displaying the problem-solving results fed back by the server and corresponding to the matched test questions this time, it also includes: When a first mask removal instruction is received, remove the first opaque mask under the instruction of the first mask removal instruction; When a second mask removal instruction is received, the second non-transparent mask layer is removed under the instruction of the second mask removal instruction. 2. The method of searching for a question as claimed in claim 1, wherein mapping the captured motion track to the corresponding position of the screen, and intercepting the preview image in the corresponding position of the screen comprises: Mapping the captured motion trajectory to a corresponding position on the screen, and when the trajectory mapped to the corresponding position on the screen forms a closed loop, intercepting a preview image in the closed loop. 3. The method of searching for a question as claimed in claim 1, characterized in that, mapping the captured motion track to the corresponding position of the screen, and intercepting the preview picture in the corresponding position of the screen comprises: Mapping the captured motion trajectory to a corresponding position on the screen, and intercepting a preview image in the corresponding position on the screen when a photographing instruction is received. 4. The method for searching questions as claimed in claim 3, wherein the photographing instruction comprises: When the front camera captures the blinking action of the user, a camera instruction is triggered. 5. A question-searching device applied to an electronic terminal, characterized in that, the question-searching device comprises: The starting module is used to start the rear camera of the electronic terminal when receiving the trigger command of the search question; A first display module, configured to display a preview image captured by the rear camera on the screen of the electronic terminal; The capture module is used to start the front camera of the electronic terminal, and capture the movement track of the user's pupil through the front camera; An interception module, configured to map the captured motion trajectory to a corresponding position on the screen, and intercept a preview image in the corresponding position of the screen, wherein the shape of the trajectory mapped to the corresponding position on the screen is consistent with the captured movement of the user's pupil The trajectories have the same shape; The upload module is used to upload the intercepted preview screen to the server for identification and test item matching; The second display module is used to receive and display the problem-solving results fed back by the server and corresponding to the matched test questions; The problem-solving result includes: a problem-solving process and an answer; The second display module is specifically configured to display the problem-solving process fed back by the server corresponding to the matched test question this time and use the preset first non-transparent mask to cover the problem-solving process, and display the The answer corresponding to the matched test question fed back by the server and using the preset second non-transparent mask to cover the answer; The device also includes: The elimination module is configured to, after displaying the problem-solving results fed back by the server and corresponding to the matched test questions this time, when receiving the first mask layer removal instruction, eliminate the problem under the instruction of the first mask layer removal instruction The first non-transparent mask layer; when receiving a second mask layer removal instruction, remove the second non-transparent mask layer under the instruction of the second mask layer removal instruction. 6. The device for searching a question as claimed in claim 5, wherein said intercepting module is specifically used for: Mapping the captured motion trajectory to a corresponding position on the screen, and when the trajectory mapped to the corresponding position on the screen forms a closed loop, intercepting a preview image in the closed loop. 7. The device for searching a question as claimed in claim 5, wherein said intercepting module is specifically used for: Mapping the captured motion trajectory to a corresponding position on the screen, and intercepting a preview image in the corresponding position on the screen when a photographing instruction is received. 8. The device for searching questions as claimed in claim 7, wherein said intercepting module is specifically used for: Map the captured motion trajectory to the corresponding position of the screen, and when the front camera captures the user's blinking action, trigger a camera instruction, and intercept the preview image in the corresponding position of the screen.