CN104618698A - Method and device for terminal control - Google Patents

Abstract

The present invention discloses a terminal control method and device. The method includes: acquiring a projection image formed when a part of the projection light path of a projection unit is blocked by a manipulated object through a video acquisition unit; Compared with the obtained original projection image corresponding to the obtained projection image, determine the position information of the shadow formed by the manipulation object in the obtained projection image relative to the original projection image; The information is sent to the current application of the terminal as manipulation data information, instructing the current application to perform a corresponding manipulation action according to the manipulation data information. In the technical solution of the present invention, without changing the terminal hardware configuration, a new control method based on terminal projection is proposed. On the basis of expanding the terminal control method, the flexibility of terminal control is improved. with the purpose of intelligence.

Description

A terminal control method and device

technical field

The present invention relates to the technical field of terminals, in particular to a method and device for controlling a terminal.

Background technique

Currently, common terminal control methods mainly include mouse control, keyboard control, touch screen control, and motion recognition control. Among them, the control methods such as mouse control and keyboard control are relatively traditional and low in intelligence; touch screen control is relatively high in intelligence, but because it can only be applied to terminal devices with touch screens, the cost of control is relatively high; motion recognition The intelligence of the control is also relatively high, but because its technology is not yet mature, and the recognition device with motion recognition capability needs to be connected as a peripheral device, the cost of control is relatively high. The camera for motion recognition needs to be set in front of the operator and facing the operator. In the case of projection, the camera will face the projection lens. The interference of strong light will greatly affect the accuracy of motion recognition. Therefore, , the control effect is not good.

That is to say, currently there are limited types of terminal control methods, and to a certain extent, there are problems such as low intelligence or poor control effects.

Contents of the invention

Embodiments of the present invention provide a method and device for controlling a terminal, which are used to solve problems such as low control intelligence or poor effect caused by limited types of terminal control methods.

An embodiment of the present invention provides a terminal manipulation method, including:

Obtaining a projection image formed when part of the projection light path of the projection unit is blocked by a manipulated object through the video capture unit;

Comparing the acquired projection image with the original projection image projected in the video memory of the terminal corresponding to the acquired projection image, and determining that the shadow formed by the manipulated object in the acquired projection image is relatively location information of the original projected image;

Sending the location information as manipulation data information to a current application of the terminal, instructing the current application to perform a corresponding manipulation action according to the manipulation data information.

Specifically, the video capture unit acquires the projection image formed when part of the projection light path of the projection unit is blocked by the manipulated object, including:

The projection image formed when a part of the projection light path of the projection unit is blocked by a manipulated object is obtained through the video capture unit by means of real-time acquisition or timing acquisition.

Wherein, the number of image acquisition frames used by the video capture unit to acquire the projected image is the same as the number of projection frames used by the projection unit to perform projection.

Further, the acquired projection image is compared with the original projection image corresponding to the acquired projection image, and it is determined that the shadow formed by the manipulation object in the acquired projection image is relative to the original projection image. location information, including:

converting the acquired projection image into an image of the same size as the original projection image, and when it is determined that the number of pixels satisfying the set condition in the converted projection image is greater than a set threshold, according to the Determine the location information of the shadow formed by the manipulated object in the acquired projection image relative to the original projection image in the region where the pixel points satisfy the set condition;

Wherein, the setting condition is: they are arranged continuously with each other and the projection color is darker than the deepest projection color that can be projected by the pixels in the corresponding area of the original projection image.

Further, the video capture unit and the projection unit are arranged on the same side of the terminal.

In the technical solution described in the embodiment of the present invention, the shadowed projection image formed when part of the projected light path of the terminal projection unit is blocked by the manipulated object can be captured by the video capture unit, and compared with the original projected image, it can be determined that the manipulated object is relatively The position information of the original projection image is converted into manipulation data information and then fed back to the current running application to achieve the purpose of manipulation and interaction. Therefore, without changing the terminal hardware configuration, a method based on The new control mode of terminal projection, on the basis of expanding the terminal control mode, achieves the purpose of improving the flexibility and intelligence of terminal control.

The embodiment of the present invention also provides a terminal control device, including:

a projection unit, configured to perform a projection operation on the image to be projected in the terminal;

A video capture unit, configured to acquire a projection image formed when part of the projection light path of the projection unit is blocked by a manipulated object;

A position determination unit, configured to compare the projection image acquired by the video capture unit with the original projection image corresponding to the acquired projection image projected in the terminal video memory, and determine that the manipulated object is within the acquired projection image. Position information of the shadow formed in the projected image relative to the original projected image;

An instruction sending unit, configured to send the position information as manipulation data information to a current application of the terminal, and instruct the current application to perform a corresponding manipulation action according to the manipulation data information.

Specifically, the video capture unit is specifically configured to acquire a projection image formed when a part of the projection light path of the projection unit is blocked by a manipulated object by means of real-time acquisition or timing acquisition.

Wherein, the number of image acquisition frames used by the video capture unit to acquire the projected image is the same as the number of projection frames used by the projection unit to perform projection.

Further, the position determination unit is specifically configured to convert the projection image acquired by the video capture unit into an image of the same size as the original projection image, and determine whether the converted projection image satisfies the set condition When the number of pixels is greater than the set threshold, according to the area where the pixels satisfying the set conditions are located, determine the position of the shadow formed by the manipulation object in the acquired projection image relative to the original projection image information;

Wherein, the setting condition is: they are arranged continuously with each other and the projection color is darker than the deepest projection color that can be projected by the pixels in the corresponding area of the original projection image.

Further, the video capture unit and the projection unit are arranged on the same side of the terminal.

The beneficial effects of the present invention are as follows:

The embodiment of the present invention provides a method and device for controlling a terminal. The video capture unit captures the shadowed projection image formed when part of the projected light path of the terminal projection unit is blocked by the manipulated object, and compares it with the original projected image to determine the control. The position information of the object relative to the original projected image, and this position information is fed back to the current running application as the control data information to achieve the purpose of control and interaction. Therefore, without changing the terminal hardware configuration, a method based on The new control mode of terminal projection, on the basis of expanding the terminal control mode, achieves the purpose of improving the flexibility and intelligence of terminal control.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings that need to be used in the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For Those of ordinary skill in the art can also obtain other drawings based on these drawings without making creative efforts.

FIG. 1 is a schematic flowchart of the terminal manipulation method described in Embodiment 1 of the present invention;

FIG. 2 is a schematic diagram 1 of the terminal manipulation method described in Embodiment 2 of the present invention;

FIG. 3 is a schematic diagram 2 of the terminal manipulation method described in Embodiment 2 of the present invention;

FIG. 4 is a schematic diagram 3 of the terminal manipulation method described in Embodiment 2 of the present invention;

FIG. 5 is a schematic diagram 4 of the terminal manipulation method described in Embodiment 2 of the present invention;

FIG. 6 is a schematic diagram 5 of the terminal manipulation method described in Embodiment 2 of the present invention;

FIG. 7 is a schematic structural diagram of the terminal control device in Embodiment 3 of the present invention.

Detailed ways

An embodiment of the present invention provides a terminal control method and device. The method includes: acquiring a projection image formed when a part of the projection light path of the projection unit is blocked by a manipulated object through a video acquisition unit; comparing the original projected image corresponding to the obtained projected image, to determine the position information of the shadow formed by the manipulated object in the obtained projected image relative to the original projected image; The location information is sent to the current application of the terminal as manipulation data information, instructing the current application to perform a corresponding manipulation action according to the manipulation data information. In the technical solution described in the embodiment of the present invention, without changing the hardware configuration of the terminal, a new control method based on terminal projection is proposed. On the basis of expanding the terminal control method, the terminal control is improved. The purpose of flexibility and intelligence.

In order to make the purpose, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only some of the embodiments of the present invention, rather than all of them. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

Embodiment one:

As shown in Figure 1, it is a schematic flow chart of the terminal control method described in Embodiment 1 of the present invention. The terminal control method can be applied to terminal devices such as mobile phones, tablet computers, and PDAs (Personal Digital Assistant, handheld computers). The embodiment of the invention does not make any limitation on this. Specifically, the terminal manipulation method may include the following steps:

Step 101: Obtain a projection image formed when part of the projection light path of the projection unit is blocked by a manipulated object through the video capture unit.

It should be noted that, in the embodiments of the present invention, the terminal applicable to the terminal manipulation method involved may generally be a terminal equipped with a video capture unit (such as a camera unit, etc.) and a projection unit, and usually Next, the video capture unit and the projection unit are always in the closed state. When using the terminal control method described in the embodiment of the present invention, the video capture unit and the projection unit can be set manually or automatically. is in an open state, which will not be described in detail in this embodiment of the present invention.

Specifically, the video capture unit and the projection unit can be arranged on the same side of the terminal; further, in order to avoid the video capture being blocked by the manipulation object caused when the projection lens of the projection unit is too close to the video capture lens of the video capture unit lens problem, you can set the video capture lens as an external lens, or you can set the distance between the video capture lens (such as the built-in video capture lens) and the projection lens to be greater than the set Threshold to keep a certain distance from the projection lens, so as to avoid a large overlap between the two optical paths; moreover, if the video capture lens (such as a built-in video capture lens) and the projection lens are located At both ends of the same side of the terminal, you can also change the way the terminal is placed (such as horizontal or vertical) to avoid manipulating objects from blocking the video capture lens; in addition, software intelligent recognition can also be used to filter out the effects of manipulating objects blocking the video capture lens. The interference caused so that the terminal control method can work normally in most states, which is not limited in this embodiment of the present invention.

Furthermore, in the embodiments of the present invention, real-time acquisition or timing acquisition can usually be adopted, and the video capture unit installed in the terminal can be used to acquire the partial projection optical path of the projection unit installed in the terminal when it is blocked by the manipulated object. The formed projected image. Preferably, in order to achieve the purpose of real-time acquisition of the projection image formed by the projection unit to improve the control accuracy, usually a real-time acquisition method can be used to obtain the projection image formed by the projection unit during projection, so as to instantly The projection image formed when part of the projection light path of the projection unit is blocked by the manipulation object is obtained, so as to complete the manipulation action of the terminal more accurately and in real time.

It should be noted that, when acquiring projection images, the number of image acquisition frames used by the video capture unit is the same as the number of projection frames used by the projection unit when performing projection, so as to ensure the consistency of image data.

Step 102: Compare the acquired projection image with the original projection image projected in the video memory of the terminal corresponding to the acquired projection image, and determine the shadow formed by the manipulated object in the acquired projection image Position information relative to the original projected image.

Specifically, in the embodiment of the present invention, the acquired projection image is compared with the original projection image corresponding to the acquired projection image, and it is determined that the manipulation object is in the acquired projection image The position information of the formed shadow relative to the original projected image may include:

converting the acquired projection image into an image of the same size as the original projection image, it should be noted that, when the acquired projection image is of the same size as the original projection image, the conversion does not need to be performed operation; and,

When it is determined that the number of pixels satisfying the set condition in the converted projection image is greater than the set threshold, determine the obtained projection of the manipulated object according to the area where the pixel points satisfying the set condition are located. The position information of the shadow formed in the image relative to the original projected image, wherein the setting condition is: they are arranged consecutively with each other (that is, the determined pixel points satisfying the setting condition are not mutually isolated pixel points) and the projection The color is darker than the deepest projected color that can be projected by the pixels in the corresponding area of the original projected image.

Specifically, the set threshold may be adjusted and set according to actual conditions, which is not limited in this embodiment of the present invention. For example, when the control object used to block the projection light path of the projection unit is an object with a small block area such as a finger or a wooden stick, or the terminal control is mostly performed by blocking the edge position of the screen, the The above-mentioned setting threshold is set to a relatively small value; and when the manipulation object used to block the projection light path of the projection unit is an object with a large blocking area such as a hand, or the method of blocking the center of the screen is often used Or when the terminal is manipulated in a way of covering with both hands, the set threshold may be set to a relatively large value, which is not limited in this embodiment of the present invention.

Step 103: Send the location information as manipulation data information to a current application of the terminal, instructing the current application to perform a corresponding manipulation action according to the manipulation data information.

Specifically, in the embodiment of the present invention, when the current application of the terminal receives the location information, it can intelligently analyze the location information according to the set software processing method, and according to the analyzed data, according to the set In order to achieve the purpose of manipulation and interaction, the corresponding manipulation actions can be performed in a certain manipulation mode; wherein, the manipulation actions can be toggle operations, pinch operations, slide operations, push operations, and window drag operations, etc., and the implementation of the present invention The example does not limit this.

Embodiment 1 of the present invention provides a terminal control method. The video capture unit captures the shadowed projection image formed when part of the projection light path of the terminal projection unit is blocked by the control object, and compares it with the original projection image to determine the control object. Relative to the position information of the original projected image, this position information is fed back to the current running application as control data information, so as to achieve the purpose of control and interaction. Therefore, without changing the terminal hardware configuration, a terminal-based The new control method of projection achieves the purpose of improving the flexibility and intelligence of terminal control on the basis of expanding the terminal control method.

Embodiment two:

Embodiment 2 of the present invention takes controlling a football on the terminal screen to complete a goal in a shooting game as an example, and specifically describes the process of the terminal control method described in Embodiment 1 of the present invention, as shown in FIGS. 2 to 6 , which are The schematic diagrams of each operation corresponding to the terminal operation method in Embodiment 2 of the present invention, the terminal operation method may specifically include the following steps:

Step 201: The terminal enters the shooting game, and turns on the projection unit and the video capture unit provided in the terminal, as shown in FIG. 2 .

At this time, the video capture unit can acquire the projection images formed by the projection unit during projection in real time, and combine each acquired projection image with the original projection projected in the terminal video memory. The images are compared; wherein, when acquiring the projection image, the number of image acquisition frames used by the video capture unit is the same as the number of projection frames used by the projection unit to ensure the consistency of the image data.

Step 202: block part of the projection light path of the projection unit by manipulating objects such as wooden sticks, so that shadows appear on the projection image acquired by the video capture unit, as shown in FIG. 3 .

At this time, the terminal can compare the shadowed projection image acquired by the video capture unit with the original projection image corresponding to the acquired shadowed projection image projected in the terminal video memory, and determine that the wooden stick is The position information of the shadow formed in the shadow projection image relative to the original projection image, and feed back the position information to the terminal shooting game program; the terminal shooting game program can obtain the stick position information according to the position information fed back by the terminal, But because the position information of the wooden stick is not in contact with the football in the image, the terminal shooting game program may not perform any operations.

Step 203: By moving the wooden stick, the shadow on the projected image acquired by the video capture unit can block the pixels next to the football, as shown in FIG. 4 .

At this time, since the stick position information obtained by the terminal shooting game program has certain contact with the football in the image, the shooting operation can be performed according to the set control mode, such as the toggle control mode, as shown in Figure 5 and Figure 6 shows.

Embodiment 2 of the present invention provides a terminal control method. The video capture unit captures the shadowed projection image formed when part of the projection light path of the terminal projection unit is blocked by the control object, and compares it with the original projection image to determine the control object. Relative to the position information of the original projected image, this position information is fed back to the current running application as control data information, so as to achieve the purpose of control and interaction. Therefore, without changing the terminal hardware configuration, a terminal-based The new control method of projection achieves the purpose of improving the flexibility and intelligence of terminal control on the basis of expanding the terminal control method.

Embodiment three:

As shown in Figure 7, it is a schematic structural diagram of the terminal control device described in Embodiment 3 of the present invention. The terminal control device can be an independent terminal device such as a mobile phone, a tablet computer, or a PDA, or a device integrated in each of the above-mentioned terminal devices. The integrated device is not limited in this embodiment of the present invention. Specifically, the terminal control device may include a projection unit 11, a video capture unit 12, a position determination unit 13, and an indication sending unit 14, wherein:

The projection unit 11 may be used to project an image to be projected in the terminal; wherein, the image to be projected may be a dynamic image or a static image, which is not limited in this embodiment of the present invention.

The video capture unit 12 can be used to acquire a projection image formed when a part of the projection optical path of the projection unit 11 is blocked by a manipulation object, wherein the manipulation object can be a finger, a stick, etc. Do not make any restrictions.

Specifically, the video capture unit 12 can be used to obtain the projection image formed when part of the projection light path of the projection unit 11 is blocked by the manipulated object by means of real-time acquisition or timing acquisition; The projection image formed by 11 is collected in real time to improve the accuracy of manipulation. The video capture unit 12 can usually acquire the projection image formed by the projection unit 11 in real time in order to obtain the real-time The projection image formed when part of the projected optical path to the projection unit 11 is blocked by the manipulated object, thereby completing the terminal's manipulation action more accurately and in real time. Wherein, the number of image acquisition frames used by the video capture unit 12 to acquire the projected image is usually the same as the number of projection frames used by the projection unit 11 when performing projection, so as to ensure the consistency of image data.

The position determination unit 13 can be configured to compare the projection image acquired by the video capture unit 12 with the original projection image corresponding to the acquired projection image projected in the terminal video memory, and determine that the manipulated object is Position information of the shadow formed in the acquired projection image relative to the original projection image.

Specifically, the position determination unit 13 can be used to convert the projection image acquired by the video capture unit 12 into an image of the same size as the original projection image, and determine whether the converted projection image satisfies the set condition When the number of pixels is greater than the set threshold, according to the area where the pixels satisfying the set conditions are located, determine the position of the shadow formed by the manipulation object in the acquired projection image relative to the original projection image information, wherein, the setting condition is: they are arranged consecutively with each other and the projection color is deeper than the deepest projection color that can be projected by the pixels in the corresponding area of the original projection image.

The instruction sending unit 14 may be configured to send the location information as manipulation data information to a current application of the terminal, and instruct the current application to perform a corresponding manipulation action according to the manipulation data information.

Specifically, in the embodiment of the present invention, when the current application of the terminal receives the location information, it can intelligently analyze the location information according to the set software processing method, and according to the analyzed data, according to the set In order to achieve the purpose of manipulation and interaction, the corresponding manipulation actions can be performed in a certain manipulation mode; wherein, the manipulation actions can be toggle operations, pinch operations, slide operations, push operations, and window drag operations, etc., and the implementation of the present invention The example does not limit this.

Embodiment 3 of the present invention provides a terminal control device, which uses a video capture unit to capture a shadowed projection image formed when part of the projection light path of the terminal projection unit is blocked by a control object, and compares it with the original projection image to determine the control object Relative to the position information of the original projected image, this position information is fed back to the current running application as control data information, so as to achieve the purpose of control and interaction. Therefore, without changing the terminal hardware configuration, a terminal-based The new control method of projection achieves the purpose of improving the flexibility and intelligence of terminal control on the basis of expanding the terminal control method.

Those skilled in the art should understand that the embodiments of the present invention may be provided as methods, devices (devices), or computer program products. Accordingly, the present invention can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (devices) and computer program products according to embodiments of the invention. It should be understood that each procedure and/or block in the flowchart and/or block diagram, and a combination of procedures and/or blocks in the flowchart and/or block diagram can be realized by computer program instructions. These computer program instructions may be provided to a general purpose computer, special purpose computer, embedded processor, or processor of other programmable data processing equipment to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing equipment produce a An apparatus for realizing the functions specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.

These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to operate in a specific manner, such that the instructions stored in the computer-readable memory produce an article of manufacture comprising instruction means, the instructions The device realizes the function specified in one or more procedures of the flowchart and/or one or more blocks of the block diagram.

These computer program instructions can also be loaded onto a computer or other programmable data processing device, causing a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby The instructions provide steps for implementing the functions specified in the flow chart or blocks of the flowchart and/or the block or blocks of the block diagrams.

While preferred embodiments of the invention have been described, additional changes and modifications to these embodiments can be made by those skilled in the art once the basic inventive concept is appreciated. Therefore, it is intended that the appended claims be construed to cover the preferred embodiment as well as all changes and modifications which fall within the scope of the invention.

Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention also intends to include these modifications and variations.

Claims (10)

1. A terminal manipulation method, characterized in that, comprising: Obtaining a projection image formed when part of the projection light path of the projection unit is blocked by a manipulated object through the video capture unit; Comparing the acquired projection image with the original projection image projected in the video memory of the terminal corresponding to the acquired projection image, and determining that the shadow formed by the manipulated object in the acquired projection image is relatively location information of the original projected image; Sending the location information as manipulation data information to a current application of the terminal, instructing the current application to perform a corresponding manipulation action according to the manipulation data information. 2. The terminal control method according to claim 1, wherein the video capture unit acquires the projection image formed when part of the projection light path of the projection unit is blocked by the manipulation object, comprising: The projection image formed when a part of the projection light path of the projection unit is blocked by a manipulated object is obtained through the video capture unit by means of real-time acquisition or timing acquisition. 3. The terminal manipulation method according to any one of claims 1 or 2, characterized in that: The number of image acquisition frames used by the video capture unit to acquire the projected image is the same as the number of projection frames used by the projection unit to perform projection. 4. The terminal manipulation method according to any one of claims 1 or 2, wherein the acquired projection image is compared with the original projection image corresponding to the acquired projection image to determine the manipulation object The position information of the shadow formed in the acquired projection image relative to the original projection image includes: converting the acquired projection image into an image of the same size as the original projection image, and when it is determined that the number of pixels satisfying the set condition in the converted projection image is greater than a set threshold, according to the Determine the location information of the shadow formed by the manipulated object in the acquired projection image relative to the original projection image in the region where the pixel points satisfy the set condition; Wherein, the setting condition is: they are arranged continuously with each other and the projection color is darker than the deepest projection color that can be projected by the pixels in the corresponding area of the original projection image. 5. The terminal manipulation method according to any one of claims 1 or 2, characterized in that: The video capture unit and the projection unit are arranged on the same side of the terminal. 6. A terminal control device, characterized in that it comprises: a projection unit, configured to perform a projection operation on the image to be projected in the terminal; A video capture unit, configured to acquire a projection image formed when part of the projection light path of the projection unit is blocked by a manipulated object; A position determining unit, configured to compare the projection image acquired by the video capture unit with the original projection image corresponding to the acquired projection image projected in the terminal video memory, and determine that the manipulated object is within the acquired projection image. Position information of the shadow formed in the projected image relative to the original projected image; An instruction sending unit, configured to send the position information as manipulation data information to a current application of the terminal, and instruct the current application to perform a corresponding manipulation action according to the manipulation data information. 7. The terminal control device according to claim 6, characterized in that: The video capture unit is specifically configured to acquire a projection image formed when a part of the projection light path of the projection unit is blocked by a manipulated object by means of real-time acquisition or timing acquisition. 8. The terminal control device according to any one of claims 6 or 7, characterized in that, The number of image acquisition frames used by the video capture unit to acquire the projected image is the same as the number of projection frames used by the projection unit to perform projection. 9. The terminal control device according to any one of claims 6 or 7, characterized in that, The position determination unit is specifically configured to convert the projection image acquired by the video capture unit into an image of the same size as the original projection image, and determine the pixel points in the converted projection image that meet the set conditions When the number is greater than the set threshold, according to the area where the pixels satisfying the set condition are located, determine the position information of the shadow formed by the manipulation object in the acquired projection image relative to the original projection image; Wherein, the setting condition is: they are arranged consecutively with each other and the projection color is darker than the deepest projection color that can be projected by the pixels in the corresponding area of the original projection image. 10. The terminal control device according to any one of claims 6 or 7, wherein the video capture unit and the projection unit are arranged on the same side of the terminal.