CN105204627B - A digital input method based on gesture - Google Patents

Abstract

The invention provides a kind of digital input method based on gesture, belong to computer realm.The method structure virtual interface, described virtual interface is operator's gesture operation region, and this virtual interface can change along with the body position of operator or the change of figure；Gradually input N number of numeral by a virtual interface, or by generating N number of virtual interface, each virtual interface one numeral of input, this N number of virtual interface forms virtual interface group；Display screen is defined as physical interface, and the definition space between operator and physical interface is physical space；Described physical space includes virtual interface and non-virtual interface；The gesture of operator be only only in virtual interface effective and can perception, the gesture in non-virtual interface is invalid；Numeral is inputted by described virtual interface group.

Description

A digital input method based on gesture

technical field

The invention belongs to the field of computers, and in particular relates to a gesture-based digital input method.

Background technique

Inputting numbers with gestures has always been one of the difficult and key issues in gesture interaction theory and application research, and it is also one of the pain points in industrial fields such as smart home appliances.

At present, gesture numbers are generally used to input numbers with different gestures. It first recognizes gesture images, and then interprets different gestures as different numbers. The main problems of this method are: (1) The recognition rate is affected by many factors. (such as distance, feature extraction, recognition operator, etc.), especially for gestures with similar shapes, it is difficult to distinguish them by ordinary methods; (2) when multiple people operate at the same time, they are easy to interfere with each other; (3) there is a "MidasTouch problem" .

Contents of the invention

The purpose of the present invention is to solve the problems existing in the above-mentioned prior art, and to provide a gesture-based number input method, which is convenient for inputting numbers with gestures.

The present invention is achieved through the following technical solutions:

A digital input method based on gestures, constructing a virtual interface, the virtual interface is the operator's gesture operation area, and the virtual interface can change with the change of the operator's body position or posture; input N numbers successively through a virtual interface number, or by generating N virtual interfaces, each virtual interface enters a number, and these N virtual interfaces form a virtual interface group;

The display screen is defined as the physical interface, and the space between the operator and the physical interface is defined as the physical space;

The physical space includes a virtual interface and a non-virtual interface; the operator's gestures are valid and perceivable only in the virtual interface, and gestures in the non-virtual interface are invalid;

Enter numbers through the cluster of virtual interfaces.

The construction of the virtual interface is implemented as follows:

(1) The computer detects the behavior model of the operator: if it is detected that the palm is first pushed forward and then remains stationary, then enter step (2), if it is other behaviors, then return to step (1);

(2) Calculate the position Z of the center of gravity of the gesture when the palm remains stationary;

(3) Put the virtual interface template M at a position centered on the center of gravity position Z of the gesture to obtain a virtual interface V(Z, Ω _M ), where Ω _M represents the scope of the virtual interface determined by the template M;

(4) Calculate the three-dimensional position information of each point of interest on V;

(5) Return to the virtual interface V.

The function distribution and size range of the virtual interface template M of each application are fixed; the interest points refer to the interactive objects of the operator on the virtual interface V;

The step (4) is to use the spatial depth information and gesture tracking method to calculate the three-dimensional position information of each point of interest on V.

The input of numbers through the virtual interface group is realized in this way:

Q1, using the clock dial structure to represent the virtual interface, wherein the 12 o’clock in the clock dial structure is defined as the number 0, and then the numbers 1 to 9 are sequentially defined in the clockwise direction, and the arc between the number 1 and the number 2 is defined. The midpoint is A, the midpoint of the arc between the number 2 and the number 3 is B, the center of the circle is Z, and the rays ZA and ZB enclose a fan-shaped area Θ;

Q2, for each number to be entered, the specific steps include:

(A1) generate the i-th virtual interface Vi;

(A2) The operator moves gestures on Vi;

(A3) If the gesture is in a static state, further detect whether there is a change from the bag gesture with five fingers stretched to the fist gesture with five fingers contracted (ie "grab" gesture), if yes, then enter (A4), if not, return to step (A3);

(A4) Calculate the position Pg of the center of gravity of the gesture;

(A5) If Go to step (A4);

(A6) Calculate f:

$\underset{\mathrm{<span\; class="notranslate">\; f</span>}}{\mathrm{<span\; class="notranslate">\; m</span>}\mathrm{<span\; class="notranslate">\; i</span>}\mathrm{<span\; class="notranslate">\; no</span>}}<span\; class="notranslate">\; \{</span>\mathrm{<span\; class="notranslate">\; d</span>}\mathrm{<span\; class="notranslate">\; i</span>}\mathrm{<span\; class="notranslate">\; the\; s</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; g</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; \&Theta;</span>}}_{\mathrm{<span\; class="notranslate">\; f</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \}</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>$

Among them, Θ _f represents the point set of the space fan-shaped area where the number f is located on the virtual interface V _i .

Said Q1 further includes:

Divide the 9 fan-shaped areas of numbers 0 to 9 on the virtual interface into 3 quadrants, where quadrant I is the fan-shaped area where 0-3 is located; quadrant II is the fan-shaped area where 3-6 is located; quadrant III is where 6-9 is located fan-shaped area;

The operator's gesture starts from the center point of the virtual interface to select a quadrant, and then judges the quadrant that the operator wants to select according to the direction range of the gesture trajectory;

The ray determined by the center point Z and the number 0 is the starting vector L, and the angle gradually increases in the clockwise direction. The direction of gesture movement is T, and the angle between T and L is θ=<T,L>, where T is The quadrants are:

When making digital selection, the operator’s palm moves along the T direction, and then pushes the gesture vertically forward, then the quadrant determined by T is selected, and then the corresponding virtual interface template is called, and the quadrant is set as a new virtual interface for operation or further operation;

In said new virtual interface, there are only 4 numbers in that quadrant.

The method adopts a particle filter algorithm to obtain gesture motion trajectories.

Compared with prior art, the beneficial effect of the present invention is:

(1) Solve the "MidasTouch problem" (that is, the problem of "touching stones into gold", all gestures of the user will be captured by the sensor and executed as commands, resulting in system state disorder, which greatly increases the load on cognitive users and operations).

(2) It solves the defects (for example, low recognition rate; complex gestures, inconvenient input of multi-digit numbers, and the need to memorize digital gestures, etc.) when traditionally using gestures to input numbers.

Description of drawings

Fig. 1 is used for the virtual interface template of digital input in the present invention

Fig. 2 virtual interface template in the embodiment of the present invention

10 numbers in the embodiment of the present invention are divided into three quadrants in Fig. 3-1

Figure 3-2 In the embodiment of the present invention, the first quadrant is used as a virtual interface template to generate a new virtual interface.

detailed description

Below in conjunction with accompanying drawing, the present invention is described in further detail:

(1) Virtual interface

By analyzing the operator's gesture operation behavior, the computer can form a gesture interaction sensing area in front of the operator, which may be two-dimensional (2D) or three-dimensional (3D), and it can follow the operator's body position or The change of posture changes step by step, as if there is an invisible operation screen near the operator's gesture that can change with the movement of the operator's body. The present invention refers to this operator's gesture operation area with specific structure and function as (Intangible) Virtual Interface (TI). Divide the virtual interface into several sub-blocks, combine multi-modal gestures (ie, 2D interface gestures/3D interface gestures, communication gestures/operating gestures) to build basic interactive functions, and build the basic interactive functions between these sub-blocks and interactive functions To establish a corresponding relationship, such a sub-block is called a function block.

The role of the virtual interface is that it not only reflects the behavior model of the operator's gesture operation from one side, but also portrays the operator's mental model from one side, and makes the non-contact interactive interface structured and perceivable , can be calculated, so that the contact interaction and non-contact interaction can be effectively unified.

(2) Physical space

In the present invention, the display screen is called a physical interface (PI), and the space between the operator and the physical interface is called a physical space. The physical space is divided into virtual interface and non-virtual interface. A possible physical space structure diagram is given, in which the virtual interface is divided into 3D operation area and 2D operation area, and operator gestures are effective and perceivable only in the virtual interface; The space is an invalid gesture command area, and the computer will not respond to any gesture commands or gesture operations in the invalid gesture command area.

The construction algorithm of the virtual interface is as follows:

1. The computer checks the operator's behavior model: the palm is first pushed forward, and then held still.

2. Calculate the position Z of the center of gravity of the gesture when the palm remains stationary;

3. Place the pre-designed virtual interface template M at a position centered on Z to obtain a virtual interface V(Z, Ω _M ), where Ω _M represents the scope of the virtual interface determined by the template M;

4. Use spatial depth information and gesture tracking technology to calculate the three-dimensional position information of each point of interest on V;

5. Return to the virtual interface V.

The function distribution and size range of the virtual interface template M are determined and unchanged. According to different application needs, different virtual interface templates M can be designed. The so-called point of interest on V refers to the interactive object of the operator on the virtual interface V.

The digital input algorithm based on the virtual interface group is as follows:

1 Algorithm Description

In order to input N-digit numbers through the virtual interface, N virtual interfaces need to be generated, thereby forming a group of virtual interfaces. Among them, the template of each virtual interface is the same, adopting a similar clock dial structure familiar to the operator. Among them, the "12" point is defined as 0, and then 1 to 9 are defined in the clockwise direction, as shown in Figure 1, in the figure, along the clockwise direction, the midpoint of the arc between point 1 and point 2 is A, the midpoint of the arc between point 2 and point 3 is B, and rays ZA and ZB enclose a fan-shaped area Θ. Obviously, the necessary and sufficient condition for selecting point 2 is that the center of gravity of the gesture falls within the area determined by the area Θ and the "selection" behavior of the operator is detected. The larger the arc of Θ, the lower the error rate of the selection.

Number input algorithm:

0: For(i=1; i≤N; i++)

{

1. Generate the i-th virtual interface Vi;

2. The operator moves gestures on Vi;

3. If the gesture is at rest:

4. Is the "grab" gesture detected? If "No", go to step 4.

5. Calculate the position Pg of the center of gravity of the gesture;

6. If Go to step 5;

7. Calculate f:

$\underset{\mathrm{<span\; class="notranslate">\; f</span>}}{\mathrm{<span\; class="notranslate">\; m</span>}\mathrm{<span\; class="notranslate">\; i</span>}\mathrm{<span\; class="notranslate">\; no</span>}}<span\; class="notranslate">\; \{</span>\mathrm{<span\; class="notranslate">\; d</span>}\mathrm{<span\; class="notranslate">\; i</span>}\mathrm{<span\; class="notranslate">\; the\; s</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; g</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; \&Theta;</span>}}_{\mathrm{<span\; class="notranslate">\; f</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \}</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>$

Wherein, Θf represents the point set of the space fan-shaped area where the number f is located on the virtual interface Vi;

8. Perceive whether f is correct, if the selection result f is wrong, enter the error correction process; otherwise, get the selection result f of Vi;

}

At the same time, two virtual interfaces as shown in Figure 2 are used to realize double-digit input, which can be used for smart TV channel input. For example, if the operator wants to input channel 48, he only needs to input the number "4" through one virtual interface, and then input the number "8" through another virtual interface.

2 Further improvement of the algorithm

Considering that the range of motion of gestures is often limited, and it can neither increase the operator's operating load nor affect the naturalness of the interaction, the size of the virtual interface is often limited, which determines the distance between two adjacent numbers. is relatively small, so the selection accuracy of the above algorithm will be greatly affected, which will inevitably increase the operator's operational load and cognitive load. In order to solve this problem, the present invention divides the 9 fan-shaped areas from 0 to 9 on the virtual interface into 3 quadrants (Fig. 3-1). Quadrant I: the fan-shaped area where 0-3 is located; Quadrant II: the fan-shaped area where 3-6 is located; Quadrant III: the fan-shaped area where 6-9 is located. If it is specified that the gesture starts from the center point of the virtual interface to select a quadrant, then the quadrant to be selected by the operator can be determined according to the direction range of the gesture movement. For example, it is stipulated that the ray determined by Z and the number 0 is the starting vector L, the angle gradually increases in the clockwise direction, the direction of the gesture movement is T, and the angle between T and L is θ=<T,L>. Then the quadrant where T is located is

When selecting a number, first extract the quadrant where the required number is located and place it in a new virtual interface (translate the palm along the T direction, and then push the gesture vertically forward, the quadrant determined by T will be selected and transferred to the corresponding The template of the virtual interface forms a new virtual interface for the operator to further operate), in the new virtual interface, since there are only 4 numbers, the distance between the numbers is relatively large (equivalent to enlarging the arc length of the fan-shaped area), therefore, The precision of select operations is improved (Figure 3-2)

In Figure 3-1, the 10 numbers are divided into three quadrants, and the operator’s choice of quadrant can be determined by implicit interaction technology through the direction of gesture movement. Figure 3-2 When the first quadrant is selected, use this quadrant as a virtual interface template to generate a new virtual interface, and the accuracy of selecting numbers from this virtual interface will be improved.

The present invention adopts the particle filter algorithm to obtain the gesture trajectory:

S1: Initialization.

Select the particle set according to the prior distribution p(X ₀ ) of the state of the center of gravity of the gesture:

k=1;

S2: State sampling.

S2.1: Fori=1ToN

According to the prior distribution get a sample

S2.3: Find the weight of the sample:

Fori=1ToN

${\mathrm{<span\; class="notranslate">\; \&omega;</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; \&omega;</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span>}\frac{\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; Y</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; )</span>\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; )</span>}{\mathrm{<span\; class="notranslate">\; \&pi;</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; Z</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; )</span>}$

S2.3: Standardize the weights:

Fori=1ToN

${\mathrm{<span\; class="notranslate">\; \&omega;</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; \&omega;</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span>}}{\underset{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; N</span>}}{<span\; class="notranslate">\; \&Sigma;</span>}}{\mathrm{<span\; class="notranslate">\; \&omega;</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span>}}$

S3: State Estimation:

${\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; x</span>}}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; N</span>}}{<span\; class="notranslate">\; \&Sigma;</span>}}{\mathrm{<span\; class="notranslate">\; \&omega;</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span>}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span>}$

S4: resampling:

Resample the sample to produce a new set of samples makes:

$\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; (</span>{\stackrel{<span\; class="notranslate">\; ~</span>}{\mathrm{<span\; class="notranslate">\; x</span>}}}_{\mathrm{<span\; class="notranslate">\; k</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; \&omega;</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}$

That is, in the new sample The probability of occurrence is ω _i

S5: k=k+1, go to S2.

Gesture recognition algorithm:

The gesture recognition method based on the shape context of the present invention realizes the gesture recognition by comparing the shape context features corresponding to the gesture points on the gesture image.

First build the gesture database.

(1) Select m gestures, select n gesture images for each gesture, m=5, n=10; that is, select 5 gestures, and select 10 gesture images for each gesture.

(2) Find out the gesture point in each gesture image; traverse the whole image, if it is black, it is considered as the background; otherwise, it is considered as the gesture point, and record the coordinates of the gesture point and the number of gesture points.

(3) Calculate the center of gravity of the gesture and the maximum distance between the center of gravity of the gesture and the gesture point;

(4) Make a circle with the maximum distance as the maximum radius, and then divide the maximum radius into k parts on average to make k concentric circles, where k is 12, and a ring is formed between two adjacent concentric circles; statistics fall Gesture points in each ring, and then calculate the center point of each ring as the gesture feature point;

(5) Extract shape context features on the basis of gesture feature points and gesture points, and finally write shape context features into text files and store them in the gesture database; there are 50 files in the gesture database, 10 files for each gesture.

Among them, the function of counting gesture points voidHandsDetection(D2POINTedgepoint[], BYTE*image, int*HandpointsNO) The main function of this function is to count gesture points based on the segmented gesture image, record their coordinates, and return the number of gesture points.

D2POINT is a structure type, and the definition of this structure type is: structD2POINT{intx; inty;};

Input: pointer image to the image to be processed.

Output: The return value of the function is the edgepoint[] that stores the coordinates of the gesture point and the edgepoint[] used to record the gesture point

Number of HandpointsNO.

Specific steps:

① Traverse each pixel on the image in order from top to bottom and from left to right.

②If the pixel is black, end this cycle, and then continue traversing; if it is not black, store the x-coordinate and y-coordinate of the pixel in the array edgepoint[], and add 1 to the number of gesture points HandpointsNO .

③ Repeat step ② until the image traversal ends.

④ Return the number of gesture points HandpointsNO.

Statistical ring center point function voidCountRing(D2POINTedgepoint[],D2POINTfeaturedot[],D2POINTsumpoints[],intHandpointsNO,intcircleno)

The function of this function is to calculate the center point of each ring.

Input: edgepoint[] that stores gesture point coordinates, HandpointsNO that stores the number of gesture points, circleno that stores the number of circles, and the value of circleno is 12.

Output: return the array featuredot[] that stores the coordinates of the center point of the ring, the array sumpoints[] that stores the coordinates of the center point of the ring and the gesture point,

Specific steps:

① From the gesture point coordinates edgepoint[] and the value of HandpointsNO, find the weight of the center of gravity coordinates of the gesture.

②Find the maximum distance maxjuli from the center of gravity to the gesture point array edgepoint[].

③Use the maximum distance maxjuli as the maximum radius, and divide this radius into 12 parts on average to determine 12 circles.

④ Count the coordinates of the gesture points in each ring and the number of gesture points falling in the ring according to the range of the radius, and store them in the member variables D2POINTshixinpoint[200] and no of the array ring[12] respectively. The type of the array ring[12] is the structure CircleRing. The definition of this structure type is: structCircleRing{D2POINTshixinpoint[200]; // point coordinates intno inside the ring; // number of points inside the ring D2POINTavg; // center point coordinates of the ring};

⑤ On the basis of the fourth step, calculate the center point of each ring. If the number of points in the ring, that is, the value of no is not 0, the center point of the ring is calculated by the points falling in the ring, and stored in the member variable D2POINTavg of the array ring[12]. If the value of no is 0, the x and y coordinates of the center point of the ring are both set to 0.

⑤ Copy the center point of each ring to the array featuredot[] as the gesture feature point. Copy the ring center point and gesture point to the array sumpoints[] to prepare for the subsequent shape context feature extraction.

Shape context feature extraction function voidShapeContext(intFeatureNo[][60], D2POINTfeaturedots[], D2POINTsumpoints[], intHandpointsNO, intcircleno) The main function of this function is to obtain the shape context feature of gesture feature points.

Input: featuredots[] is the gesture feature point, that is, the center point of the circle, sumpoints[] is the collection of gesture feature points and gesture points, HandpointsNO is the number of gesture points, and circleno is the number of circles.

Output: an array FeatureNo[][60] used to store the shape context features of each gesture feature point.

Specific steps:

Perform the following operations on each gesture feature point.

① Find the maximum distance maxdistance from the current gesture feature point to the midpoint of the array sumpoints[].

② If the x and y coordinates of the gesture feature point are not 0, execute the following algorithm. As shown in the figure below: take the current gesture feature point as the pole, and the maximum distance maxdistance as the radius, divide the plane space into 60 regions. The specific division method is as follows: take the current gesture feature point as the pole to construct a polar coordinate system, divide the entire plane space into 12 directions on average, and divide it into 5 parts evenly on the radius. Therefore, the entire plane space is naturally divided into 60 areas. On the same ring, the area of each area is equal, and then the number of points in the array sumpoints[] falling in each area is counted.

The 60 attribute values of the i-th gesture feature point can form a sequence (a _i,1 ,a _i,2 ,...,a _i,60 ), so an n*60 shape matrix can be used to perform image shape describe:

The meaning of this matrix is: for each matrix element a _i,j , i represents the i-th feature point, j represents the j-th area in the 60 areas, and the meaning of a _i,j is: take the i-th feature point As the pole, establish a polar coordinate system, and the number of points falling in the jth area. The value of n is the total number of feature points, where the value of n is 12, because there are 12 center points of the ring, that is, gesture feature points. This matrix is the contextual feature representing the shape of the image. Save the values of this matrix in the two-dimensional array FeatureNo[][60]. If the x and y coordinates of the gesture feature point are both 0, all 60 attribute values of the feature point are set to 0.

Then perform gesture recognition.

(1) Read 50 gesture database files sequentially and save them in an array.

(2) Continuously select F frames of gesture images to be recognized from the video stream, F is 10, starting from the tenth frame, continuously take 10 frames of images from the video stream as gesture images to be recognized.

(3) adopting the same method as above to calculate the shape context feature of each frame of the image to be recognized in real time;

( ⁴ ) Calculate the χ2 distance between the shape context features of each frame of gesture images to be recognized and the shape context features of m*n gesture images in the gesture database, and then use each gesture image in the gesture database to participate in the calculation. All χ ² distances add up and are stored in an array, and each frame of the gesture image to be recognized corresponds to saving m*n χ ² distances and an array, and the Sort function is used to obtain m*n χ ² distances and the minimum value A of the array;

(5) According to the above-mentioned method, calculate respectively F χ ² distances and the minimum value A of the array corresponding to F frames of gesture images to be recognized, and then adopt the Sort function to find the minimum value A of the F χ ² distances and the array value B, the gesture stored in the gesture database corresponding to the minimum value B is the recognized gesture.

Read gesture template library file function voidreadfile(inttemplet[50][20][60])

The function of this function is to read the already built gesture database file and save it in the array templet[50][20][60]. Among them, the value of the first dimension is the file label, and there are 5 kinds of gestures, and there are 10 gesture library files for each gesture; the value of the second dimension represents the number of gesture feature points; the value of the third dimension represents the characteristics of each gesture The values of the 60 features corresponding to the points.

Output: An array templet[50][20][60] holding all template library files.

Specific steps:

① There are 5 gestures in total, and 10 files for each gesture, so there are 50 gesture library files in total. Read each file into the array templet[50][20][60] in sequence. The value of the first dimension of the array templet[50][20][60] represents different gestures. Among them, 0-9 is the bag, 10-19 is the scissors, 20-29 is the ok gesture, 30-39 is the fist, and 40-49 is the thumb.

② Read each file sequentially. If the read value is -1, it is considered that the reading of the file has ended, and the flag is set to 1 as the end sign of the reading of the file.

Gesture recognition function voidIdensitify(intfeaturecon[][60],floatchengben[],intn,inttemplet[50][20][60],intcircleno)

The function of this function is to compare the shape context feature of the gesture frame image to be recognized with the shape context feature in one of the 10 library files of a certain gesture in the gesture library, and find the matching cost.

Input: featurecon[][60] is the obtained shape context feature of the gesture frame image to be recognized, and templet[50][20][60] is used to store the value of the shape context feature read from the gesture library file. n is the label of the file, which means the gesture to be recognized is compared with the nth gesture library file. Because there are 50 files in total, the value of n takes 0-49. circleno is the number of circles.

Output: Returns the array chengben[] that stores the matching cost. Because there are 12 gesture feature points in total, the array has 12 values in total.

Specific steps:

① Traverse the array featurecon[][60] pointing to the gesture shape context feature to be recognized by row, and traverse the array templet[50][20][60] storing the shape context feature of the gesture library at the same time.

② If the 60 attribute values of the gesture feature point are all 0, assign the 60 attribute values of the corresponding gesture feature point in the gesture library file to it. Then compare the 60 attribute values of the gesture feature point with the shape context feature of each feature point in the gesture library file, because there is a matching cost between the feature point and the feature of each feature point in the gesture library, Therefore, a total of 12 matching cost values are obtained, and then the minimum value is taken among these 12 values as an element value of the array chengben[]. ^The matching cost is defined by the χ2 distance. ^The χ2 distance is defined as:

${\mathrm{<span\; class="notranslate">\; C</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; 2</span>}}\underset{\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; K</span>}}{<span\; class="notranslate">\; \&Sigma;</span>}}\frac{{<span\; class="notranslate">\; \&lsqb;</span>{\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&rsqb;</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}{{\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; )</span>}$

h _i (k) in the formula is the shape context feature value of the i-th gesture feature point in the gesture to be recognized, and h _j (k) is the shape context feature value of the j-th gesture feature point in a certain gesture library file, where The value of k is 1-60, representing one of the 60 attribute values. Through this formula, the matching cost value C _ij is obtained, that is, the matching cost between the two feature points i and j.

③ According to the above method, each gesture feature point is traversed, so 12 matching cost values are finally obtained, which are stored in the array chengben[].

Sort function IdensityFlagSort(IdensityFlaggross[], intn)

^The function of this function is to find the minimum value of the calculated χ2 distance sum.

IdentityFlag is a structure type, defined as follows:

tructIdensityFlag { floatsum; intflag; };

Input: The member variable sum in the array gross[] is the total matching cost, that is, the sum of matching costs between the gesture to be recognized and a certain gesture library file. The member variable flag represents the gesture label. 0 is burden, 1 is scissors, 2 is ok, 3 is fist, 4 is thumb.

Output: The variable mark of type IdentityFlag is returned. mark is the smallest of all matching cost values.

Specific steps:

①Define a variable mark of type IdensityFlag to store the minimum matching cost sum, and assign the value of the first element in the array gross[] to mark.

② Traverse the array gross[] in turn, if the sum value of the member variable of the element is less than the sum value of the mark member variable, assign the element to mark.

③Execute ② repeatedly until the traversal ends. Returns the value of mark.

voidCMainFrame::TotalIdensity(BYTE*lpImgData[],inttemplet[50][20][60]) The main function of this function is a total recognition process of 10 frames of gesture images obtained in real time.

Input: BYTE*lpImgData[] points to the obtained 10-frame image, inttemplet[50][20][60] stores the gesture template value.

Specific steps:

For each frame of image, the following operations are performed:

① Obtain the gesture point through the HandsDetection function. If the number of gesture points in the frame is 0, discard the frame. If it is not 0, it is a valid frame, and use frameNo to indicate the number of valid frames. Then perform the following calculations.

② Count the center points in each ring through the CountRing function.

③ Calculate the shape context feature of the frame gesture image through the ShapeContext function.

④Comparing the shape context features of the frame gesture image with the shape context features in all template libraries through the Idensitify function, so as to obtain 50 χ ² distance sums.

⑤ In the 50 χ ² distance sums, find the minimum value through the Sort function.

Perform steps ①-⑤ in a loop to process 10 frames of images respectively. Because each effective frame corresponds to a minimum value of the sum of χ ² distances, n effective frames correspond to n χ ² distances. Take the minimum value again through the Sort function among the n χ2 distances. The gesture corresponding to the minimum value is the recognized gesture

The method of the present invention has the advantages that: (1) the concept of the virtual interface template conforms to people's cognitive behavioral model and mental model, and helps to condense a unified and standardized interactive interface paradigm; (2) people's cognition of the clock dial interface is deeply rooted It is true that associating the 10 numbers from 0 to 9 through the clock dial is in line with people's daily life experience, requiring people to "master" this structure and interact with it has a natural cognitive and practical basis, so that even if the clock dial interface Further divided into three quadrants, it will not increase the cognitive load of the operator; (3) the virtual interface concept effectively solves the "MidasTouch problem" and can realize multi-person interaction; (4) overcomes the limitations of existing digital gesture methods. Disadvantages, especially it avoids the puzzlement of gesture recognition rate; (5) can realize the input of multi-digit number conveniently, has the advantage of fast speed, low error rate, simple and convenient, natural and efficient.

The above-mentioned technical solution is only an embodiment of the present invention. For those skilled in the art, on the basis of the application methods and principles disclosed in the present invention, it is easy to make various types of improvements or deformations, and is not limited to The methods described in the above specific embodiments of the present invention, therefore, the above-described methods are only preferred and not limiting.

Claims (6)

1. A digital input method based on gestures, characterized in that: the method constructs a virtual interface, the virtual interface is an operator's gesture operation area, and the virtual interface can change along with changes in the operator's body position or posture ; Input N numbers sequentially through a virtual interface, or generate N virtual interfaces, each virtual interface inputs a number, and these N virtual interfaces form a virtual interface group; The display screen is defined as the physical interface, and the space between the operator and the physical interface is defined as the physical space; The physical space includes a virtual interface and a non-virtual interface; the operator's gestures are valid and perceivable only in the virtual interface, and gestures in the non-virtual interface are invalid; Input numbers through the virtual interface or group of virtual interfaces; The construction of the virtual interface is implemented as follows: (1) The computer detects the behavior model of the operator: if it is detected that the palm is first pushed forward and then remains stationary, then enter step (2), if it is other behaviors, then return to step (1); (2) Calculate the position Z of the center of gravity of the gesture when the palm remains stationary; (3) Put the virtual interface template M at a position centered on the center of gravity position Z of the gesture to obtain a virtual interface V(Z, Ω _M ), where Ω _M represents the scope of the virtual interface determined by the template M; (4) Calculate the three-dimensional position information of each point of interest on V; (5) Return to the virtual interface V. 2. The gesture-based digital input method according to claim 1, characterized in that: the function distribution and size range of the virtual interface template M of each application are fixed; or interactive objects on the virtual interface V. 3. The gesture-based digital input method according to claim 2, characterized in that: said step (4) is to calculate the three-dimensional position information of each point of interest on V by using spatial depth information and gesture tracking method. 4. The gesture-based digital input method according to claim 3, characterized in that: said inputting numbers through said virtual interface group is realized in the following way: Q1, using the clock dial structure to represent the virtual interface, wherein the 12 o’clock in the clock dial structure is defined as the number 0, and then the numbers 1 to 9 are sequentially defined in the clockwise direction, and the arc between the number 1 and the number 2 is defined. The midpoint is A, the midpoint of the arc between the number 2 and the number 3 is B, the center of the circle is Z, and the rays ZA and ZB enclose a fan-shaped area Θ; Q2, for each number to be entered, the specific steps include: (A1) generate the i-th virtual interface Vi; (A2) The operator moves gestures on Vi; (A3) If the gesture is in a static state, further detect whether it is changed from a bag gesture with five fingers stretched to a fist gesture with five fingers contracted, if yes, then enter (A4), if not, return to step (A3); (A4) Calculate the position Pg of the center of gravity of the gesture; (A5) If Go to step (A4); (A6) Calculate f: $\underset{\mathrm{<span\; class="notranslate">\; f</span>}}{\mathrm{<span\; class="notranslate">\; m</span>}\mathrm{<span\; class="notranslate">\; i</span>}\mathrm{<span\; class="notranslate">\; no</span>}}<span\; class="notranslate">\; \{</span>\mathrm{<span\; class="notranslate">\; d</span>}\mathrm{<span\; class="notranslate">\; i</span>}\mathrm{<span\; class="notranslate">\; the\; s</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; g</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; \&Theta;</span>}}_{\mathrm{<span\; class="notranslate">\; f</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \}</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>$ Among them, Θ _f represents the point set of the space fan-shaped area where the number f is located on the virtual interface V _i . 5. The gesture-based digital input method according to claim 4, characterized in that: said Q1 further comprises: Divide the 9 fan-shaped areas of numbers 0 to 9 on the virtual interface into 3 quadrants, where quadrant I is the fan-shaped area where 0-3 is located; quadrant II is the fan-shaped area where 3-6 is located; quadrant III is where 6-9 is located sector area; The operator's gesture starts from the center point of the virtual interface to select a quadrant, and then judges the quadrant that the operator wants to select according to the direction range of the gesture trajectory; The ray determined by the center point Z and the number 0 is the starting vector L, and the angle gradually increases in the clockwise direction. The direction of gesture movement is T, and the angle between T and L is θ=<T,L>, where T is The quadrants are: When making digital selection, the operator’s palm moves along the T direction, and then pushes the gesture vertically forward, then the quadrant determined by T is selected, and then the corresponding virtual interface template is called, and the quadrant is set as a new virtual interface for operation or further operation; In said new virtual interface, there are only 4 numbers in that quadrant. 6. The gesture-based digital input method according to any one of claims 1 to 5, characterized in that the method uses a particle filter algorithm to obtain gesture motion trajectories.