TWI775128B - Gesture control device and control method thereof - Google Patents

Abstract

A gesture control device and a control method thereof, comprising an image processing module, the image processing module is electrically connected to a photographing unit, and the image processing module is used for receiving a two-dimensional image captured by the photographing unit The image is subjected to subsequent processing, including capturing the two-dimensional image of the hand, performing grayscale processing on the two-dimensional image of the hand, and calculating the feature points of the two-dimensional image. The image processing module is also electrically connected to an image analysis module. , the image analysis module is used to receive the calculated two-dimensional image information for subsequent analysis, including marking the calculated two-dimensional image, drawing nodes for the marked feature positions or calculating other nodes, and finally The gesture judgment unit performs gesture interpretation, and finally outputs the result of the gesture judgment.

Description

Gesture control device and control method thereof

The present invention relates to a control device, in particular to a gesture control device with human-computer interaction function and a control method thereof.

The traditional human-computer interaction interface is mouse, keyboard and joystick. With the rapid development of technology, in order to make human-computer interaction more convenient, somatosensory control provides a new input method. The most common somatosensory control is through Gesture recognition for a new human-computer interaction.

Since gestures are a common way of communication between people in daily life, it is a very intuitive and convenient method. Therefore, under the guidance of technology, gesture recognition is gradually applied in one of many fields, including human-machine interface design, medical rehabilitation Health, virtual reality and game design.

There are two main ways to identify gesture information, one is dynamic gestures and the other is static gestures; dynamic gesture information includes hand movement trajectory, position information and timing relationship, while static gesture information is mainly hand shape changes. Analyze gesture information and achieve the function of human-computer interaction according to different gestures

At present, the common gesture recognition technology is to use a depth camera to obtain a 3D image. Each 3D image must be pre-processed, such as image binarization, clearing the image background, eliminating noise, etc., and then from a series of images. Capture and analyze the user's hand position and gestures and other related information. Then use the image coordinate value of the hand position to control the movement of the cursor on the display; then, through the 3D image, it takes a long time in the pre-processing, which makes the speed and accuracy of moving the cursor difficult to compare with the mouse, and the depth The cost of such devices as cameras is too high, making it difficult to reduce the cost threshold of current gesture recognition devices and technologies.

In view of the above deficiencies, the main purpose of the present invention is to provide a gesture control device and a control method thereof, through the combination of a camera for capturing a two-dimensional image and an algorithm, to determine the change of gestures and achieve the purpose of human-computer interaction, And thereby reduce the cost of the gesture control device.

In order to achieve the above object, the present invention mainly provides a gesture control device and a control method thereof, comprising an image processing module, the image processing module is electrically connected to a photographing unit, and the image processing module is used for receiving The two-dimensional image captured by the photographing unit is subjected to subsequent processing. The image processing module further includes an image processing unit and a feature calculation unit, wherein the image processing unit receives the captured two-dimensional image. After that, post-processing is performed, including capturing the 2D image of the hand and performing grayscale processing on the 2D image of the hand. The 2D image to be processed is input into the feature calculation unit, and the calculation of the 2D image is started. feature points of the image; the image processing module is also electrically connected to an image analysis module, and the image analysis module is used for receiving the two-dimensional image information completed by the calculation for subsequent analysis, wherein the image analysis module further includes a An image analysis unit and a gesture determination unit, wherein the image analysis unit is used for receiving two-dimensional image information, marking the calculated two-dimensional image, and performing node drawing for the marked feature position or calculating other nodes, Finally, the gesture judgment unit performs gesture interpretation, and finally outputs the result of the gesture judgment.

In order to make the above-mentioned and other objects, features and advantages of the present invention more clearly understood, preferred embodiments are hereinafter described in detail in conjunction with the accompanying drawings.

First, the “feature points” mentioned in this embodiment refer to several points obtained through image detection and analysis, which may be joint positions, skin lines at the joints, fingernails, palm lines, or around the wrist. appearance, etc. The obtained feature points are compared with the joint points set in the previous training, and then the image key points at the joints are marked. Using the pattern formed by the connection between the key points and comparing the shape of the gestures preset before, it can be used to activate the corresponding function.

Please refer to the first figure, which is a system block diagram of the present invention. The gesture control device of the present invention mainly includes an image processing module 1 , the image processing module 1 is electrically connected to a photographing unit 2 , and the image processing module 1 is used for receiving images captured from the photographing unit 2 . The two-dimensional image of the two-dimensional image is processed later, and the photographing unit 2 is a camera with the function of capturing two-dimensional images; the image processing module 1 further includes an image processing unit 11 and a feature calculation unit 12, the image The processing unit 11 is electrically connected with the feature calculation unit 12 , wherein the image processing unit 11 performs post-processing after receiving the captured 2D image, including capturing the 2D image of the hand and processing the 2D image of the hand. The image is subjected to gray-scale processing, and the 2D image to be processed is input into the feature calculation unit 12, which starts to calculate the scale-invariant feature transform descriptor of the 2D image, and the calculation method of the feature point is For each keypoint (keypoint), the pixel size of 16×16 is captured, and then averagely cut into 4×4 grid size (cells), and the gradient and angle values of each grid cell are counted as 8 boxes (bins) The histogram (Histogram), a total of 16 grids will get 16 histograms (8 bins), which can be combined into 16×8=128 dimension data, and finally normalize these data (L2-Normalizing), that is A feature representing the key point can be obtained. ; That is, the position of the feature point can be a two-dimensional coordinate (X, Y), and after the key point is calculated and transformed, its position coordinate is a three-dimensional coordinate (X, Y, Z). The calculation and conversion method may be by comparing reference points in the image (for example, a face or an object behind), and then calculating the Z value of each key point according to the size ratio of the reference point and each key point. When obtaining the reference point, the Z value of each key point can also be obtained or corrected through hand movements.

Continue to refer to the first figure. The image processing module 1 is further electrically connected to an image analysis module 3, and the image analysis module 3 is used for receiving the two-dimensional image information completed by the calculation for subsequent analysis, wherein the image analysis module 3 further includes an image The analysis unit 31 and a gesture determination unit 32, the image analysis unit 31 is electrically connected to the gesture determination unit 32 by information, wherein the image analysis unit 31 is used for receiving the two-dimensional image information, and the calculated two-dimensional image is processed. mark, and perform node drawing for the marked feature position or calculate other nodes, and finally the gesture judgment unit 32 performs gesture interpretation, and finally outputs the result of the gesture judgment.

Continue to refer to the first figure. In addition, the image analysis module 3 is electrically connected to a gesture training module 4, and the gesture training module 4 further includes a gesture recording unit 41 and a gesture training unit 42, and the gesture recording unit 41 is electrically connected to the gesture A training unit 42, wherein the gesture recording unit 41 is used for recording gesture images, and the gesture images are loaded into the gesture training unit 42 for gesture training and a gesture training file is generated, and the files are then sent to the gesture determination unit 32 , as the basis for gesture judgment.

Please refer to the second figure, which is a flowchart of the gesture control method of the present invention. The steps of the control method include capturing a two-dimensional hand image ( S1 ), the two-dimensional hand image is operated by a camera with a function of capturing a two-dimensional image; and then calculating the characteristics of the two-dimensional hand image point (S2), including performing gray-scale processing and feature point calculation on the two-dimensional image of the hand, wherein the calculation step further includes extracting a pixel size of 16×16 for each keypoint (S21) , as shown in the feature calculation flow chart in Figure 3, and then cut each pixel into 4 × 4 cells on average (S22), and each cell takes the gradient and angle values and counts them into 8 boxes. The histogram of (bins) (S23), a total of 16 grids will get 16 histograms (8 bins), and finally merge into 16×8=128 dimension data (S24), and finally normalize the dimension data L2-Normalizing (S25), the feature representing the key point can be obtained.

Continue to refer to the third figure. The step of the control method further includes marking the feature points of the two-dimensional hand image (S3), which is used to mark the calculated two-dimensional image in this step, and perform node drawing for the marked feature position or calculate other node; finally, judge the gesture of the two-dimensional hand image based on the marked images (S4), and interpret the gesture file stored in the system to output the final interpretation result, as shown in the fourth to seventh figures. As shown in the embodiment diagram, according to different gestures, when the system interprets, each key point will generate corresponding movement or create corresponding gestures.

However, the above-mentioned embodiments are preferred implementation examples, which should not limit the scope of the present invention. Any equivalent changes or modifications made according to the scope of the patent application of the present invention and the contents of the description shall belong to the following aspects of the present invention. the scope of patent coverage.

1: Image processing module

11: Image processing unit

12: Feature calculation unit

2: Photography unit

3: Image analysis module

31: Image Analysis Unit

32: Gesture judgment unit

4: Gesture training module

41: Gesture Recording Unit

42: Gesture training unit

The first figure is a system block diagram of the present invention.

The second figure is a flow chart of the present invention.

The third figure is a flow chart of the characteristic calculation of the present invention.

The fourth figure is a figure (1) of the embodiment of the present invention.

The fifth figure is the figure (2) of the embodiment of the present invention.

The sixth figure is the figure (3) of the embodiment of the present invention.

The seventh figure is the figure (4) of the embodiment of the present invention.

1: Image processing module

2: Photography unit

3: Image analysis module

4: Gesture training module

11: Image processing unit

12: Feature calculation unit

31: Image Analysis Unit

32: Gesture judgment unit

41: Gesture Recording Unit

42: Gesture training unit

Claims (6)

A gesture control device includes: an image processing module, the image processing module is used for receiving the captured image for subsequent processing, and the image processing module further includes: an image processing unit, which is located in the After receiving the captured 2D image, post-processing is performed, including only capturing the 2D image of the hand and performing grayscale processing on the 2D image of the hand; a feature calculation unit, the feature calculation unit and the image are The processing unit is electrically connected, the two-dimensional image to be processed is input into the feature calculation unit, and the feature points of the two-dimensional image are calculated; a photographing unit is electrically connected to the image processing module, and the photographing unit is It has the function of capturing two-dimensional images; an image analysis module is electrically connected to the image processing module for receiving the two-dimensional image information completed by the calculation for subsequent analysis, and the image analysis module further includes: an image analysis The unit is used to receive the two-dimensional image information, mark the calculated two-dimensional image, and perform node drawing for the marked feature position or calculate other nodes; a gesture judgment unit is electrically connected to the image analysis unit. The gesture judgment unit is used for interpreting the gesture image, and then outputting the result of the gesture judgment. The gesture control device according to claim 1, wherein the device further comprises a gesture training module which is electrically connected to the image analysis module, and the gesture training module further comprises a gesture recording unit and a gesture training module unit, the gesture recording unit is electrically connected to the gesture training unit, the gesture recording unit is used for recording gesture images, and the gesture images are loaded into the gesture training unit for execution The gesture training generates gesture training files, and the files are then sent to the gesture judgment unit to be used as the basis for gesture judgment. A gesture control method for a gesture control device as claimed in claim 1, the steps of which include: a. the image processing unit only captures a two-dimensional hand image; b. the feature calculation unit calculates feature points of the two-dimensional hand image ; c. The image analysis unit marks the feature points of the two-dimensional hand image; d. The gesture judgment unit judges the gesture of the two-dimensional hand image. The gesture control method as described in claim 3, wherein step b further includes performing grayscale processing and feature point calculation on the two-dimensional image of the hand. The gesture control method according to claim 3, wherein the step of calculating the feature points further comprises: b1: extracting a pixel size of 16×16 for each key point; b2: cutting each pixel into 4 on average ×4 grid size; b3: take the gradient and angle values of each grid as a histogram of 8 boxes; b4: merge into 16×8=128 dimensional data (S24); b5: normalize the dimensional data Unified processing. The gesture control method described in claim 3, wherein the step c is used to mark the calculated two-dimensional image, and to draw nodes or calculate other nodes for the marked feature positions.

Images