CN105160323B - A kind of gesture identification method - Google Patents
A kind of gesture identification method Download PDFInfo
- Publication number
- CN105160323B CN105160323B CN201510563293.9A CN201510563293A CN105160323B CN 105160323 B CN105160323 B CN 105160323B CN 201510563293 A CN201510563293 A CN 201510563293A CN 105160323 B CN105160323 B CN 105160323B
- Authority
- CN
- China
- Prior art keywords
- hand
- sequence
- image
- characteristic point
- user
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/107—Static hand or arm
- G06V40/113—Recognition of static hand signs
Landscapes
- Engineering & Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
Abstract
The present invention provides a kind of gesture identification method, which includes：Obtain the testing image sequence of user's hand containing depth information；According to image depth information and image color information, the hand profile of user is detected in every frame image of testing image sequence；For every hand of user, using preset hand structure template, the characteristic point sequence to be measured of this hand is determined in every frame image of testing image sequence；For every hand of user, the matching sequence of the characteristic point sequence to be measured of this hand is determined in multiple default characteristic point sequences, to determine denomination of dive and the position of this hand according to matching sequence；The gesture that selection matches with the denomination of dive of user's both hands and position in default gesture table, the gesture identification result as testing image sequence.Above-mentioned technology of the invention can accurately identify the gesture of user, and accuracy of identification is higher, and recognition speed is very fast.
Description
Technical field
The present invention relates to signal processing technology more particularly to a kind of gesture identification methods.
Background technique
With the development of science and technology, the electronic equipment of laptop, mobile phone, tablet computer etc. gradually has more
Carry out more functions, and gesture identification is exactly one of this various functions.
Currently, the algorithm that the identification processing procedure of existing Gesture Recognition utilizes is complex, the time is not only expended,
And thus identify that the precision of gesture is lower, accuracy is poor.In addition, the reality of the existing Gesture Recognition based on pattern-recognition
When property is poor, is not suitable for the occasion of real-time interaction demand.
Summary of the invention
It has been given below about brief overview of the invention, in order to provide about the basic of certain aspects of the invention
Understand.It should be appreciated that this summary is not an exhaustive overview of the invention.It is not intended to determine pass of the invention
Key or pith, nor is it intended to limit the scope of the present invention.Its purpose only provides certain concepts in simplified form,
Taking this as a prelude to a more detailed description discussed later.
In consideration of it, being deposited the present invention provides a kind of gesture identification method at least solving existing Gesture Recognition
Gesture identification precision, poor accuracy the problem of.
According to an aspect of the invention, there is provided a kind of gesture identification method, the gesture identification method include：Step
One, the testing image sequence of user's hand containing depth information is obtained；Step 2: according to image depth information and image face
Color information detects the hand profile of the user in every frame image of the testing image sequence；Step 3: for institute
Every hand for stating user, using preset hand structure template, determining in every frame image of the testing image sequence should
The characteristic point sequence to be measured of hand；Step 4: being directed to every hand of the user, determined in multiple default characteristic point sequences
The matching sequence of the characteristic point sequence to be measured of this hand, to determine denomination of dive and the position of this hand according to the matching sequence
It sets；Step 5: the gesture that selection matches with the denomination of dive of user's both hands and position in default gesture table, as
The gesture identification result of the testing image sequence.
Further, step 3 may include：Step 3 one, every hand for the user, utilize preset hand
Portion's stay in place form determines the predetermined number feature of this hand in the hand profile of every frame image of the testing image sequence
Point；Step 3 two, every hand for the user are right in each frame image of the testing image sequence using this hand
The predetermined number characteristic point answered obtains the characteristic point sequence to be measured of this hand.
Further, step 1 may include：By capturing the image of user's hand in predetermined imaging region,
Obtain Detection Method in Optical Image SequencesAnd infrared image sequenceFor the Detection Method in Optical Image Sequences
Pixel value at i-th frame image coordinate (x, y), andAt infrared image sequence the i-th frame image coordinate (x, y)
Pixel value, according to the following formula obtain extract user's both hands information image sequence：
Wherein, α, β, λ are parameter preset threshold value,For the figure of user's both hands containing depth information of acquisition
As sequence, as the testing image sequence, i=1,2 ..., M, the number of image frames that M includes by the testing image sequence.
Further, step 2 may include：For the testing image sequenceIn every frame imageThe color combining information deletion frame imageIn noise spot and non-area of skin color, calculated using edge detection
Sub- E () is to obtained image after the deletion noise spot and the non-area of skin colorEdge detection is carried out, is obtained
Edge imageThe edge imageIt as only include user's hand
The image of contouring.
Further, step 3 one may include：Following processing is executed for every frame image of the testing image sequence：
The finger tip point in the contour line is found according to the curvature in the contour line of the image and refers to root artis；Using the finger tip point as
Setting base matches each finger root artis singly referred to, obtains benchmark of each length singly referred to as scaling；Based on described
Finger tip point and the position for referring to root artis and each length singly referred to zoom in and out the corresponding hand structure template
And deformation, each articulations digitorum manus characteristic point and wrist midpoint characteristic point of every hand are obtained by matching；Wherein, the hand structure mould
Plate includes left-handed configuration template and right hand configurations template, and the left-handed configuration template and right hand configurations template respectively include：Each hand
The fingertip characteristic point of finger, respectively refers between root joint characteristic point, wrist midpoint characteristic point and each characteristic point each articulations digitorum manus characteristic point
Topological relation.
Further, step 4 may include：Step 4 one, for the characteristic point sequence to be measured of every hand, according to predetermined
The characteristic point sequence to be measured is divided into multiple subsequences by time window, and obtains the corresponding mean place of each subsequence；Step
Four or two, it is directed to the corresponding each subsequence of every hand, by each of the subsequence and the multiple default characteristic point sequence
It is matched respectively, selects to be higher than preset matching with the matching degree of the subsequence in the multiple default characteristic point sequence
Threshold value and maximum default characteristic point sequence, the matching sequence as the subsequence；Step 4 three, by each subsequence pair
The mean place answered denomination of dive corresponding with the matching sequence of the subsequence is associated；Step 4 four is directed to every hand, by this
The matching sequence of the corresponding each subsequence of hand is and each by multiple matching sequence as the corresponding multiple matching sequences of this hand
Multiple denomination of dive of the self-corresponding denomination of dive as this hand.
Further, step 5 may include：Step 5 one presets following map listing as the default hand
Gesture table：The left end of each mapping in the map listing is the position of set title pair and each denomination of dive pair；This is reflected
The right end for each mapping penetrated in list is a gesture HandSignal；Step 5 two, will be every in the default gesture table
The left end of a mapping is matched with the denomination of dive of user's both hands and position, wherein the matching of denomination of dive executes
Stringent matching, and position is then relative position information to be calculated by the respective mean place of user's both hands, and then calculate
Similarity between the relative position information and the position for mapping left end is realized.
Above-mentioned gesture identification method according to an embodiment of the present invention using first identification single-handed exercise and then passes through both hands
The mode of action recognition gesture realizes that can accurately identify the gesture of user, accuracy of identification is higher, and recognition speed compared with
Fastly.
Above-mentioned gesture identification method of the invention uses Hierarchical Design algorithm, and algorithm complexity is low, is easy to implement.
It is right when needing to change (such as modify, increase or decrease) in addition, using above-mentioned gesture identification method of the invention
It, can be only by adjusting template (that is, by modifying the default corresponding movement of characteristic point sequence when movement and/or the definition of gesture
Title changes the definition of movement, increases, subtracts movement by increasing or decreasing default characteristic point sequence and its respective action title)
And default gesture table (that is, changing the definition of gesture by modifying the corresponding multiple movements of gesture in default gesture table, passes through
Gesture and its respective action in default gesture table are increased or decreased to increase, subtract gesture), without changing algorithm or instructing again
Practice classifier, substantially increases the adaptability of algorithm.
In addition, the strong real-time of above-mentioned gesture identification method of the invention, can be suitble to the occasion of real-time interaction demand.
By the detailed description below in conjunction with attached drawing to highly preferred embodiment of the present invention, these and other of the invention is excellent
Point will be apparent from.
Detailed description of the invention
The present invention can be by reference to being better understood, wherein in institute below in association with description given by attached drawing
Have and has used the same or similar appended drawing reference in attached drawing to indicate same or similar component.The attached drawing is together with following
It is described in detail together comprising in the present specification and forming a part of this specification, and is used to that this is further illustrated
The preferred embodiment and explanation the principle of the present invention and advantage of invention.In the accompanying drawings：
Fig. 1 is the flow chart for showing an example process of gesture identification method of the invention；
Fig. 2 is the flow chart for showing the example process of step 3 shown in FIG. 1；
Fig. 3 is the flow chart for showing the example process of step 4 shown in FIG. 1；
Fig. 4 is the flow chart for showing the example process of step 5 shown in FIG. 1.
It will be appreciated by those skilled in the art that element in attached drawing is just for the sake of showing for the sake of simple and clear,
And be not necessarily drawn to scale.For example, the size of certain elements may be exaggerated relative to other elements in attached drawing, with
Just the understanding to the embodiment of the present invention is helped to improve.
Specific embodiment
Exemplary embodiment of the invention is described hereinafter in connection with attached drawing.For clarity and conciseness,
All features of actual implementation mode are not described in the description.It should be understood, however, that developing any this actual implementation
Much decisions specific to embodiment must be made during example, to realize the objectives of developer, for example, symbol
Restrictive condition those of related to system and business is closed, and these restrictive conditions may have with the difference of embodiment
Changed.In addition, it will also be appreciated that although development is likely to be extremely complex and time-consuming, to having benefited from the disclosure
For those skilled in the art of content, this development is only routine task.
Here, and also it should be noted is that, in order to avoid having obscured the present invention because of unnecessary details, in the accompanying drawings
Illustrate only with closely related apparatus structure and/or processing step according to the solution of the present invention, and be omitted and the present invention
The little other details of relationship.
The embodiment provides a kind of gesture identification method, which includes：Contain Step 1: obtaining
There is the testing image sequence of user's hand of depth information；Step 2: according to image depth information and image color information,
The hand profile of the user is detected in every frame image of the testing image sequence；Step 3: being directed to the user
Every hand, using preset hand structure template, determined in every frame image of the testing image sequence this hand to
Survey characteristic point sequence；Step 4: being directed to every hand of the user, this hand is determined in multiple default characteristic point sequences
The matching sequence of characteristic point sequence to be measured, to determine denomination of dive and the position of this hand according to the matching sequence；Step 5:
The gesture that selection matches with the denomination of dive of user's both hands and position in default gesture table, as described to mapping
As the gesture identification result of sequence.
Fig. 1 shows the flow chart of an example process of gesture identification method of the invention.Above-mentioned gesture identification method
Exemplary process start from step 1.
As shown in Figure 1, obtaining the testing image sequence of user's hand containing depth information in step 1.
According to a kind of implementation, the processing of step 1 can be implemented by the following steps：By capturing predetermined imaging
The image of user's hand in region, (such as can use visible light image sensor and infrared figure in depth camera
As sensor) obtain Detection Method in Optical Image SequencesAnd infrared image sequenceFor visible light figure
As the pixel value at sequence the i-th frame image coordinate (x, y), andAt infrared image sequence the i-th frame image coordinate (x, y)
Pixel value, according to the following formula it is available extract user's both hands information image sequence：
Wherein, α, β, λ are parameter preset threshold value, these parameter preset threshold values can set based on experience value, can also be with
Determined by the method for test (such as the collected sample image training of the depth camera by actually using specific model
Obtain), which is not described herein again.For the image sequence of user's both hands containing depth information of acquisition, as upper
State testing image sequence.In addition, i=1,2 ..., M, M is number of image frames included in testing image sequence.
It should be noted that the difference (single or double) of the hand quantity according to used in user's gesture, is making a reservation for into
As institute's captured image may be the image comprising user's both hands in region, it is also possible to only include the figure of single hand of user
Picture.In addition, the testing image sequence obtained can be and obtain in a period of time, which can be previously according to experience
Value setting, may be, for example, 10 seconds.
Then, in step 2, according to image depth information and image color information, in every frame figure of testing image sequence
The hand profile of user is detected as in.Wherein, the hand profile detected may be both hands profile, it is also possible to single hand wheel
It is wide.
According to a kind of implementation, the processing of step 2 can be implemented by the following steps：For testing image sequenceIn every frame imageThe color combining information deletion frame imageIn noise spot and the non-colour of skin
Region, using edge detection operator E () to obtained image after erased noise point and non-area of skin colorCarry out side
Edge detection, to obtain edge image
Edge imageIt as only include the image of user's hand profile.
Wherein, in the treatment process of " noise spot and non-area of skin color in the color combining information deletion frame image ",
Existing denoising method be can use to delete the noise spot in image, and can be by calculating imageMean value obtain skin
Color region, then the region except area of skin color is non-area of skin color, and the deletion to non-area of skin color can be realized.For example,
To imageMean value after, float up and down a range in the mean value, obtain a color gamut comprising the mean value, work as figure
The color value of certain point is fallen within this color gamut as in, then is colour of skin point by the point determination, otherwise it is assumed that not being colour of skin point；
All colour of skin points constitute area of skin color, remaining is non-area of skin color.
As a result, by the processing of step 2, the hand profile of user can be quickly detected, improve and entirely handle
Speed and efficiency.
Then, in step 3, for every hand of user, using preset hand structure template, in testing image
The characteristic point sequence to be measured of this hand is determined in every frame image of sequence.
Wherein, hand structure template includes left-handed configuration template and right hand configurations template, left-handed configuration template and right hand knot
Structure template respectively includes the topological relation between predetermined number characteristic point and each characteristic point.
In one example, left-handed configuration template and right hand configurations template can respectively include following 20 (as predetermined number
Purpose example, but predetermined number is not limited to 20, can also be the numerical value such as 19,21) a characteristic point：The fingertip characteristic point (5 of each finger
It is a), each articulations digitorum manus characteristic point (9), respectively refer to root joint characteristic point (5), wrist midpoint characteristic point (1).
According to a kind of implementation, the processing in step 3 can pass through step three as shown in Figure 2 one and step 3 two
To realize.
As shown in Fig. 2, for every hand of user, can use above-mentioned preset hand structure in step 3 one
Template, respectively by every frame image of testing image sequence hand profile and hand structure template (tiled configuration template and the right side
Hand stay in place form) it matched, be aligned, obtain predetermined number (such as 20) characteristic point in the frame image hand profile.
Then, in step 3 two, for every hand of user, it can use this hand in each of testing image sequence
Corresponding predetermined number characteristic point (i.e. feature point set), obtains the characteristic point sequence to be measured of this hand in frame image.
In this way, passing through hand structure template and each hand profile (the i.e. every frame figure of testing image sequence obtained before
Hand profile as in) it carries out the processing such as matching, the predetermined number that can quickly and accurately obtain in each hand profile is special
Sign point.Thereby, it is possible to subsequent processing using the predetermined number characteristic point in these profiles to further realize hand
Gesture identification, compared with the prior art, improves the speed and accuracy of entire gesture recognition process.
In the prior art, when needing to change (such as modify, increase or decrease) to movement according to different application scene
Definition when, need to modify algorithm and re -training classifier；In the present invention, template only can be acted (i.e.
Default characteristic point sequence) realize the change to action definition, substantially increase the adaptability of algorithm.
In one example, the processing of step 3 one can be realized in the following way.
According to the physiological structure feature of mankind's both hands, portable 20 (example as predetermined number) a spies can be taken to every
Sign point.For every frame image of testing image sequenceExecute following processing：Firstly, according to the imageIn
Profile curvature of a curve finds the finger tip point in the contour line and refers to root artis；Then using finger tip point as setting base, matching
Each finger root artis singly referred to, obtains benchmark of each length singly referred to as scaling；Finally based on the finger tip found
Point and refer to the position of root artis and obtained each length for singly referring to of both parameter to corresponding hand structure template into
Row scaling and deformation obtain remaining 10 characteristic point of every hand, i.e., each articulations digitorum manus characteristic point and hand of every hand by matching
Wrist midpoint characteristic point.
For example, looking for contour lineIn finger tip point and refer to root artis during, can be by its mean curvature most
Big salient point is as finger tip point, using the concave point of maximum curvature as webs minimum point, and by each finger tip point to the finger tip point phase
Adjacent the distance between webs minimum point is defined as the corresponding unit length of the finger tip point.It is minimum to every two adjacent webs
Point, by this 2 points midpoint, toward volar direction extension one third unit length, (unit length at this time is thus between 2 points again
The corresponding unit length of finger tip point) point, the corresponding finger root artis of the finger tip point is defined as, it is hereby achieved that every hand
The finger root artis of centre 3.It in addition to this, can be by during subsequent scaling and deformation for every hand
Obtain two finger root artis of head and the tail of this hand；Alternatively, can also be adjacent by two (such as arbitrarily the selecting two) of this hand
The distance between webs minimum point be used as finger reference width, then by two webs minimum points of the head and the tail of this hand respectively along cutting
Line direction, extend outwardly half of finger reference width, the two finger root artis of head and the tail of obtained point respectively as this hand.
It should be noted that if the salient point found for single hand more than 5, can be by itself and hand structure mould
Plate remove extra salient point during matching alignment.
It is matched to obtain 20 characteristic point Pl={ pl of the corresponding left hand of each frame image with such method as a result,1,
pl2..., pl20And the right hand 20 characteristic point Pr={ pr1, pr2..., pr20}.It should be noted that if user's gesture is only
It is then 20 characteristic point (referred to as features of the single hand in every frame image by matching above obtained comprising single hand
Point set), i.e. Pl={ pl1, pl2..., pl20Or Pr={ pr1, pr2..., pr20}.Wherein, pl1,pl2,…,pl20Respectively
The position of 20 characteristic points of left hand, and pr1,pr2,…,pr20The respectively position of 20 characteristic points of the right hand.
If user's gesture includes both hands, pass through the available left hand of the processing of above step 31 and step 3 two
Characteristic point sequence { Pl to be measuredi, i=1,2 ..., M and the right hand characteristic point sequence { Pr to be measuredi, i=1,2 ..., M }.Its
In, PliFor user's left hand, corresponding 20 (examples as predetermined number) are a in the i-th frame image of testing image sequence
Characteristic point, and PriFor user's right hand corresponding 20 (as showing for predetermined number in the i-th frame image of testing image sequence
Example) a characteristic point.
If user's gesture only includes single hand, every frame image in the testing image sequence captured is only comprising being somebody's turn to do
The image of single hand, so that the characteristic point to be measured of the single hand can be obtained later by the processing of step 3 one and step 3 two
Sequence, i.e. { Pli, i=1,2 ..., M } or { Pri, i=1,2 ..., M }.
In this way, executing step four as shown in Figure 1 after executing the step three.
In step 4, for every hand of user, the to be measured of this hand is determined in multiple default characteristic point sequences
The matching sequence of characteristic point sequence, to determine denomination of dive and the position of this hand according to the matching sequence.
As a result, by step 4, the characteristic point sequence to be measured and each default characteristic point sequence progress of every hand are utilized
Match, obtain matching sequence, and then quickly determines the movement of this hand according to the corresponding denomination of dive of matching sequence.
According to a kind of implementation, step 4 can be realized to step 4 four by step four as shown in Figure 3 one
Processing.
Firstly, a hand motion list of file names is preset, including basic hand motion, such as：It waves, push away, drawing, opening and closing, turning
There is unique name identification Deng, each movement and with normalized hand-characteristic point sequence (preset characteristic point sequence)
The template of expression.It should be noted that every hand all has an above-mentioned hand motion name for the both hands of user
List.That is, include in the hand motion list of file names (abbreviation left hand acts list of file names) of left hand is every for left hand
A movement also has a left hand template (i.e. default characteristic point sequence of left hand other than being respectively provided with respective title
Column)；For the right hand, each movement for including in the hand motion list of file names of the right hand (the abbreviation right hand acts list of file names) in addition to
It is respectively provided with except respective title, also there is a right hand template (i.e. a default characteristic point sequence of the right hand).
For example, multiple default characteristic point sequences of single hand are denoted as sequence A respectively1, sequence A2..., sequence AH, wherein H
The sequence number that above-mentioned multiple default characteristic point sequences for the single hand are included, then in the hand motion list of file names of the single hand
In：The name identification of movement 1 is " waving " and corresponding template (i.e. default characteristic point sequence) is sequence A1；The title mark of movement 2
Knowing is " pushing away " and corresponding template is sequence A2 ；…；The name identification for acting H is " turning " and corresponding template is sequence AH 。
In step 4 one, for the characteristic point sequence to be measured of every hand, according to predetermined time window by the characteristic point to be measured
Sequences segmentation is multiple subsequences, and obtains the corresponding mean place of each subsequence.Wherein, each subsequence is corresponding average
Position can choose average bit of the specific characteristic point (such as wrist midpoint, or can also be other characteristic points) in the subsequence
It sets.Wherein, predetermined time window is about the time of a singlehanded elemental motion (i.e. singlehanded holds, grabs) from start to end, can
It sets, or can be determined by the method for test based on experience value, such as can be 2.5 seconds.
In one example, it is assumed that characteristic point sequence to be measured was acquired in 10 seconds, can be with using 2.5 seconds time windows
The characteristic point sequence to be measured of the characteristic point sequence to be measured of left hand and the right hand is divided into 4 subsequences respectively.With left hand to
Survey characteristic point sequence { Pli, i=1,2 ..., M for the (characteristic point sequence { Pr to be measured of the right handi, i=1,2 ..., M } and its
Similar, I will not elaborate), it is assumed that 10 frame images of acquisition per second, then it is 100 frame images, i.e. M that characteristic point sequence to be measured is corresponding
=100, that is to say, that { Pli, i=1,2 ..., M } it include 100 groups of feature point set Pl1、Pl2、…、Pl100.In this way, by upper
2.5 seconds time windows are stated, it can be by { Pli, i=1,2 ..., M } it is divided into { Pli, i=1,2 ..., 25, { Pli, i=25,
26,...,50}、{Pli, i=51,52 ..., 75 } and { Pli, i=76,77 ..., 100 } and 4 subsequences, and every sub- sequence
Each 25 frame image of correspondence is arranged, that is, each subsequence respectively includes 25 groups of feature point sets.Specific characteristic point chooses wrist midpoint, with
Subsequence { Pli, i=1,2 ..., 25 } for (its excess-three sub- sequence handled to it similar, I will not elaborate), in wrist
Point is in { Pli, i=1,2 ..., 25 } position in corresponding 25 groups of feature point sets is respectively position p1、p2、…、p25, then hand
Wrist midpoint is in subsequence { Pli, i=1,2 ..., 25 in mean place be (p1+p2+…+p25)/25, as subsequence
{Pli, i=1,2 ..., 25 } corresponding mean place.
Then, in step 4 two, for the corresponding each subsequence of every hand, by the subsequence and multiple default features
Each of point sequence is matched respectively, is selected in multiple default characteristic point sequences high with the matching degree of the subsequence
In preset matching threshold, (matching threshold can be set based on experience value, or can also be by the method for test come really
It is fixed) and that maximum default characteristic point sequence, the matching sequence as the subsequence.Wherein it is possible to calculate subsequence with
Similarity between default characteristic point sequence, as matching degree between the two.
It should be noted that for each subsequence not necessarily this can be found in multiple default characteristic point sequences
The corresponding matching sequence of subsequence.When some subsequence for single hand, which does not find it, matches sequence, then by the sub- sequence
The matching sequence of column is denoted as " sky ", but the mean place of the subsequence can not be " sky ".According to a kind of implementation, if sub- sequence
The matching sequence of column is " sky ", then the mean place of the subsequence is set as " sky "；According to another implementation, if subsequence
Matching sequence be " sky ", the mean place of the subsequence is the actual average position of specified characteristic point in the subsequence；According to
The mean place of the subsequence is set as "+∞ " if the matching sequence of subsequence is " sky " by a kind of other implementations.
In addition, according to a kind of implementation, if there is no specific characteristic points (namely the specific characteristic is not present in subsequence
The actual average position of point), the mean place of the subsequence can be set as "+∞ ".
Then, in step 4 three, the corresponding mean place of each subsequence is corresponding with the matching sequence of the subsequence
Denomination of dive it is associated.
It, can be using the matching sequence of the corresponding each subsequence of this hand as this for every hand in step 4 four
The corresponding multiple matching sequences of hand, and the corresponding denomination of dive of multiple matching sequence (after sorting in chronological order) is made
For multiple denomination of dive of this hand.
For example, it is assumed that being { Pl for multiple subsequences of the characteristic point sequence to be measured of left handi, i=1,2 ..., 25 },
{Pli, i=25,26 ..., 50, { Pli, i=51,52 ..., 75 } and { Pli, i=76,77 ..., 100 }, respectively in left hand
Multiple default characteristic point sequences in find { Pli, i=1,2 ..., 25, { Pli, i=25,26 ..., 50, { Pli, i=
51,52 ..., 75 matching sequence be followed successively by Pl1'、Pl2'、Pl3', and { Pl is not foundi, i=76,77 ..., 100
With sequence.Assuming that Pl1'、 Pl2'、Pl3' left hand movement list of file names in corresponding denomination of dive respectively be " waving ", " pushing away ",
" drawing ", { Pli, i=1,2 ..., 25, { Pli, i=25,26 ..., 50, { Pli, i=51,52 ..., 75 } and { Pli,i
=76,77 ..., 100 } respective mean place is respectively pm1、pm2、pm3And pm4, then the movement name of thus obtained left hand
Claim and position includes：" waving " (position pm1)；" pushing away " (position pm2)；" drawing " (position pm3)；" sky " (position " pm4").It needs to infuse
It anticipates to being, in different embodiments, pm4It may be actual position value, it is also possible to " sky " or "+∞ " etc..
The corresponding multiple movements of every hand of user can be obtained to the processing of step 4 four by step 4 one as a result,
Title (denomination of dive as this hand, that is to say, that the denomination of dive of this hand), and each denomination of dive has been respectively associated
One mean place (it include one or more mean places as the position of this hand, in " position of this hand ", quantity and dynamic
The quantity for making title is identical).For only identifying individual part as the identification technology of gesture, above-mentioned processing is identified
The respective multiple movements of both hands and position, provide more flexible combination, on the one hand make the accuracy of identification of gesture more
Height, the gesture on the other hand making it possible to identify are more various, abundant.
Finally, selection matches with the denomination of dive of user's both hands and position in default gesture table in step 5
Gesture, the gesture identification result as testing image sequence.
According to a kind of implementation, step 5 can be realized by step May Day as shown in Figure 4 and step 5 two
Processing.
As shown in figure 4, step 5 one, predefined one make manually from two and the element of position two is arranged to the mapping of gesture
Table is as default gesture table：The left end of each mapping is the position of set title pair and each denomination of dive pair；Each reflect
The right end penetrated is a gesture HandSignal.Then, step 5 two is executed.
Wherein, " set title to " includes multiple denomination of dive pair, and each denomination of dive is to including that left hand acts name
Claim ActNameleftWith right hand denomination of dive ActNameright, the position of each denomination of dive pair includes the opposite position of two hands
It sets.
For example, in default gesture table, mapping one for (" drawing ", " sky "), (" drawing ", " drawing "), (" sky ", " conjunction "),
(" sky ", " sky ") } (as element one), { (x1, y1), (x2, y2), (x3, y3), (x4, y4) (relative position, as element two)
To the mapping of gesture " switch "；Mapping two for (" drawing ", " drawing "), (" opening ", " opening "), (" sky ", " sky "), (" sky ",
" sky ") }, { (x5, y5), (x6, y6), (x7, y7), (x8, y8) arrive gesture " explosion " mapping；Etc..Wherein, each movement pair
The denomination of dive on the left side is acted corresponding to left hand in (such as (" drawing ", " sky ")), and the denomination of dive on the right is acted corresponding to the right hand.
By taking mapping one as an example, (x1, y1) what is indicated is between left hand first element " drawing " and right hand first element " sky "
Relative position (act to act left hand in (" drawings ", " sky ") and act the relative positions of corresponding two hands with the right hand)；
(x2, y2) what is indicated is the relative position between second, left hand movement " drawing " and second, right hand movement " drawing "；(x3, y3) table
What is shown is the relative position between left hand third movement " sky " and right hand third movement " conjunction "；And (x4, y4) what is indicated is left
Relative position between the 4th, hand movement " sky " and the 4th, right hand movement " sky ".Other mapping in elocutionary meaning with it is such
Seemingly, it repeats no more.
In step 5 two, by the denomination of dive and position of the left end of each mapping in default gesture table and user's both hands
It sets and is matched.
Wherein, the matching of denomination of dive executes stringent matching, that is, situation of verbatim account between two denomination of dive
Lower the two denomination of dive of judgement are matched；And position is then that phase is calculated by the respective mean place of user's both hands
To location information, and then calculating the similarity between the relative position information and the position for mapping left end (such as can be with come what is realized
A similarity threshold is set, determines that position is matched when the similarity of calculating is greater than or equal to the similarity threshold).
For example, it is assumed that by step 4 obtain the respective denomination of dive of user's both hands be (" drawing ", " drawing "), (" opening ",
" opening "), (" sky ", " sky "), (" sky ", " sky "), position be { (x11, y12)、(x21, y22)、(x31, y32)、(x41, y42) (right
Answer left hand)；(x'11, y '12)、(x’21, y '22)、(x’31, y '32)、 (x’41, y '42) (the corresponding right hand).
Then, the denomination of dive of user's both hands is matched with the left end of each mapping in default gesture table.
When being matched with mapping one, it can be deduced that, the left end of the denomination of dive and mapping one of user's both hands is moved
Make title mismatch, therefore ignore mapping one, continues matching mapping two.
When being matched with mapping two, it can be deduced that, the left end of the denomination of dive and mapping two of user's both hands is moved
Make title exact matching, then again matches the position of user's both hands with the relative position of the left end of mapping two.
During the relative position of the position of user's both hands and the left end of mapping two is carried out matched, calculate first
The relative position of user's both hands is as follows：{(x'11-x11, y '12-y12)、(x’21-x21, y '22-y22)、 (x’31-x31, y '32-
y32)、(x’41-x41, y '42-y42) (corresponding left hand).Then, by the above-mentioned relative position for the user's both hands being calculated with
Map the relative position { (x of two left ends5, y5), (x6, y6), (x7, y7), (x8, y8) matched, that is, calculating { (x '11-
x11, y '12-y12)、(x’21-x21, y '22-y22)、 (x’31-x31, y '32-y32)、(x’41-x41, y '42-y42) (corresponding left hand) with
{(x5, y5), (x6, y6), (x7, y7), (x8, y8) between similarity, it is assumed that the similarity being calculated be 95%.In the example
In son, if similarity threshold is 80%, two left ends of relative position and mapping for the user's both hands being calculated then are determined
Relative position is matched.As a result, in this example embodiment, the result of gesture identification is " explosion ".
Above step May Day and step 5 two, by between the respective multiple movements of both hands and position and prearranged gesture table
Matching, to determine that the gesture of user, the precision of identification are higher；When according to different application scene need change (such as modification, increase
Add deduct few etc.) to the definition of gesture when, do not need modification algorithm or re -training classifier, can be only by adjusting predetermined hand
The modes such as gesture title or the corresponding denomination of dive of gesture in gesture table realize the change to definition of gesture, substantially increase
The adaptability of algorithm.
Above-mentioned gesture identification method according to an embodiment of the present invention using first identification single-handed exercise and then passes through both hands
The mode of action recognition gesture realizes that can accurately identify the gesture of user, accuracy of identification is higher, and recognition speed compared with
Fastly.
Above-mentioned gesture identification method of the invention uses Hierarchical Design algorithm, and algorithm complexity is low, is easy to implement.
It is right when needing to change (such as modify, increase or decrease) in addition, using above-mentioned gesture identification method of the invention
When movement and/or the definition of gesture, can only (it be modified in advance that is, passing through by adjusting template and/or default gesture table to realize
If the corresponding denomination of dive of characteristic point sequence changes the definition of movement, by increasing or decreasing default characteristic point sequence and its right
Denomination of dive is answered to increase, subtract movement；Change the definition of gesture by modifying the corresponding multiple movements of gesture in default gesture table,
Increased by increasing or decreasing gesture in default gesture table and its respective action, subtract gesture), without changing algorithm or again
New training classifier, substantially increases the adaptability of algorithm.
In addition, the strong real-time of above-mentioned gesture identification method of the invention, can be suitble to the occasion of real-time interaction demand.
Although the embodiment according to limited quantity describes the present invention, above description, the art are benefited from
It is interior it is clear for the skilled person that in the scope of the present invention thus described, it can be envisaged that other embodiments.Additionally, it should be noted that
Language used in this specification primarily to readable and introduction purpose and select, rather than in order to explain or limit
Determine subject of the present invention and selects.Therefore, without departing from the scope and spirit of the appended claims, for this
Many modifications and changes are obvious for the those of ordinary skill of technical field.For the scope of the present invention, to this
Invent done disclosure be it is illustrative and not restrictive, it is intended that the scope of the present invention be defined by the claims appended hereto.
Claims (6)
1. a kind of gesture identification method, which is characterized in that the gesture identification method includes：
Step 1: obtaining the testing image sequence of user's hand containing depth information；
Step 2: being detected in every frame image of the testing image sequence according to image depth information and image color information
The hand profile of the user out；
Step 3: every hand of the user is directed to, using preset hand structure template, in the testing image sequence
The characteristic point sequence to be measured of this hand is determined in every frame image；
Step 4: being directed to every hand of the user, the feature to be measured of this hand is determined in multiple default characteristic point sequences
The matching sequence of point sequence, to determine denomination of dive and the position of this hand according to the matching sequence；
Step 5: the gesture that selection matches with the denomination of dive of user's both hands and position in default gesture table, makees
For the gesture identification result of the testing image sequence；
Wherein, step 5 includes：
Step 5 one presets following map listing as the default gesture table：Each mapping in the map listing
Left end be set title pair and each denomination of dive pair position；The right end of each mapping in the map listing is one
A gesture；
Step 5 two, denomination of dive and position by the left end of each mapping in the default gesture table and user's both hands
It sets and is matched, wherein the matching of denomination of dive executes stringent matching, and position is then respective average by user's both hands
Relative position information is calculated in position, and then calculates the similarity between the relative position information and the position for mapping left end and come
It realizes.
2. gesture identification method according to claim 1, which is characterized in that step 3 includes：
Step 3 one, every hand for the user, using preset hand structure template, in the testing image sequence
Every frame image hand profile in determine this hand predetermined number characteristic point；
Step 3 two, every hand for the user, using this hand in each frame image of the testing image sequence
Corresponding predetermined number characteristic point obtains the characteristic point sequence to be measured of this hand.
3. gesture identification method according to claim 1 or 2, which is characterized in that step 1 includes：
By capturing the image of user's hand in predetermined imaging region, Detection Method in Optical Image Sequences is obtainedWith
Infrared image sequenceFor the pixel at Detection Method in Optical Image Sequences the i-th frame image coordinate (x, y)
Value, andFor the pixel value at infrared image sequence the i-th frame image coordinate (x, y), taken out according to the following formula
Take the image sequence of user's both hands information：
Wherein, α, β, λ are parameter preset threshold value,For the image sequence of user's both hands containing depth information of acquisition
Column, as the testing image sequence, i=1,2 ..., M, the number of image frames that M includes by the testing image sequence.
4. gesture identification method according to claim 1 or 2, which is characterized in that step 2 includes：
For the testing image sequenceIn every frame imageThe color combining information deletion frame imageIn noise spot and non-area of skin color, using edge detection operator E () to deleting the noise spot and the non-skin
Obtained image behind color regionEdge detection is carried out, edge image is obtained
The edge imageIt as only include the image of user's hand profile.
5. gesture identification method according to claim 2, which is characterized in that step 3 one includes：
Following processing is executed for every frame image of the testing image sequence：According to the profile curvature of a curve in the image, look for
To the finger tip point in the contour line and refer to root artis；Using the finger tip point as setting base, each finger root singly referred to is matched
Artis obtains benchmark of each length singly referred to as scaling；Based on the finger tip point and the position for referring to root artis
It sets and each length singly referred to is zoomed in and out to the corresponding hand structure template and deformation, every hand is obtained by matching
Each articulations digitorum manus characteristic point and wrist midpoint characteristic point；
Wherein, the hand structure template includes left-handed configuration template and right hand configurations template, the left-handed configuration template and the right side
Hand stay in place form respectively includes：The fingertip characteristic point of each finger, each articulations digitorum manus characteristic point respectively refer to root joint characteristic point, in wrist
Topological relation between point feature point and each characteristic point.
6. gesture identification method according to claim 1 or 2, which is characterized in that step 4 includes：
Step 4 one, for the characteristic point sequence to be measured of every hand, the characteristic point sequence to be measured is divided according to predetermined time window
For multiple subsequences, and obtain the corresponding mean place of each subsequence；
Step 4 two is directed to the corresponding each subsequence of every hand, will be in the subsequence and the multiple default characteristic point sequence
Each matched respectively, select to be higher than with the matching degree of the subsequence in the multiple default characteristic point sequence pre-
If matching threshold and maximum default characteristic point sequence, the matching sequence as the subsequence；
It is step 4 three, the corresponding mean place of each subsequence is related to the corresponding denomination of dive of matching sequence of the subsequence
Connection；
Step 4 four is directed to every hand, and the matching sequence of the corresponding each subsequence of this hand is corresponding multiple as this hand
Sequence is matched, and using multiple matching corresponding denomination of dive of sequence as multiple denomination of dive of this hand.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510563293.9A CN105160323B (en) | 2015-09-07 | 2015-09-07 | A kind of gesture identification method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510563293.9A CN105160323B (en) | 2015-09-07 | 2015-09-07 | A kind of gesture identification method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105160323A CN105160323A (en) | 2015-12-16 |
| CN105160323B true CN105160323B (en) | 2018-11-27 |
Family
ID=54801175
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510563293.9A Expired - Fee Related CN105160323B (en) | 2015-09-07 | 2015-09-07 | A kind of gesture identification method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105160323B (en) |
Families Citing this family (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106971130A (en) * | 2016-01-14 | 2017-07-21 | 芋头科技（杭州）有限公司 | A kind of gesture identification method using face as reference |
| CN106971131A (en) * | 2016-01-14 | 2017-07-21 | 芋头科技（杭州）有限公司 | A kind of gesture identification method based on center |
| CN106971135A (en) * | 2016-01-14 | 2017-07-21 | 芋头科技（杭州）有限公司 | A kind of slip gesture recognition methods |
| CN106327486B (en) * | 2016-08-16 | 2018-12-28 | 广州视源电子科技股份有限公司 | Method and device for tracking finger web position |
| CN106648068A (en) * | 2016-11-11 | 2017-05-10 | 哈尔滨工业大学深圳研究生院 | Method for recognizing three-dimensional dynamic gesture by two hands |
| CN106650628B (en) * | 2016-11-21 | 2021-03-23 | 南京邮电大学 | Fingertip detection method based on three-dimensional K curvature |
| CN106791763B (en) * | 2016-11-24 | 2019-02-22 | 深圳奥比中光科技有限公司 | A kind of application specific processor for 3D display and 3D interaction |
| CN107038424B (en) * | 2017-04-20 | 2019-12-24 | 华中师范大学 | A gesture recognition method |
| CN111045511B (en) * | 2018-10-15 | 2022-06-07 | 华为技术有限公司 | Gesture-based control method and terminal equipment |
| CN109685037B (en) * | 2019-01-08 | 2021-03-05 | 北京汉王智远科技有限公司 | Real-time action recognition method and device and electronic equipment |
| CN110298314A (en) * | 2019-06-28 | 2019-10-01 | 海尔优家智能科技（北京）有限公司 | The recognition methods of gesture area and device |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102426480A (en) * | 2011-11-03 | 2012-04-25 | 康佳集团股份有限公司 | Human-computer interaction system and real-time gesture tracking processing method thereof |
| CN104281265A (en) * | 2014-10-14 | 2015-01-14 | 京东方科技集团股份有限公司 | Application program control method, application program control device and electronic equipment |
| CN104571482A (en) * | 2013-10-22 | 2015-04-29 | 中国传媒大学 | Digital device control method based on somatosensory recognition |
| CN104598915A (en) * | 2014-01-24 | 2015-05-06 | 深圳奥比中光科技有限公司 | Gesture recognition method and gesture recognition device |
| CN104750397A (en) * | 2015-04-09 | 2015-07-01 | 重庆邮电大学 | Somatosensory-based natural interaction method for virtual mine |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2003071410A2 (en) * | 2002-02-15 | 2003-08-28 | Canesta, Inc. | Gesture recognition system using depth perceptive sensors |
-
2015
- 2015-09-07 CN CN201510563293.9A patent/CN105160323B/en not_active Expired - Fee Related
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102426480A (en) * | 2011-11-03 | 2012-04-25 | 康佳集团股份有限公司 | Human-computer interaction system and real-time gesture tracking processing method thereof |
| CN104571482A (en) * | 2013-10-22 | 2015-04-29 | 中国传媒大学 | Digital device control method based on somatosensory recognition |
| CN104598915A (en) * | 2014-01-24 | 2015-05-06 | 深圳奥比中光科技有限公司 | Gesture recognition method and gesture recognition device |
| CN104281265A (en) * | 2014-10-14 | 2015-01-14 | 京东方科技集团股份有限公司 | Application program control method, application program control device and electronic equipment |
| CN104750397A (en) * | 2015-04-09 | 2015-07-01 | 重庆邮电大学 | Somatosensory-based natural interaction method for virtual mine |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105160323A (en) | 2015-12-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105160323B (en) | A kind of gesture identification method | |
| US8254627B2 (en) | Method for automatically following hand movements in an image sequence | |
| CN105045398B (en) | A kind of virtual reality interactive device based on gesture identification | |
| CN105045399B (en) | A kind of electronic equipment with 3D camera assemblies | |
| CN105069444B (en) | A kind of gesture identifying device | |
| CN107563494B (en) | First-view-angle fingertip detection method based on convolutional neural network and heat map | |
| CN103135758B (en) | Realize the method and system of shortcut function | |
| CN105320248B (en) | Aerial gesture input method and device | |
| CN101901052B (en) | Target control method based on mutual reference of both hands | |
| CN107885327B (en) | A Fingertip Detection Method Based on Kinect Depth Information | |
| CN105493078B (en) | Colored sketches picture search | |
| CN105302295B (en) | A kind of virtual reality interactive device with 3D camera assemblies | |
| CN105302294B (en) | A kind of interactive virtual reality apparatus for demonstrating | |
| KR100660725B1 (en) | Handheld terminal with face tracking device | |
| CN103488294B (en) | A kind of Non-contact gesture based on user's interaction habits controls to map method of adjustment | |
| KR20160101973A (en) | System and method for identifying faces in unconstrained media | |
| CN101593022A (en) | A Fast Human-Computer Interaction Method Based on Fingertip Tracking | |
| CN106503651B (en) | A kind of extracting method and system of images of gestures | |
| US11282257B2 (en) | Pose selection and animation of characters using video data and training techniques | |
| CN110147767A (en) | Three-dimension gesture attitude prediction method based on two dimensional image | |
| CN105912126B (en) | A kind of gesture motion is mapped to the adaptive adjusting gain method at interface | |
| CN105068662B (en) | A kind of electronic equipment for man-machine interaction | |
| CN105046249B (en) | A kind of man-machine interaction method | |
| CN109948450A (en) | A kind of user behavior detection method, device and storage medium based on image | |
| CN205080499U (en) | Mutual equipment of virtual reality based on gesture recognition |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: 150016 Heilongjiang Province, Harbin Economic Development Zone haping Road District Dalian road and Xingkai road junction Applicant after: HARBIN YISHE TECHNOLOGY Co.,Ltd. Address before: 150016 Heilongjiang City, Harbin province Daoli District, quiet street, unit 54, unit 2, layer 4, No. 3 Applicant before: HARBIN YISHE TECHNOLOGY Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20181127 |