CN106327486B - Method and device for tracking finger web position - Google Patents
Method and device for tracking finger web position Download PDFInfo
- Publication number
- CN106327486B CN106327486B CN201610675192.5A CN201610675192A CN106327486B CN 106327486 B CN106327486 B CN 106327486B CN 201610675192 A CN201610675192 A CN 201610675192A CN 106327486 B CN106327486 B CN 106327486B
- Authority
- CN
- China
- Prior art keywords
- profile
- point
- hand
- finger web
- finger
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
- G06V40/28—Recognition of hand or arm movements, e.g. recognition of deaf sign language
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Psychiatry (AREA)
- Social Psychology (AREA)
- Human Computer Interaction (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
- Collating Specific Patterns (AREA)
Abstract
The invention discloses a method for tracking the position of a webbed finger, which comprises the following steps: acquiring a depth image and a color image of the same image of the hand of a user; extracting an outer contour of the hand from the depth image; selecting contour points which are farthest away from a straight line connecting the adjacent fingertips from contour points between the adjacent fingertips of the outer contour as contour inflection points; and taking the contour inflection point as the current position of the finger web, and carrying out iterative correction on the current position of the finger web by utilizing the color image to obtain the output position of the finger web. By adopting the embodiment of the invention, the position information of the webbed finger can be accurately tracked from the image data.
Description
Technical field
The present invention relates to image technique process field more particularly to a kind of method and device thereof for tracking the finger web position.
Background technique
The finger web refers at metacarpal head, the across grain of aponeurosis palmaris deep layer and its four beam stringer fiber issued to distal end it
Between three fibre gaps being surrounded, be the channel between the palm of palm, the back of the hand and finger, back side.And it is tried in virtual ring
Or other are needed to track in the application scenarios of hand the finger web location information of user, it is difficult to the image data shot by camera
Accurately extract the location information of the finger web.
Summary of the invention
The embodiment of the present invention proposes a kind of method for extracting the finger web position, and the finger web can be accurately traced into from image data
Location information.
In a first aspect, the embodiment of the present invention provides a kind of method for tracking the finger web position, comprising:
Obtain the depth image and color image of the same image of record user's hand；
The outer profile of the hand is extracted from the depth image；
From the profile point between the adjacent finger tip of the outer profile, selected distance connects the straight line of the adjacent finger tip most
Remote profile point is as profile inflection point；
Using the profile inflection point as the current location of the finger web, using the color image to the current location of the finger web
It is iterated amendment, obtains the output position of the finger web.
With reference to first aspect, in the first possible implementation of the first aspect, described from the depth image
The outer profile of the hand is extracted, specifically:
According to preset hand joint point model, each artis of the hand is calculated from the depth image
Depth；
Take the intermediate value of the depth for having artis as reference depth dref；
Depth is extracted from the depth image in hand depth bounds [dref-δ,dref+ δ] in region outer profile；
Wherein, δ is the parameter value for measuring thickness between the back of the hand and palm of the hand；
The average distance that artis described in the centroid distance of profile is chosen from the outer profile is nearest, and contour curve is total
Outer profile of the longest outer profile of length as the hand.
With reference to first aspect, in the second possible implementation of the first aspect, described to be with the profile inflection point
The current location of the finger web is iterated amendment to the current location of the finger web using the color image of the hand, obtains institute
The output position of the finger web is stated, specifically:
Using the profile inflection point as the current location of the finger web, extract from the color image of the hand with the present bit
It is set to the regional area of central point；
The current location is subjected to offset and obtains multiple deviation posts, and for each deviation post, from the hand
Point is extracted using centered on the deviation post in the color image in portion and with the region of the regional area same shape as candidate
Region；
Calculate the structural deviation degree of each described candidate region Yu the regional area；
If the structural deviation degree of each described candidate region and the regional area is all larger than preset threshold, by institute
State output position of the current location as the finger web；
The structural deviation degree of a candidate region and the regional area is not more than the preset threshold if it exists
When, then it is described to update to choose deviation post corresponding to the smallest candidate region of structural deviation degree with the regional area
Current location, and update the regional area and the candidate region.
In the possible implementation of second with reference to first aspect, in the third possible implementation of first aspect
In, for each candidate region, the structural deviation degree of the candidate region and the regional area is d (P, Q),Wherein, P is the pixel value of each pixel comprising the regional area
Set, Q be each pixel comprising the candidate region pixel value set, μPFor all pixels value in set P
Mean value, μQFor the mean value of all pixels value in set Q, σPQFor the covariance of set P and set Q, σPFor the variance of set P, σQ
For the variance of set Q, c1And c2For preset constant.
In the possible implementation of second with reference to first aspect, in the 4th kind of possible implementation of first aspect
In, the current location is (x, y), then the deviation post is (x+ δx,y+δy)；Wherein, δx∈ { -1,0,1 }, δy∈{-1,
, and δ 0,1 }xAnd δyIt is not simultaneously 0.
In the possible implementation of second with reference to first aspect, in the 5th kind of possible implementation of first aspect
In, before using the profile inflection point as the current location of the finger web, further includes:
A continuous profile points of the left side M that the abscissa of the profile inflection point is revised the profile inflection point and right side M are a
The intermediate value of the abscissa of continuous profile point, and the ordinate of the profile inflection point is changed to the left side of the profile inflection point
The intermediate value of the ordinate of M continuous profile points and right side M continuous profile points；
Gaussian Blur processing is carried out to the color image.
With reference to first aspect, in the sixth possible implementation of the first aspect, from the adjacent of the outer profile
In profile point between finger tip, selected distance connect the farthest profile point of straight line of the adjacent finger tip as profile inflection point it
Before, further includes:
For each of outer profile profile point, it is left that the abscissa of the profile point is changed to the profile point
The mean value of the abscissa of the N number of continuous profile point of the N number of continuous profile point in side and right side, and, the ordinate of the profile point is repaired
Order the mean value of the ordinate for continuous profile point N number of on the left of the profile point and the N number of continuous profile point in right side.
Correspondingly, in second aspect, the present invention also provides a kind of devices for tracking the finger web position, comprising:
Image collection module, the depth image and color image of the same image for obtaining record user's hand；
Outside contour extraction module, for extracting the outer profile of the hand from the depth image；
Profile inflection point chooses module, for from the profile point between the adjacent finger tip of the outer profile, selected distance to connect
The farthest profile point of straight line of the adjacent finger tip is connect as profile inflection point；
The finger web position determination module, for utilizing the cromogram using the profile inflection point as the current location of the finger web
As being iterated amendment to the current location of the finger web, the output position of the finger web is obtained.
In conjunction with second aspect, in the first possible implementation of the second aspect, the Outside contour extraction module, tool
Body includes:
Artis depth calculation unit, for being calculated from the depth image according to preset hand joint point model
The depth of each artis of the hand out；
Reference depth determination unit, for taking the intermediate value of the depth for having artis as reference depth dref；
Contours extract unit, for extracting depth from the depth image in hand depth bounds [dref-δ,dref+δ]
The outer profile in interior region；Wherein, δ is the parameter value for measuring thickness between the back of the hand and palm of the hand；
Profile selection unit, for choosing the average distance of artis described in the centroid distance of profile from the outer profile
Recently, and outer profile of the longest outer profile of contour curve total length as the hand.
In conjunction with second aspect, in a second possible implementation of the second aspect, the finger web position determination module,
It specifically includes:
Regional area determination unit, for using the profile inflection point as the current location of the finger web, from the colour of the hand
The regional area put centered on the current location is extracted in image；
Candidate region determination unit obtains multiple deviation posts for the current location to be carried out offset, and for every
One deviation post, from the color image of the hand extract centered on the deviation post point and with the regional area phase
The region of similar shape is as candidate region；
Extent of deviation computing unit, for calculating the structural deviation journey of each described candidate region Yu the regional area
Degree；
Output position determination unit, for working as the structural deviation degree of each described candidate region and the regional area
When being all larger than preset threshold, using the current location as the output position of the finger web；
Current location updating unit, for when there are the structural deviation journeys of a candidate region and the regional area
When degree is no more than the preset threshold, then choose corresponding to the smallest candidate region of structural deviation degree with the regional area
Deviation post update the current location, and then update the regional area and the candidate region.
In conjunction with second of possible implementation of second aspect, in the third possible implementation of second aspect
In, the finger web position determination module further include:
Profile inflection point revises unit, for the abscissa of the profile inflection point to be revised to left side M of the profile inflection point
The intermediate value of the abscissa of continuous profile point and right side M continuous profile points, and the ordinate of the profile inflection point is repaired
Order the intermediate value for left side M continuous profile points of the profile inflection point and the ordinate of right side M continuous profile points；
High phase Fuzzy Processing unit, for carrying out Gaussian Blur processing to the color image.
In conjunction with second aspect, in the fourth possible implementation of the second aspect, the dress of the tracking the finger web position
It sets further include:
Profile point revises module, is used for for each of outer profile profile point, by the horizontal seat of the profile point
Mark is changed to the mean value of the abscissa of N number of continuous profile point and the N number of continuous profile point in right side on the left of the profile point, and, it will
The ordinate of the profile point is changed to the vertical seat of N number of continuous profile point and the N number of continuous profile point in right side on the left of the profile point
Target mean value.
The implementation of the embodiments of the present invention has the following beneficial effects:
The method and apparatus of tracking the finger web position provided in an embodiment of the present invention, by obtaining the same of record user's hand
The depth image and color image of image；The outer profile of the hand is extracted from the depth image；From the outer profile
In profile point between adjacent finger tip, the farthest profile point of straight line that selected distance connects the adjacent finger tip is turned as profile
Point；Using the profile inflection point as the current location of the finger web, carried out using current location of the color image to the finger web
Iterated revision, to obtain the output position of the finger web, to accurately trace into the location information of the finger web from image data.
Detailed description of the invention
Fig. 1 is the flow diagram of one embodiment of the method for tracking the finger web position provided by the invention；
Fig. 2 is the flow diagram of one embodiment of the step S4 in the method for the tracking the finger web position that Fig. 1 is provided；
Fig. 3 is the flow diagram of another embodiment of the step S4 in the method for the tracking the finger web position that Fig. 1 is provided；
Fig. 4 is the structural schematic diagram of one embodiment of the device of tracking the finger web position provided by the invention；
Fig. 5 is the structure of one embodiment of the Outside contour extraction module of the device of tracking the finger web position provided by the invention
Schematic diagram；
Fig. 6 is the knot of one embodiment of the finger web position determination module of the device of tracking the finger web position provided by the invention
Structure schematic diagram.
Specific embodiment
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete
Site preparation description, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.It is based on
Embodiment in the present invention, it is obtained by those of ordinary skill in the art without making creative efforts every other
Embodiment shall fall within the protection scope of the present invention.
It is the flow diagram of one embodiment of the method for tracking the finger web position provided by the invention, the party referring to Fig. 1
Method includes step S1 to S4, specific as follows:
S1 obtains the depth image and color image of the same image of record user's hand；
S2 extracts the outer profile of the hand from the depth image；
S3, from the profile point between the adjacent finger tip of the outer profile, selected distance connects the straight of the adjacent finger tip
The farthest profile point of line is as profile inflection point；
S4, using the profile inflection point as the current location of the finger web, using the color image to the current of the finger web
Position is iterated amendment, obtains the output position of the finger web.
It should be noted that depth image is the image for capturing subject by depth camera device, included
Each pixel pixel value reflection be between subject positional distance camera corresponding with the pixel
Range information；Color image is the image that subject is captured by common photographic device, each picture for being included
What the pixel value of vegetarian refreshments reflected is the appearance color information of subject position corresponding with the pixel.In addition, hand
The finger web it is general there are four, the acquisition of the output position of each the finger web can be obtained according to above-mentioned steps S1 to step S4
It takes.
When getting the outer profile of hand by step S2, outer profile at this time is the collection being made of a group coordinate points
It closes, is connected between two neighboring coordinate points by straight line, then outer profile is interconnected to by multiple thin short broken lines, is needed at this time
Slight smooth operation is carried out to outer profile, specific as follows:
For each of outer profile profile point, it is left that the abscissa of the profile point is changed to the profile point
The mean value of the abscissa of the N number of continuous profile point of the N number of continuous profile point in side and right side, and, the ordinate of the profile point is repaired
Order the mean value of the ordinate for continuous profile point N number of on the left of the profile point and the N number of continuous profile point in right side.The number of N herein
Value can be configured according to actual needs.
And the determination for above-mentioned profile inflection point, there are two types of modes:
First, seek convex closure to the outer profile, the straight line that adjacent finger tip is connected in convex closure is then chosen, and then from adjacent finger
The farthest profile point of the selected distance straight line is as profile inflection point in profile point between point；
Second, the finger tip is expressed as the finger tip described in distance in the profile point for surrounding the corresponding artis of the finger tip
Then the farthest profile point of corresponding artis connects adjacent finger tip and obtains straight line, and then from the wheel between the adjacent finger tip
The farthest profile point of the selected distance straight line is as profile inflection point in exterior feature point.
It should be noted that by the obtained profile inflection point of aforesaid operations can not directly as the output position of the finger web,
Since the profile extracted is easy by the profile point on image background, the influence of finger gesture and outer profile with noise
Influence, thus directly extract the profile inflection point as the output position of the finger web there are biggish ambiguousness, for example, profile inflection point
Somewhere position in finger seam, and is not at the finger web, then need at this time through the above steps S4 profile inflection point is carried out
Amendment, obtains the actual location (output position of the finger web) of the finger web.
And the outer profile of the hand is extracted from depth image for above-mentioned steps S2, specific implementation process are as follows:
According to preset hand joint point model, each artis of the hand is calculated from the depth image
Depth；
Take the intermediate value of the depth for having artis as reference depth dref；
Depth is extracted from the depth image in hand depth bounds [dref-δ,dref+ δ] in region outer profile；
Wherein, δ is the parameter value for measuring thickness between the back of the hand and palm of the hand；
The average distance that artis described in the centroid distance of profile is chosen from the outer profile is nearest, and contour curve is total
Outer profile of the longest outer profile of length as the hand.
It should be noted that hand joint point model is the training for advancing with the depth image that a large amount of record has hand
Collection trains the model come, which includes: hand joint point track model, more Random Forest models etc. based on kinect,
It is the information training generation of the depth image based on hand, preferably utilizes random forests algorithm training hand joint point mould
Type.Hand joint point provides the approximate location of each artis of hand, and is estimated that entirely by the depth of each artis
The depth bounds of hand.In addition, in a few cases, the partial joint point calculated may also exceed the hand because of precision deficiency
Region, or because depth image noise causes the depth error of artis larger, thus, in order to reduce these abnormal joint points
Influence, take the intermediate value of the depth of the artis as reference depth, then the depth of entire hand is in hand depth bounds
[dref-δ,dref+ δ] in, δ is the parameter value for measuring thickness between the back of the hand and palm of the hand, then extracts within the scope of this
The outer profile at the edge in region comes out, as the outer profile of the hand.But due to the shadow by noise or other interference regions
Ring, the outer profile extracted might have it is multiple, at this point, therefrom choose profile centroid distance described in artis average departure
From nearest, and the longest outer profile of contour curve total length.
As shown in Fig. 2, Fig. 2 is the stream of one embodiment of the step S4 in the method for the tracking the finger web position that Fig. 1 is provided
Journey schematic diagram, the then specific embodiment of above-mentioned steps S4 are as follows:
S41 is extracted from the color image of the hand and is worked as with described using the profile inflection point as the current location of the finger web
The regional area put centered on front position；
The current location is carried out offset and obtains multiple deviation posts by S42, and for each deviation post, from institute
State in the color image of hand extract using centered on the deviation post point and with the region of the regional area same shape as
Candidate region；For example, then the deviation post is (x+ δ when the current location is (x, y)x,y+δy)；Wherein, δx∈{-
1,0,1 }, δy∈ { -1,0,1 }, and δxAnd δyIt is not simultaneously 0, δxAnd δySetting be not limited to as above-mentioned numerical value, can be according to reality
Border situation is adjusted.
S43 calculates the structural deviation degree of each described candidate region Yu the regional area；For each candidate
The structural deviation degree of region, the candidate region and the regional area is d (P, Q),Wherein, P is the pixel value of each pixel comprising the regional area
Set, Q be each pixel comprising the candidate region pixel value set, μPFor all pixels value in set P
Mean value, μQFor the mean value of all pixels value in set Q, σPQFor the covariance of set P and set Q, σPFor the variance of set P, σQ
For the variance of set Q, c1And c2For preset constant.Since candidate region is to mention from the color image of hand with regional area
It takes out, then the above-mentioned structural deviation degree can face on each point of both accurate description candidate region and regional area
Color is distributed (pixel Distribution value) situation.
S44, if the structural deviation degree of each described candidate region and the regional area is all larger than preset threshold,
Using the current location as the output position of the finger web；
S45, the structural deviation degree of a candidate region and the regional area is default no more than described if it exists
When threshold value, then deviation post corresponding to the smallest candidate region of structural deviation degree with the regional area is chosen to update
The current location, and update the regional area and the candidate region.
It should be noted that when regional area is in the finger web, the color point of the regional area and neighbouring candidate region
Cloth (i.e. pixel Distribution value) has bigger difference；When regional area is in webs, the regional area with along webs direction
The distribution of color difference of neighbouring candidate region is relatively small, and the regional area and the distribution of color of other candidate regions are poor
It is larger.Thus, when compare structural deviation degree of each described candidate region with the regional area be all larger than it is default
When threshold value, it can judge that regional area is fallen in the finger web, by center (above-mentioned current location) conduct of the regional area
The output position of the finger web, to complete the amendment to the output position of the finger web；Conversely, can determine whether out that the regional area falls in webs
On, it needs to continue to be modified the current location of the finger web, and choose the smallest with the structural deviation degree of the regional area
Center corresponding to candidate region (above-mentioned deviation post) is updated to the current location of the finger web, it can be ensured that subsequent update
The current location of the finger web afterwards is displaced on the position of non-webs still on webs.
As shown in figure 3, Fig. 3 is another embodiment of the step S4 in the method for the tracking the finger web position that Fig. 1 is provided
Flow diagram；On the basis of upper one embodiment, i.e., before above-mentioned steps S41 further include:
The abscissa of the profile inflection point is revised left side M continuous profile points and the right side of the profile inflection point by S46
The intermediate value of the abscissa of M continuous profile points, and the ordinate of the profile inflection point is changed to the profile inflection point
The intermediate value of the ordinate of left side M continuous profile points and right side M continuous profile points；
S47 carries out Gaussian Blur processing to the color image.This step can filter out influence of the noise to color image,
Convenient for the judgement of subsequent step.
The method of the tracking the finger web position of above-mentioned offer be not limited to tracking human hands, can also track foot or other
By mold structure at hand or foot or even animal hand or foot.In addition, the side of the tracking the finger web position of above-mentioned offer
Method can be applied to VR (Virtual Reality, virtual reality), for example, virtual ring is tried on, it may be assumed that user is in client
Hand is lifted before camera, which shoots the hand of user, and the depth image taken and colored images' transmission are arrived
Another receiving end, receiving end are traced into from depth image and color image according to the method for tracking the finger web position provided above
The accurate location of the finger web of the hand of user, then receiving end determines ring on finger according to the position of two neighboring the finger web
Wearing position, and ring is worn on the user finger wearing position on image back to client, and in client
It is shown on the display at end, so that user can know that it wears the effect of ring from the image in display.
The method of tracking the finger web position provided in an embodiment of the present invention, records the same image of user's hand by acquisition
Depth image and color image；The outer profile of the hand is extracted from the depth image；From the adjacent finger of the outer profile
In profile point between point, selected distance connects the farthest profile point of straight line of the adjacent finger tip as profile inflection point；With institute
Current location of the profile inflection point as the finger web is stated, is iterated and is repaired using current location of the color image to the finger web
Just, the output position of the finger web is obtained, to accurately trace into the location information of the finger web from image data.
Correspondingly, tracking provided by the above embodiment is able to achieve the present invention also provides a kind of device for tracking the finger web position to refer to
Whole processes of the method for web position are referring to fig. 4 one embodiment of the device of tracking the finger web position provided by the invention
Structural schematic diagram, the device specifically include:
Image collection module 10, the depth image and color image of the same image for obtaining record user's hand；
Outside contour extraction module 20, for extracting the outer profile of the hand from the depth image；
Profile inflection point chooses module 30, for from the profile point between the adjacent finger tip of the outer profile, selected distance
The farthest profile point of straight line of the adjacent finger tip is connected as profile inflection point；
The finger web position determination module 40, for utilizing the colour using the profile inflection point as the current location of the finger web
Image is iterated amendment to the current location of the finger web, obtains the output position of the finger web.
It is offer of the present invention referring to Fig. 5 in the first possible implementation of the second aspect in conjunction with second aspect
Tracking the finger web position device Outside contour extraction module one embodiment structural schematic diagram, the Outside contour extraction mould
Block 20, specifically includes:
Artis depth calculation unit 21, for being fallen into a trap from the depth image according to preset hand joint point model
Calculate the depth of each artis of the hand；
Reference depth determination unit 22, for taking the intermediate value of the depth for having artis as reference depth dref；
Contours extract unit 23, for extracting depth from the depth image in hand depth bounds [dref-δ,dref+
δ] in region outer profile；Wherein, δ is the parameter value for measuring thickness between the back of the hand and palm of the hand；
Profile selection unit 24, for choosing the average departure of artis described in the centroid distance of profile from the outer profile
From nearest, and outer profile of the longest outer profile of contour curve total length as the hand.
It is offer of the present invention referring to Fig. 6 in a second possible implementation of the second aspect in conjunction with second aspect
Tracking the finger web position device the finger web position determination module one embodiment structural schematic diagram, the finger web position is true
Cover half block 40, specifically includes:
Regional area determination unit 41, for using the profile inflection point as the current location of the finger web, from the coloured silk of the hand
The regional area put centered on the current location is extracted in chromatic graph picture；
Candidate region determination unit 42 obtains multiple deviation posts for the current location to be carried out offset, and for
Each deviation post, from the color image of the hand extract centered on the deviation post point and with the regional area
The region of same shape is as candidate region；For example, then the deviation post is (x+ when the current location is (x, y)
δx,y+δy)；Wherein, δx∈ { -1,0,1 }, δy∈ { -1,0,1 }, and δxAnd δyIt is not simultaneously 0.
Extent of deviation computing unit 43, for calculating the structural deviation of each candidate region and the regional area
Degree；For each candidate region, the structural deviation degree of the candidate region and the regional area is d (P, Q),Wherein, P is the pixel of each pixel comprising the regional area
The set of value, Q are the set of the pixel value of each pixel comprising the candidate region, μPFor all pixels in set P
The mean value of value, μQFor the mean value of all pixels value in set Q, σPQFor the covariance of set P and set Q, σPFor the side of set P
Difference, σQFor the variance of set Q, c1And c2For preset constant.
Output position determination unit 44, for working as the structural deviation journey of each described candidate region and the regional area
When degree is all larger than preset threshold, using the current location as the output position of the finger web；
Current location updating unit 45, for when there are the structural deviations of the candidate region and the regional area
When degree is not more than the preset threshold, then it is right with the smallest candidate region institute of the structural deviation degree of the regional area to choose
The deviation post answered updates the current location, and then updates the regional area and the candidate region.
In conjunction with second of possible implementation of second aspect, in the third possible implementation of second aspect
In, referring to Fig. 6, the finger web position determination module 40 further include:
Profile inflection point revises unit 46, for the abscissa of the profile inflection point to be revised to the left side M of the profile inflection point
The intermediate value of the abscissa of a continuous profile point and right side M continuous profile points, and by the ordinate of the profile inflection point
It is changed to the intermediate value of left side M continuous profile points of the profile inflection point and the ordinate of right side M continuous profile points；
High phase Fuzzy Processing unit 47, for carrying out Gaussian Blur processing to the color image.
In conjunction with second aspect, in the fourth possible implementation of the second aspect, as shown in figure 4, the tracking refers to
The device of web position further include:
Profile point revises module 50, is used for for each of outer profile profile point, by the cross of the profile point
Coordinate is changed to the mean value of the abscissa of N number of continuous profile point and the N number of continuous profile point in right side on the left of the profile point, and,
The ordinate of the profile point is changed to the vertical of N number of continuous profile point and the N number of continuous profile point in right side on the left of the profile point
The mean value of coordinate.
The device of tracking the finger web position provided in an embodiment of the present invention, records the same image of user's hand by acquisition
Depth image and color image；The outer profile of the hand is extracted from the depth image；From the adjacent finger of the outer profile
In profile point between point, selected distance connects the farthest profile point of straight line of the adjacent finger tip as profile inflection point；With institute
Current location of the profile inflection point as the finger web is stated, is iterated and is repaired using current location of the color image to the finger web
Just, the output position of the finger web is obtained, to accurately trace into the location information of the finger web from image data.
Those of ordinary skill in the art will appreciate that realizing all or part of the process in above-described embodiment method, being can be with
Relevant hardware is instructed to complete by computer program, the program can be stored in a computer-readable storage medium
In, the program is when being executed, it may include such as the process of the embodiment of above-mentioned each method.Wherein, the storage medium can be magnetic
Dish, CD, read-only memory (Read-Only Memory, ROM) or random access memory (Random Access
Memory, RAM) etc..
The above is a preferred embodiment of the present invention, it is noted that for those skilled in the art
For, various improvements and modifications may be made without departing from the principle of the present invention, these improvements and modifications are also considered as
Protection scope of the present invention.
Claims (10)
1. a kind of method for tracking the finger web position characterized by comprising
Obtain the depth image and color image of the same image of record user's hand；
The outer profile of the hand is extracted from the depth image；
From the profile point between the adjacent finger tip of the outer profile, the straight line that selected distance connects the adjacent finger tip is farthest
Profile point is as profile inflection point；
Using the profile inflection point as the current location of the finger web, carried out using current location of the color image to the finger web
Iterated revision obtains the output position of the finger web；
It is described using the profile inflection point as the current location of the finger web, using the color image of the hand to the current of the finger web
Position is iterated amendment, obtains the output position of the finger web, specifically:
Using the profile inflection point as the current location of the finger web, extraction is from the color image of the hand with the current location
The regional area of central point；
The current location is subjected to offset and obtains multiple deviation posts, and for each deviation post, from the hand
Point is extracted using centered on the deviation post in color image and with the region of the regional area same shape as candidate region；
Calculate the structural deviation degree of each described candidate region Yu the regional area；
If the structural deviation degree of each described candidate region and the regional area is all larger than preset threshold, work as by described in
Output position of the front position as the finger web；
When the structural deviation degree of a candidate region and the regional area is not more than the preset threshold if it exists, then
It is described current to update to choose deviation post corresponding to the smallest candidate region of structural deviation degree with the regional area
Position, and update the regional area and the candidate region.
2. the method for tracking the finger web position as described in claim 1, which is characterized in that described to be extracted from the depth image
The outer profile of the hand, specifically:
According to preset hand joint point model, the depth of each artis of the hand is calculated from the depth image
Degree；
Take the intermediate value of the depth of all artis as reference depth dref；
Depth is extracted from the depth image in hand depth bounds [dref-δ,dref+ δ] in region outer profile；Wherein,
δ is the parameter value for measuring thickness between the back of the hand and palm of the hand；
The average distance that artis described in the centroid distance of profile is chosen from the outer profile is nearest, and contour curve total length
Outer profile of the longest outer profile as the hand.
3. the method for tracking the finger web position as described in claim 1, which is characterized in that
For each candidate region, the structural deviation degree of the candidate region and the regional area is d (P, Q),Wherein, P is the pixel value of each pixel comprising the regional area
Set, Q be each pixel comprising the candidate region pixel value set, μPFor all pixels value in set P
Mean value, μQFor the mean value of all pixels value in set Q, σPQFor the covariance of set P and set Q, σPFor the variance of set P, σQ
For the variance of set Q, c1And c2For preset constant.
4. the method for tracking the finger web position as described in claim 1, which is characterized in that the current location is (x, y), then institute
Stating deviation post is (x+ δx,y+δy)；Wherein, δx∈ { -1,0,1 }, δy∈ { -1,0,1 }, and δxAnd δyIt is not simultaneously 0.
5. the method for tracking the finger web position as described in claim 1, which is characterized in that using the profile inflection point as the finger web
Before current location, further includes:
A continuous profile points of the left side M that the abscissa of the profile inflection point is revised the profile inflection point and right side M are a continuous
Profile point abscissa intermediate value, and the ordinate of the profile inflection point is changed to the left side M of the profile inflection point
The intermediate value of the ordinate of continuous profile point and right side M continuous profile points；
Gaussian Blur processing is carried out to the color image.
6. the method for tracking the finger web position as described in claim 1, which is characterized in that in the adjacent finger tip from the outer profile
Between profile point in, before selected distance connects the farthest profile point of straight line of the adjacent finger tip as profile inflection point, also
Include:
For each of outer profile profile point, the abscissa of the profile point is changed to N on the left of the profile point
The mean value of the abscissa of the N number of continuous profile point of a continuous profile point and right side, and, the ordinate of the profile point is changed to
The mean value of the ordinate of N number of continuous profile point and the N number of continuous profile point in right side on the left of the profile point.
7. a kind of device for tracking the finger web position characterized by comprising
Image collection module, the depth image and color image of the same image for obtaining record user's hand；
Outside contour extraction module, for extracting the outer profile of the hand from the depth image；
Profile inflection point chooses module, for from the profile point between the adjacent finger tip of the outer profile, selected distance to connect institute
The farthest profile point of straight line of adjacent finger tip is stated as profile inflection point；
The finger web position determination module, for utilizing the color image pair using the profile inflection point as the current location of the finger web
The current location of the finger web is iterated amendment, obtains the output position of the finger web；
The finger web position determination module, specifically includes:
Regional area determination unit, for using the profile inflection point as the current location of the finger web, from the color image of the hand
It is middle to extract the regional area put centered on the current location；
Candidate region determination unit obtains multiple deviation posts for the current location to be carried out offset, and for each
Deviation post, from the color image of the hand extract centered on the deviation post point and with the regional area phase similar shape
The region of shape is as candidate region；
Extent of deviation computing unit, for calculating the structural deviation degree of each described candidate region Yu the regional area；
Output position determination unit is big for working as each described candidate region and the structural deviation degree of the regional area
When preset threshold, using the current location as the output position of the finger web；
Current location updating unit, for when the structural deviation degree that there is a candidate region and the regional area not
When greater than the preset threshold, then choose inclined corresponding to the smallest candidate region of structural deviation degree with the regional area
Pan position updates the current location, and then updates the regional area and the candidate region.
8. the device of tracking the finger web position as claimed in claim 7, which is characterized in that the Outside contour extraction module, specifically
Include:
Artis depth calculation unit, for calculating institute from the depth image according to preset hand joint point model
State the depth of each artis of hand；
Reference depth determination unit, for take all artis depth intermediate value as reference depth dref；
Contours extract unit, for extracting depth from the depth image in hand depth bounds [dref-δ,dref+ δ] in
The outer profile in region；Wherein, δ is the parameter value for measuring thickness between the back of the hand and palm of the hand；
Profile selection unit, for choosing the average distance of artis described in the centroid distance of profile from the outer profile most
Closely, and outer profile of the longest outer profile of contour curve total length as the hand.
9. the device of tracking the finger web position as claimed in claim 7, which is characterized in that the finger web position determination module is also wrapped
It includes:
Profile inflection point revises unit, and left side M for the abscissa of the profile inflection point to be revised to the profile inflection point are continuous
Profile point and the continuous profile point of right side M abscissa intermediate value, and the ordinate of the profile inflection point is changed to
The intermediate value of the ordinate of the continuous profile point of left side M of the profile inflection point and right side M continuous profile points；
High phase Fuzzy Processing unit, for carrying out Gaussian Blur processing to the color image.
10. the device of tracking the finger web position as claimed in claim 7, which is characterized in that the device of the tracking the finger web position
Further include:
Profile point revises module, for for each of outer profile profile point, the abscissa of the profile point to be repaired
The mean value of the abscissa for continuous profile point N number of on the left of the profile point and the N number of continuous profile point in right side is ordered, and, it will be described
The ordinate of profile point is changed to the ordinate of N number of continuous profile point and the N number of continuous profile point in right side on the left of the profile point
Mean value.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610675192.5A CN106327486B (en) | 2016-08-16 | 2016-08-16 | Method and device for tracking finger web position |
| PCT/CN2016/113492 WO2018032700A1 (en) | 2016-08-16 | 2016-12-30 | Method for tracking finger web position and apparatus thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610675192.5A CN106327486B (en) | 2016-08-16 | 2016-08-16 | Method and device for tracking finger web position |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106327486A CN106327486A (en) | 2017-01-11 |
| CN106327486B true CN106327486B (en) | 2018-12-28 |
Family
ID=57739990
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610675192.5A Active CN106327486B (en) | 2016-08-16 | 2016-08-16 | Method and device for tracking finger web position |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN106327486B (en) |
| WO (1) | WO2018032700A1 (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112083870B (en) * | 2020-09-09 | 2022-03-22 | 青岛海信商用显示股份有限公司 | Information identification method and equipment |
| CN116309349A (en) * | 2023-02-15 | 2023-06-23 | 珞石（北京）科技有限公司 | A Vision-Based Cloth Sewing Trajectory Calibration Method |
| CN117393485B (en) * | 2023-11-01 | 2024-05-03 | 东莞触点智能装备有限公司 | Chip high-precision die bonder visual positioning system based on deep learning |
| CN117495967B (en) * | 2023-12-29 | 2024-04-05 | 四川高速公路建设开发集团有限公司 | Tunnel face displacement field monitoring method |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103530892A (en) * | 2013-10-21 | 2014-01-22 | 清华大学深圳研究生院 | Kinect sensor based two-hand tracking method and device |
| CN103886303A (en) * | 2014-03-28 | 2014-06-25 | 上海云享科技有限公司 | Palmprint recognition method and device |
| CN104598915A (en) * | 2014-01-24 | 2015-05-06 | 深圳奥比中光科技有限公司 | Gesture recognition method and gesture recognition device |
| KR20150068550A (en) * | 2013-12-11 | 2015-06-22 | 삼성전자주식회사 | Method and apparatus for acquiring information of a front object. |
| CN104899600A (en) * | 2015-05-28 | 2015-09-09 | 北京工业大学 | Depth map based hand feature point detection method |
| CN105261038A (en) * | 2015-09-30 | 2016-01-20 | 华南理工大学 | Bidirectional optical flow and perceptual hash based fingertip tracking method |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH10336439A (en) * | 1997-05-28 | 1998-12-18 | Minolta Co Ltd | Image reader |
| CN105806315B (en) * | 2014-12-31 | 2018-06-26 | 上海新跃仪表厂 | Noncooperative target relative measurement system and measuring method based on active coding information |
| CN105160323B (en) * | 2015-09-07 | 2018-11-27 | 哈尔滨市一舍科技有限公司 | A kind of gesture identification method |
-
2016
- 2016-08-16 CN CN201610675192.5A patent/CN106327486B/en active Active
- 2016-12-30 WO PCT/CN2016/113492 patent/WO2018032700A1/en active Application Filing
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103530892A (en) * | 2013-10-21 | 2014-01-22 | 清华大学深圳研究生院 | Kinect sensor based two-hand tracking method and device |
| KR20150068550A (en) * | 2013-12-11 | 2015-06-22 | 삼성전자주식회사 | Method and apparatus for acquiring information of a front object. |
| CN104598915A (en) * | 2014-01-24 | 2015-05-06 | 深圳奥比中光科技有限公司 | Gesture recognition method and gesture recognition device |
| CN103886303A (en) * | 2014-03-28 | 2014-06-25 | 上海云享科技有限公司 | Palmprint recognition method and device |
| CN104899600A (en) * | 2015-05-28 | 2015-09-09 | 北京工业大学 | Depth map based hand feature point detection method |
| CN105261038A (en) * | 2015-09-30 | 2016-01-20 | 华南理工大学 | Bidirectional optical flow and perceptual hash based fingertip tracking method |
Non-Patent Citations (1)
| Title |
|---|
| 骨龄自动评价系统中基于活动轮廓模型的手腕骨边缘提取;龚玉兰;<中国优秀硕士学位论文全文数据库（电子期刊）-信息科技辑>;20150831;正文第1-50页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2018032700A1 (en) | 2018-02-22 |
| CN106327486A (en) | 2017-01-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP1870038B1 (en) | Motion capture apparatus and method, and motion capture program | |
| CN113706699B (en) | Data processing method and device, electronic equipment and computer readable storage medium | |
| US10055851B2 (en) | Determining dimension of target object in an image using reference object | |
| CN106327486B (en) | Method and device for tracking finger web position | |
| JP6302132B2 (en) | Image processing apparatus, image processing system, image processing method, and program | |
| WO2010073432A1 (en) | Image processing device and image processing method | |
| US20130100140A1 (en) | Human body and facial animation systems with 3d camera and method thereof | |
| US20070098250A1 (en) | Man-machine interface based on 3-D positions of the human body | |
| CN106548519A (en) | Augmented reality method based on ORB SLAM and the sense of reality of depth camera | |
| CN106250867A (en) | A kind of skeleton based on depth data follows the tracks of the implementation method of system | |
| CN105930767A (en) | Human body skeleton-based action recognition method | |
| CN108351522A (en) | Direction of gaze maps | |
| JP4936491B2 (en) | Gaze direction estimation apparatus, gaze direction estimation method, and program for causing computer to execute gaze direction estimation method | |
| JP2008102902A (en) | Gaze direction estimation apparatus, gaze direction estimation method, and program for causing computer to execute gaze direction estimation method | |
| US11676357B2 (en) | Modification of projected structured light based on identified points within captured image | |
| CN110443205A (en) | A kind of hand images dividing method and device | |
| CN111696140A (en) | Monocular-based three-dimensional gesture tracking method | |
| CN114882121A (en) | Human-computer interaction virtual-real fusion method and device based on mixed reality head display | |
| JP5468773B2 (en) | Image processing apparatus and image processing method | |
| CN106340040B (en) | Method and device for tracking finger contour | |
| JP6810442B2 (en) | A camera assembly, a finger shape detection system using the camera assembly, a finger shape detection method using the camera assembly, a program for implementing the detection method, and a storage medium for the program. | |
| CN106355591B (en) | Method and device for tracking finger contour | |
| JPWO2020026677A1 (en) | Detection device, processing device, detection method, and processing program | |
| CN115205744A (en) | Intelligent exercise assisting method and device for figure skating | |
| CN106340039B (en) | Method and device for tracking finger contour |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |