CN112822470A - Projection interaction system and method based on human body image tracking - Google Patents
Projection interaction system and method based on human body image tracking Download PDFInfo
- Publication number
- CN112822470A CN112822470A CN202011639843.8A CN202011639843A CN112822470A CN 112822470 A CN112822470 A CN 112822470A CN 202011639843 A CN202011639843 A CN 202011639843A CN 112822470 A CN112822470 A CN 112822470A
- Authority
- CN
- China
- Prior art keywords
- image
- characteristic curve
- human body
- target human
- image characteristic
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims description 99
- 230000003993 interaction Effects 0.000 title claims description 10
- 238000012544 monitoring process Methods 0.000 claims description 93
- 230000008569 process Effects 0.000 claims description 35
- 238000012937 correction Methods 0.000 claims description 33
- 230000000007 visual effect Effects 0.000 claims description 19
- 230000002194 synthesizing effect Effects 0.000 claims description 17
- 230000002452 interceptive effect Effects 0.000 claims description 10
- 230000015572 biosynthetic process Effects 0.000 claims description 9
- 238000003786 synthesis reaction Methods 0.000 claims description 9
- 230000002708 enhancing effect Effects 0.000 claims description 6
- 230000009467 reduction Effects 0.000 claims description 6
- 238000003384 imaging method Methods 0.000 claims description 3
- 230000000694 effects Effects 0.000 description 7
- 238000005259 measurement Methods 0.000 description 5
- 238000013461 design Methods 0.000 description 3
- 238000001514 detection method Methods 0.000 description 3
- 238000001914 filtration Methods 0.000 description 3
- 239000002245 particle Substances 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000010354 integration Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 108010001267 Protein Subunits Proteins 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000007477 logistic regression Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000001681 protective effect Effects 0.000 description 1
- 238000012706 support-vector machine Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N9/00—Details of colour television systems
- H04N9/12—Picture reproducers
- H04N9/31—Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N9/00—Details of colour television systems
- H04N9/12—Picture reproducers
- H04N9/31—Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM]
- H04N9/3141—Constructional details thereof
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N9/00—Details of colour television systems
- H04N9/12—Picture reproducers
- H04N9/31—Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM]
- H04N9/3179—Video signal processing therefor
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Image Processing (AREA)
Abstract
The invention belongs to the technical field of image processing, and particularly relates to a projection interaction system and method based on human body image tracking. The system comprises: the real-time image acquisition device is configured for acquiring a target human body image, tracking the position of the target human body in real time, monitoring whether the target human body moves or not and whether the target human body moves out of an area, and if the target human body moves out of the area to be monitored, adjusting the position of the target human body in real time and continuously acquiring the target human body image; and the image denoising device is configured for denoising the image of the target human body acquired in real time to obtain a denoised image. The method and the device have the advantages that the target human body image is tracked in real time, the target human body image can move along with the target human body image after moving, then the image denoising is carried out on the target human body image, the image synthesis is carried out, then the projection is carried out, the projection on the moving target is realized, and the method and the device have the advantages of good projection effect, high intelligent degree and high projected image quality.
Description
Technical Field
The invention belongs to the technical field of image processing, and particularly relates to a projection interaction system and method based on human body image tracking.
Background
The "image tracking" technology refers to positioning an object shot in a camera by some means (such as image recognition, infrared, ultrasonic, etc.), and the "image tracking" technology in a narrow sense is commonly referred to in our daily life, and tracking and shooting are performed by the "image recognition" means.
The projection system is an optical system which illuminates an object and then images the object on a projection screen, is based on a motion tracking technology, and is suitable for any projector, liquid crystal display screen, LED large screen, plasma, digital video wall and the like. The projection system converts the interactive participant's actions into graphical image interactive feedback. The system has 24 sets of practical interactive effects and customizable high-resolution contents, and can realize an unequally projected area in the same industry so as to meet the interactive requirements of different users. In short, interactive projection systems allow users to experience unprecedented smooth interactive experiences by creating immersive interactive experiences.
Patent No. CN201310249924.0A discloses a projector, a control method thereof, and an image display system. Provided is a technique capable of easily correcting the deviation between a plurality of combined projection images. When the 1 st projector displays the measurement pattern on the projection screen, the 2 nd projector displays the measurement pattern on the projection screen. The 2 nd projector acquires the photographed image of the projected image of the 2 measurement patterns by the photographing unit, and detects the coordinates of the measurement points shown in the measurement patterns from the photographed image. The 2 nd projector corrects the image to be projected so that the projection image of the 1 st projector and the projection image of the 1 st projector are in a desired relationship based on the coordinates.
The image quality of a formed projection image is improved by acquiring a plurality of projection images in a stacked projection, and the plurality of projection images are superimposed on each other with high accuracy.
Disclosure of Invention
In view of the above, the main object of the present invention is to provide a projection interaction system and method based on human body image tracking, which utilize real-time tracking of a target human body image, move along with the target human body image after moving, then perform image denoising on the target human body image, perform image synthesis, and then perform projection, thereby implementing projection on a moving target, and having the advantages of good projection effect, high intelligent degree, and high quality of projected images.
In order to achieve the purpose, the technical scheme of the invention is realized as follows:
projection interactive system based on human image tracking, the system includes: the real-time image acquisition device is configured for acquiring a target human body image, tracking the position of the target human body in real time, monitoring whether the target human body moves or not and whether the target human body moves out of an area, and if the target human body moves out of the area to be monitored, adjusting the position of the real-time image acquisition device in real time to track the target human body and continuously acquire the target human body image; the image denoising device is configured for denoising an image of a target human body image acquired in real time, reducing noise interference in the process of acquiring the target human body image and acquiring a denoised image; and the image synthesis device is configured to synthesize a stereo image based on every 3 frames of the denoised images, then perform image superposition correction on every 8 stereo images to obtain a synthesized stereo image, and project the synthesized stereo image.
Further, the real-time image acquisition apparatus includes: a target monitoring unit for monitoring a tracked target human body within a monitoring area of a target human body image represented by image data obtained by continuously imaging the target human body; a monitoring area updating unit, responsive to the object monitoring unit monitoring an object, for updating the monitoring area in such a manner that the monitored object will occupy a center of the monitoring area; a movement monitoring unit, responsive to the object monitoring unit no longer monitoring the object, for monitoring the magnitude and direction of movement of the entire object human image in which the object is no longer monitored; a monitoring region setting unit that sets a monitoring region at a central portion of the target human body image if the movement of the entire target human body image monitored by the movement monitoring unit is equal to or greater than a designated magnitude, and sets a monitoring region at an edge of the target human body image in a direction opposite to a movement direction of the entire target human body image monitored by the movement monitoring unit if the movement of the entire target human body image monitored by the movement monitoring unit is less than the designated magnitude; and a control unit for controlling the target monitoring unit, the monitoring area updating unit, the movement monitoring unit, and the monitoring area setting unit to repeat a process for monitoring a target, a process for updating a monitoring area, a process for monitoring the magnitude and direction of movement, and a process for setting a monitoring area.
Further, the image synthesizing apparatus, based on the denoised image of every 3 frames, synthesizes a stereo image by the method including: respectively taking 3 frames of images as a first image of a first visual angle, a second image of a second visual angle and a third image of a third visual angle; acquiring disparity maps corresponding to a first image, a second image and a third image, and judging a zero disparity area according to the disparity maps, wherein the disparity maps comprise a plurality of disparity values corresponding to the first image, the second image and the third image; adjusting the disparity map to be a shifted disparity map corresponding to the zero disparity region; linearly or nonlinearly adjusting the shifted disparity map according to the shifted disparity map and a preset maximum range to obtain an adjusted disparity map; generating a plurality of virtual perspective images according to the adjusted disparity map, the first image, the second image and the third image; and enhancing a two-dimensional depth cue of the virtual perspective image according to the adjusted disparity map.
Further, the method in which the image synthesizing apparatus performs image superimposition correction on every 8 stereoscopic images to obtain a synthesized stereoscopic image includes: after the pixel values of the pixels corresponding to each pixel point in the 8 stereo images are superposed, an average value is obtained, and then the average value is used as the final value of the pixel.
Further, the image denoising device performs image denoising on the target human body image acquired in real time, so as to reduce noise interference in the process of acquiring the target human body image, and the method for acquiring the denoised image includes: the target human body image is expressed by the following formula: pPBsin (wt + kx + 1); the image characteristic curve density is as follows:
wherein, B is the amplitude of the image characteristic curve; w is the phase of the image characteristic curve; t is an image characteristic curve time parameter; k is a correction coefficient and is any positive integer; x is the correction amplitude and is any positive decimal number; rho is an energy density constant; c is an image characteristic curve propagation velocity constant; the image characteristic curve adjusting unit divides the received image characteristic curve into three parts according to the received image characteristic curve, wherein the three parts are respectively as follows: noise image characteristic curve portion, image characteristic curve portion and error mapLike a characteristic curve portion; according to the noise image characteristic curve generated by the noise image characteristic curve part and the error image characteristic curve generated by the error image characteristic curve part, adjusting the output image characteristic curve to ensure that the output of the output image characteristic curve is as follows: pSβ Bsin (wt-kx + α); wherein, beta is an amplitude enhancement coefficient of the output image characteristic curve and is any positive number less than 1; alpha is a phase correction value, and the setting range is 0-180; the image characteristic curve energy density of the output image characteristic curve is as follows:
the image characteristic curve level difference is calculated as follows:
and finally, the image characteristic curve output unit adjusts parameters of the output image characteristic curve according to the image characteristic curve level difference, so that beta is 1 and alpha is pi, a final output image characteristic curve is obtained, the final output image characteristic curve is filtered, the final image characteristic curve is output, and the noise reduction of the image characteristic curve is completed.
A projection interaction method based on human body image tracking, the method executes the following steps: step 1: acquiring a target human body image, simultaneously tracking the position of the target human body in real time, monitoring whether the target human body moves or not, and whether the target human body moves out of a region, and if the target human body moves out of the region to be monitored, adjusting the position of the target human body in real time to track the target human body, and continuously acquiring the target human body image; step 2: carrying out image denoising on a target human body image acquired in real time, reducing noise interference in the process of acquiring the target human body image, and acquiring a denoised image; and step 3: and synthesizing a stereo image based on every 3 frames of the denoised images, then performing image overlapping correction on every 8 stereo images to obtain a synthesized stereo image, and projecting the synthesized stereo image.
Further, when the synthesized stereo image is projected in step 2, the synthesized stereo image is projected at a time interval of one image per second.
Further, the image synthesizing apparatus, based on the denoised image of every 3 frames, synthesizes a stereo image by the method including: respectively taking 3 frames of images as a first image of a first visual angle, a second image of a second visual angle and a third image of a third visual angle; acquiring disparity maps corresponding to a first image, a second image and a third image, and judging a zero disparity area according to the disparity maps, wherein the disparity maps comprise a plurality of disparity values corresponding to the first image, the second image and the third image; adjusting the disparity map to be a shifted disparity map corresponding to the zero disparity region; linearly or nonlinearly adjusting the shifted disparity map according to the shifted disparity map and a preset maximum range to obtain an adjusted disparity map; generating a plurality of virtual perspective images according to the adjusted disparity map, the first image, the second image and the third image; and enhancing a two-dimensional depth cue of the virtual perspective image according to the adjusted disparity map.
Further, the method in which the image synthesizing apparatus performs image superimposition correction on every 8 stereoscopic images to obtain a synthesized stereoscopic image includes: after the pixel values of the pixels corresponding to each pixel point in the 8 stereo images are superposed, an average value is obtained, and then the average value is used as the final value of the pixel.
Further, the image denoising device performs image denoising on the target human body image acquired in real time, so as to reduce noise interference in the process of acquiring the target human body image, and the method for acquiring the denoised image includes: the target human body image is expressed by the following formula: pPBsin (wt + kx + 1); the image characteristic curve density is as follows:
wherein, B is the amplitude of the image characteristic curve; w is the phase of the image characteristic curve; t is an image characteristic curve time parameter; k is a correction coefficient and is any positive integer; x is the correction amplitude and is any positive decimal number; rho is an energy density constant; c is an image characteristic curve propagation velocity constant; the image characteristic curve adjusting unit divides the received image characteristic curve into three parts according to the received image characteristic curve, wherein the three parts are respectively as follows: noise image characteristic curve portion, and error image characteristicA curved portion; according to the noise image characteristic curve generated by the noise image characteristic curve part and the error image characteristic curve generated by the error image characteristic curve part, adjusting the output image characteristic curve to ensure that the output of the output image characteristic curve is as follows: pSβ Bsin (wt-kx + α); wherein, beta is an amplitude enhancement coefficient of the output image characteristic curve and is any positive number less than 1; alpha is a phase correction value, and the setting range is 0-180; the image characteristic curve energy density of the output image characteristic curve is as follows:
the image characteristic curve level difference is calculated as follows:
and finally, the image characteristic curve output unit adjusts parameters of the output image characteristic curve according to the image characteristic curve level difference, so that beta is 1 and alpha is pi, a final output image characteristic curve is obtained, the final output image characteristic curve is filtered, the final image characteristic curve is output, and the noise reduction of the image characteristic curve is completed.
The projection interaction system and method based on human body image tracking have the following beneficial effects: the method and the device have the advantages that the target human body image is tracked in real time, the target human body image can move along with the target human body image after moving, then the image denoising is carried out on the target human body image, the image synthesis is carried out, then the projection is carried out, the projection on the moving target is realized, and the method and the device have the advantages of good projection effect, high intelligent degree and high projected image quality. The method is mainly realized by the following steps:
1. image tracking: the method can track the target human body, and is realized by setting a detection area, if the monitored movement of the whole target human body image is equal to or larger than a specified amplitude, the central part of the target human body image is provided with the monitoring area, and if the monitored movement of the whole target human body image is smaller than the specified amplitude, the edge of the target human body image in the direction opposite to the movement direction of the whole target human body image monitored by the movement monitoring unit is provided with the monitoring area, and the method monitors the movement of the target human body to realize image tracking;
2. denoising an image: the method comprises the steps of carrying out image denoising on a target human body image acquired in real time, reducing noise interference in the process of acquiring the target human body image, and acquiring a denoised image, so that the subsequent image quality is higher, and the projected image effect is better;
3. image synthesis: and synthesizing a stereo image based on every 3 frames of denoised images, then performing image overlapping correction on every 8 stereo images to obtain a synthesized stereo image, and projecting the synthesized stereo image, so that an image is constructed every 24 frames of images, image deviation caused by moving the removed image or position factors of an image acquisition device is corrected, and the final image quality is higher.
Drawings
Fig. 1 is a schematic flowchart of a method of a projection interaction method based on human body image tracking according to an embodiment of the present invention;
fig. 2 is a schematic system structure diagram of a projection interaction system based on human body image tracking according to an embodiment of the present invention.
Detailed Description
The method of the present invention will be described in further detail below with reference to the accompanying drawings and embodiments of the invention.
Example 1
As shown in fig. 1, a projection interactive system based on human body image tracking, the system comprises: the real-time image acquisition device is configured for acquiring a target human body image, tracking the position of the target human body in real time, monitoring whether the target human body moves or not and whether the target human body moves out of an area, and if the target human body moves out of the area to be monitored, adjusting the position of the real-time image acquisition device in real time to track the target human body and continuously acquire the target human body image; the image denoising device is configured for denoising an image of a target human body image acquired in real time, reducing noise interference in the process of acquiring the target human body image and acquiring a denoised image; and the image synthesis device is configured to synthesize a stereo image based on every 3 frames of the denoised images, then perform image superposition correction on every 8 stereo images to obtain a synthesized stereo image, and project the synthesized stereo image.
By adopting the technical scheme, the target human body image can be tracked in real time and can move along with the target human body image after moving, then the image denoising is carried out on the target human body image, the image synthesis is carried out, and then the projection is carried out, so that the projection of the moving target is realized, and the method has the advantages of good projection effect, high intelligent degree and high projection image quality. The method is mainly realized by the following steps:
1. image tracking: the method can track the target human body, and is realized by setting a detection area, if the monitored movement of the whole target human body image is equal to or larger than a specified amplitude, the central part of the target human body image is provided with the monitoring area, and if the monitored movement of the whole target human body image is smaller than the specified amplitude, the edge of the target human body image in the direction opposite to the movement direction of the whole target human body image monitored by the movement monitoring unit is provided with the monitoring area, and the method monitors the movement of the target human body to realize image tracking;
2. denoising an image: the method comprises the steps of carrying out image denoising on a target human body image acquired in real time, reducing noise interference in the process of acquiring the target human body image, and acquiring a denoised image, so that the subsequent image quality is higher, and the projected image effect is better;
3. image synthesis: synthesizing a stereo image based on every 3 frames of denoised images, then performing image overlapping correction on every 8 stereo images to obtain a synthesized stereo image, and projecting the synthesized stereo image, so that an image is constructed every 24 frames of images, image deviation caused by moving the removed image or position of an image acquisition device is corrected, and the final image has higher quality
Example 2
On the basis of the above embodiment, the real-time image acquisition apparatus includes: a target monitoring unit for monitoring a tracked target human body within a monitoring area of a target human body image represented by image data obtained by continuously imaging the target human body; a monitoring area updating unit, responsive to the object monitoring unit monitoring an object, for updating the monitoring area in such a manner that the monitored object will occupy a center of the monitoring area; a movement monitoring unit, responsive to the object monitoring unit no longer monitoring the object, for monitoring the magnitude and direction of movement of the entire object human image in which the object is no longer monitored; a monitoring region setting unit that sets a monitoring region at a central portion of the target human body image if the movement of the entire target human body image monitored by the movement monitoring unit is equal to or greater than a designated magnitude, and sets a monitoring region at an edge of the target human body image in a direction opposite to a movement direction of the entire target human body image monitored by the movement monitoring unit if the movement of the entire target human body image monitored by the movement monitoring unit is less than the designated magnitude; and a control unit for controlling the target monitoring unit, the monitoring area updating unit, the movement monitoring unit, and the monitoring area setting unit to repeat a process for monitoring a target, a process for updating a monitoring area, a process for monitoring the magnitude and direction of movement, and a process for setting a monitoring area.
Specifically, the task of tracking a visual target (single target) is to predict the size and position of the target in a subsequent frame given the size and position of the target in an initial frame of a video sequence.
Inputting an initialization target frame, generating a plurality of candidate frames (Motion Model) in the next frame, extracting features (Feature Extractor) of the candidate frames, then scoring the candidate frames (observer Model), and finally finding a candidate frame with the highest score in the scores as a predicted target (Prediction A) or fusing a plurality of predicted values (Ensemble) to obtain a better predicted target.
Motion Model (Motion Model): the speed and quality of generating candidate samples directly determine how good the tracking system performs. Two methods are commonly used: particle filtering (Particle Filter) and Sliding Window (Sliding Window). Particle filtering is a sequential Bayes inference method, which infers the hidden state of the target in a recursive manner. While a sliding window is an exhaustive search method that lists all possible samples near the target as candidate samples.
Feature extraction (Feature Extractor) a distinctive Feature representation is one of the keys to target tracking. Common features are divided into two types: hand-designed features (handled-feature) and depth features (Deep feature). Common features of manual design include Gray scale features (Gray), Histogram of Oriented Gradient (HOG), Haar-like features, Scale Invariant Features (SIFT), etc. Unlike the artificially designed features, the depth features are features that are learned through a large number of training samples and are more discriminative than the manually designed features. Therefore, tracking methods using depth features are generally easy to achieve with a good result.
Observation Model (observer Model) most tracking methods focus mainly on the design of this block. According to different ideas, observation models can be divided into two categories: generative Model (Generative Model) and discriminant Model (discriminant Model) Generative models generally find the candidate that is most similar to the target template as the tracking result, and this process can be regarded as template matching. Common theoretical methods include: subspace, sparse representation, dictionary learning, etc. And the discriminant model distinguishes the target from the background by training a classifier, and selects the candidate sample with the highest confidence coefficient as a prediction result. Discriminant methods have become the dominant method in target tracking because there are a large number of machine learning methods available. Common theoretical methods include: logistic regression, ridge regression, support vector machines, multi-instance learning, correlation filtering, and the like.
Model Update (Model Update) the Model Update mainly updates the observation Model to adapt to the change of the target appearance and prevent the tracking process from drifting. Model updates do not have a uniform standard, and the appearance of the target is generally considered to change continuously, so the model is often updated every frame. However, it is also considered that the past appearance of the target is important for tracking, and continuous updating may lose past appearance information and introduce excessive noise, so that the problem is solved by combining long and short-term updating.
An integration Method (Ensemble Method) is beneficial to improving the prediction accuracy of a model and is often regarded as an effective means for improving the tracking accuracy. The integration methods can be broadly divided into two categories: the best of the multiple predictors is selected, or a weighted average of all of the predictors is used.
Example 3
On the basis of the above embodiment, the image synthesizing apparatus, based on every 3 frames of the denoised image, synthesizes a stereo image, including: respectively taking 3 frames of images as a first image of a first visual angle, a second image of a second visual angle and a third image of a third visual angle; acquiring disparity maps corresponding to a first image, a second image and a third image, and judging a zero disparity area according to the disparity maps, wherein the disparity maps comprise a plurality of disparity values corresponding to the first image, the second image and the third image; adjusting the disparity map to be a shifted disparity map corresponding to the zero disparity region; linearly or nonlinearly adjusting the shifted disparity map according to the shifted disparity map and a preset maximum range to obtain an adjusted disparity map; generating a plurality of virtual perspective images according to the adjusted disparity map, the first image, the second image and the third image; and enhancing a two-dimensional depth cue of the virtual perspective image according to the adjusted disparity map.
Specifically, the disparity map is an image in which any one of the pair of images is used as a reference, the size of the disparity map is the size of the reference image, and the element value is the disparity value.
Since the disparity map includes distance information of a scene, image matching of the disparity map is extracted from the stereo image pair.
Example 4
On the basis of the above embodiment, the method in which the image synthesizing apparatus performs image superimposition correction on every 8 stereoscopic images to obtain a synthesized stereoscopic image includes: after the pixel values of the pixels corresponding to each pixel point in the 8 stereo images are superposed, an average value is obtained, and then the average value is used as the final value of the pixel.
Example 5
On the basis of the previous embodiment, the image denoising device performs image denoising on a target human body image acquired in real time, so as to reduce noise interference in the process of acquiring the target human body image, and the method for acquiring the denoised image comprises the following steps: the target human body image is expressed by the following formula: pPBsin (wt + kx + 1); the image characteristic curve density is as follows:
wherein, B is the amplitude of the image characteristic curve; w is the phase of the image characteristic curve; t is an image characteristic curve time parameter; k is a correction coefficient and is any positive integer; x is the correction amplitude and is any positive decimal number; rho is an energy density constant; c is an image characteristic curve propagation velocity constant; the image characteristic curve adjusting unit divides the received image characteristic curve into three parts according to the received image characteristic curve, wherein the three parts are respectively as follows: a noise image characteristic curve portion, an image characteristic curve portion, and an error image characteristic curve portion; according to the noise image characteristic curve generated by the noise image characteristic curve part and the error image characteristic curve generated by the error image characteristic curve part, adjusting the output image characteristic curve to ensure that the output of the output image characteristic curve is as follows: pSβ Bsin (wt-kx + α); wherein, beta is an amplitude enhancement coefficient of the output image characteristic curve and is any positive number less than 1; alpha is a phase correction value, and the setting range is 0-180; the image characteristic curve energy density of the output image characteristic curve is as follows:
the image characteristic curve level difference is calculated as follows:
finally, the image characteristic curve output unit adjusts the parameters of the output image characteristic curve according to the image characteristic curve level difference, so that beta is 1 and alpha is pi, a final output image characteristic curve is obtained, the final output image characteristic curve is filtered,and outputting the final image characteristic curve to finish the noise reduction of the image characteristic curve.
Example 6
A projection interaction method based on human body image tracking, the method executes the following steps: step 1: acquiring a target human body image, simultaneously tracking the position of the target human body in real time, monitoring whether the target human body moves or not, and whether the target human body moves out of a region, and if the target human body moves out of the region to be monitored, adjusting the position of the target human body in real time to track the target human body, and continuously acquiring the target human body image; step 2: carrying out image denoising on a target human body image acquired in real time, reducing noise interference in the process of acquiring the target human body image, and acquiring a denoised image; and step 3: and synthesizing a stereo image based on every 3 frames of the denoised images, then performing image overlapping correction on every 8 stereo images to obtain a synthesized stereo image, and projecting the synthesized stereo image.
Specifically, the target human body image is tracked in real time, the target human body image can move along with the target human body image after moving, then the target human body image is subjected to image denoising, image synthesis and projection, the moving target is projected, and the method has the advantages of good projection effect, high intelligent degree and high projection image quality.
Example 7
On the basis of the above embodiment, when the synthesized stereo image is projected in step 2, the projection is performed at a time interval of one image per second.
Specifically, the method can track the target human body by setting a detection area, if the monitored movement of the whole target human body image is equal to or larger than a specified amplitude, the monitoring area is arranged at the central part of the target human body image, and if the monitored movement of the whole target human body image is smaller than the specified amplitude, the monitoring area is arranged at the edge of the target human body image in the direction opposite to the movement direction of the whole target human body image monitored by the movement monitoring unit, and the method monitors the movement of the target human body to realize image tracking.
Example 8
On the basis of the above embodiment, the image synthesizing apparatus, based on every 3 frames of the denoised image, synthesizes a stereo image, including: respectively taking 3 frames of images as a first image of a first visual angle, a second image of a second visual angle and a third image of a third visual angle; acquiring disparity maps corresponding to a first image, a second image and a third image, and judging a zero disparity area according to the disparity maps, wherein the disparity maps comprise a plurality of disparity values corresponding to the first image, the second image and the third image; adjusting the disparity map to be a shifted disparity map corresponding to the zero disparity region; linearly or nonlinearly adjusting the shifted disparity map according to the shifted disparity map and a preset maximum range to obtain an adjusted disparity map; generating a plurality of virtual perspective images according to the adjusted disparity map, the first image, the second image and the third image; and enhancing a two-dimensional depth cue of the virtual perspective image according to the adjusted disparity map.
Specifically, the method and the device perform image denoising on the target human body image acquired in real time, reduce noise interference in the process of acquiring the target human body image, and acquire a denoised image, so that the subsequent image quality is higher, and the projected image effect is better.
Example 9
On the basis of the above embodiment, the method in which the image synthesizing apparatus performs image superimposition correction on every 8 stereoscopic images to obtain a synthesized stereoscopic image includes: after the pixel values of the pixels corresponding to each pixel point in the 8 stereo images are superposed, an average value is obtained, and then the average value is used as the final value of the pixel.
Specifically, a stereo image is synthesized based on every 3 frames of denoised images, then every 8 stereo images are subjected to image overlapping correction to obtain a synthesized stereo image, and the synthesized stereo image is projected, so that an image is constructed every 24 frames of images, image deviation caused by movement of the removed image or position factors of an image acquisition device is corrected, and the final image quality is higher.
Example 10
On the basis of the previous embodiment, the image denoising device performs image denoising on a target human body image acquired in real time, so as to reduce noise interference in the process of acquiring the target human body image, and the method for acquiring the denoised image comprises the following steps: the target human body image is expressed by the following formula: pPBsin (wt + kx + 1); the image characteristic curve density is as follows:
wherein, B is the amplitude of the image characteristic curve; w is the phase of the image characteristic curve; t is an image characteristic curve time parameter; k is a correction coefficient and is any positive integer; x is the correction amplitude and is any positive decimal number; rho is an energy density constant; c is an image characteristic curve propagation velocity constant; the image characteristic curve adjusting unit divides the received image characteristic curve into three parts according to the received image characteristic curve, wherein the three parts are respectively as follows: a noise image characteristic curve portion, an image characteristic curve portion, and an error image characteristic curve portion; according to the noise image characteristic curve generated by the noise image characteristic curve part and the error image characteristic curve generated by the error image characteristic curve part, adjusting the output image characteristic curve to ensure that the output of the output image characteristic curve is as follows: pSβ Bsin (wt-kx + α); wherein, beta is an amplitude enhancement coefficient of the output image characteristic curve and is any positive number less than 1; alpha is a phase correction value, and the setting range is 0-180; the image characteristic curve energy density of the output image characteristic curve is as follows:
the image characteristic curve level difference is calculated as follows:
and finally, the image characteristic curve output unit adjusts parameters of the output image characteristic curve according to the image characteristic curve level difference, so that beta is 1 and alpha is pi, a final output image characteristic curve is obtained, the final output image characteristic curve is filtered, the final image characteristic curve is output, and the noise reduction of the image characteristic curve is completed.
It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working process and related description of the system described above may refer to the corresponding process in the foregoing method embodiments, and will not be described herein again.
It should be noted that, the system provided in the foregoing embodiment is only illustrated by dividing the functional units, and in practical applications, the functions may be distributed by different functional units according to needs, that is, the units or steps in the embodiments of the present invention are further decomposed or combined, for example, the units in the foregoing embodiment may be combined into one unit, or may be further decomposed into multiple sub-units, so as to complete all or the functions of the units described above. The names of the units and steps involved in the embodiments of the present invention are only for distinguishing the units or steps, and are not to be construed as unduly limiting the present invention.
It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes and related descriptions of the storage device and the processing device described above may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
Those of skill in the art would appreciate that the various illustrative elements, method steps, described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that programs corresponding to the elements, method steps may be located in Random Access Memory (RAM), memory, Read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. To clearly illustrate this interchangeability of electronic hardware and software, various illustrative components and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as electronic hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
The terms "first," "second," and the like are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order.
The terms "comprises," "comprising," or any other similar term are intended to cover a non-exclusive inclusion, such that a process, method, article, or unit/apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or unit/apparatus.
So far, the technical solutions of the present invention have been described in connection with the preferred embodiments shown in the drawings, but it is easily understood by those skilled in the art that the scope of the present invention is obviously not limited to these specific embodiments. Equivalent modifications or substitutions of the related art marks may be made by those skilled in the art without departing from the principle of the present invention, and the technical solutions after such modifications or substitutions will fall within the protective scope of the present invention.
The above description is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention.
Claims (10)
1. Projection interactive system based on human image tracking, characterized in that, the system includes: the real-time image acquisition device is configured for acquiring a target human body image, tracking the position of the target human body in real time, monitoring whether the target human body moves or not and whether the target human body moves out of an area, and if the target human body moves out of the area to be monitored, adjusting the position of the real-time image acquisition device in real time to track the target human body and continuously acquire the target human body image; the image denoising device is configured for denoising an image of a target human body image acquired in real time, reducing noise interference in the process of acquiring the target human body image and acquiring a denoised image; and the image synthesis device is configured to synthesize a stereo image based on every 3 frames of the denoised images, then perform image superposition correction on every 8 stereo images to obtain a synthesized stereo image, and project the synthesized stereo image.
2. The system of claim 1, wherein the real-time image acquisition device comprises: a target monitoring unit for monitoring a tracked target human body within a monitoring area of a target human body image represented by image data obtained by continuously imaging the target human body; a monitoring area updating unit, responsive to the object monitoring unit monitoring an object, for updating the monitoring area in such a manner that the monitored object will occupy a center of the monitoring area; a movement monitoring unit, responsive to the object monitoring unit no longer monitoring the object, for monitoring the magnitude and direction of movement of the entire object human image in which the object is no longer monitored; a monitoring region setting unit that sets a monitoring region at a central portion of the target human body image if the movement of the entire target human body image monitored by the movement monitoring unit is equal to or greater than a designated magnitude, and sets a monitoring region at an edge of the target human body image in a direction opposite to a movement direction of the entire target human body image monitored by the movement monitoring unit if the movement of the entire target human body image monitored by the movement monitoring unit is less than the designated magnitude; and a control unit for controlling the target monitoring unit, the monitoring area updating unit, the movement monitoring unit, and the monitoring area setting unit to repeat a process for monitoring a target, a process for updating a monitoring area, a process for monitoring the magnitude and direction of movement, and a process for setting a monitoring area.
3. The system of claim 2, wherein the image synthesizing means synthesizes a stereoscopic image based on the denoised image for every 3 frames, comprising: respectively taking 3 frames of images as a first image of a first visual angle, a second image of a second visual angle and a third image of a third visual angle; acquiring disparity maps corresponding to a first image, a second image and a third image, and judging a zero disparity area according to the disparity maps, wherein the disparity maps comprise a plurality of disparity values corresponding to the first image, the second image and the third image; adjusting the disparity map to be a shifted disparity map corresponding to the zero disparity region; linearly or nonlinearly adjusting the shifted disparity map according to the shifted disparity map and a preset maximum range to obtain an adjusted disparity map; generating a plurality of virtual perspective images according to the adjusted disparity map, the first image, the second image and the third image; and enhancing a two-dimensional depth cue of the virtual perspective image according to the adjusted disparity map.
4. The system according to claim 3, wherein the method for the image synthesizing device to perform image overlay correction on every 8 stereo images to obtain the synthesized stereo image comprises: after the pixel values of the pixels corresponding to each pixel point in the 8 stereo images are superposed, an average value is obtained, and then the average value is used as the final value of the pixel.
5. The system of claim 4, wherein the image denoising device performs image denoising on the target human body image acquired in real time, so as to reduce noise interference in the process of acquiring the target human body image, and the method for acquiring the denoised image comprises: the target human body image is expressed by the following formula: pPBsin (wt + kx + 1); the image characteristic curve density is as follows:
wherein, B is the amplitude of the image characteristic curve; w is the phase of the image characteristic curve; t is an image characteristic curve time parameter; k is a correction coefficient and is any positive integer; x is the correction amplitude and is any positive decimal number; rho is an energy density constant; c is an image characteristic curve propagation velocity constant; the image characteristic curve adjusting unit divides the received image characteristic curve into three parts according to the received image characteristic curve, wherein the three parts are respectively as follows: a noise image characteristic curve portion, an image characteristic curve portion, and an error image characteristic curve portion; adjusting the output image characteristic curve according to the noise image characteristic curve generated by the noise image characteristic curve part and the error image characteristic curve generated by the error image characteristic curve part, so that the output image characteristic curveThe output of the curve is: pSβ Bsin (wt-kx + α); wherein, beta is an amplitude enhancement coefficient of the output image characteristic curve and is any positive number less than 1; alpha is a phase correction value, and the setting range is 0-180; the image characteristic curve energy density of the output image characteristic curve is as follows:
the image characteristic curve level difference is calculated as follows:
and finally, the image characteristic curve output unit adjusts parameters of the output image characteristic curve according to the image characteristic curve level difference, so that beta is 1 and alpha is pi, a final output image characteristic curve is obtained, the final output image characteristic curve is filtered, the final image characteristic curve is output, and the noise reduction of the image characteristic curve is completed.
6. A projection interaction method based on human body image tracking based on the system of any one of claims 1 to 5, characterized in that the method performs the following steps: step 1: acquiring a target human body image, simultaneously tracking the position of the target human body in real time, monitoring whether the target human body moves or not, and whether the target human body moves out of a region, and if the target human body moves out of the region to be monitored, adjusting the position of the target human body in real time to track the target human body, and continuously acquiring the target human body image; step 2: carrying out image denoising on a target human body image acquired in real time, reducing noise interference in the process of acquiring the target human body image, and acquiring a denoised image; and step 3: and synthesizing a stereo image based on every 3 frames of the denoised images, then performing image overlapping correction on every 8 stereo images to obtain a synthesized stereo image, and projecting the synthesized stereo image.
7. The system of claim 1, wherein the step 2 of projecting the synthesized stereo image projects the synthesized stereo image at a time interval of one image per second.
8. The system of claim 1, wherein the image synthesizing means synthesizes a stereoscopic image based on the denoised image for every 3 frames, comprising: respectively taking 3 frames of images as a first image of a first visual angle, a second image of a second visual angle and a third image of a third visual angle; acquiring disparity maps corresponding to a first image, a second image and a third image, and judging a zero disparity area according to the disparity maps, wherein the disparity maps comprise a plurality of disparity values corresponding to the first image, the second image and the third image; adjusting the disparity map to be a shifted disparity map corresponding to the zero disparity region; linearly or nonlinearly adjusting the shifted disparity map according to the shifted disparity map and a preset maximum range to obtain an adjusted disparity map; generating a plurality of virtual perspective images according to the adjusted disparity map, the first image, the second image and the third image; and enhancing a two-dimensional depth cue of the virtual perspective image according to the adjusted disparity map.
9. The method according to claim 8, wherein the method for the image synthesizing device to perform image overlay correction on every 8 stereoscopic images to obtain the synthesized stereoscopic image comprises: after the pixel values of the pixels corresponding to each pixel point in the 8 stereo images are superposed, an average value is obtained, and then the average value is used as the final value of the pixel.
10. The system of claim 1, wherein the image denoising device performs image denoising on the target human body image acquired in real time, so as to reduce noise interference in the process of acquiring the target human body image, and the method for acquiring the denoised image comprises: the target human body image is expressed by the following formula: pPBsin (wt + kx + 1); the image characteristic curve density is as follows:
wherein, B is the amplitude of the image characteristic curve; w is the phase of the image characteristic curve; t is an image characteristic curve time parameter; k is a correction coefficient and is any positive integer; x is the correction amplitude and is any positive decimal number; rho is an energy density constant; c is an image characteristic curve propagation velocity constant; the image characteristic curve adjusting unit divides the received image characteristic curve into three parts according to the received image characteristic curve, wherein the three parts are respectively as follows: a noise image characteristic curve portion, an image characteristic curve portion, and an error image characteristic curve portion; according to the noise image characteristic curve generated by the noise image characteristic curve part and the error image characteristic curve generated by the error image characteristic curve part, adjusting the output image characteristic curve to ensure that the output of the output image characteristic curve is as follows: pSβ Bsin (wt-kx + α); wherein, beta is an amplitude enhancement coefficient of the output image characteristic curve and is any positive number less than 1; alpha is a phase correction value, and the setting range is 0-180; the image characteristic curve energy density of the output image characteristic curve is as follows:
the image characteristic curve level difference is calculated as follows:
and finally, the image characteristic curve output unit adjusts parameters of the output image characteristic curve according to the image characteristic curve level difference, so that beta is 1 and alpha is pi, a final output image characteristic curve is obtained, the final output image characteristic curve is filtered, the final image characteristic curve is output, and the noise reduction of the image characteristic curve is completed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011639843.8A CN112822470A (en) | 2020-12-31 | 2020-12-31 | Projection interaction system and method based on human body image tracking |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011639843.8A CN112822470A (en) | 2020-12-31 | 2020-12-31 | Projection interaction system and method based on human body image tracking |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112822470A true CN112822470A (en) | 2021-05-18 |
Family
ID=75858062
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011639843.8A Pending CN112822470A (en) | 2020-12-31 | 2020-12-31 | Projection interaction system and method based on human body image tracking |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112822470A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TWI845319B (en) * | 2023-05-25 | 2024-06-11 | 友達光電股份有限公司 | Graphics synthesis method |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101776952A (en) * | 2010-01-29 | 2010-07-14 | 联动天下科技（大连）有限公司 | Novel interactive projection system |
| CN101931753A (en) * | 2009-06-18 | 2010-12-29 | 富士胶片株式会社 | Target following and image tracking apparatus, method of controlling operation thereof and digital camera |
| CN105306919A (en) * | 2014-06-03 | 2016-02-03 | 宏碁股份有限公司 | Stereo image synthesis method and device |
| CN107105193A (en) * | 2016-02-23 | 2017-08-29 | 芋头科技（杭州）有限公司 | Robot monitoring system based on human body information |
| CN112152567A (en) * | 2020-08-21 | 2020-12-29 | 杭州伍暨通信设备有限公司 | High-efficiency high-harmonic suppression power amplification device and method |
-
2020
- 2020-12-31 CN CN202011639843.8A patent/CN112822470A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101931753A (en) * | 2009-06-18 | 2010-12-29 | 富士胶片株式会社 | Target following and image tracking apparatus, method of controlling operation thereof and digital camera |
| CN101776952A (en) * | 2010-01-29 | 2010-07-14 | 联动天下科技（大连）有限公司 | Novel interactive projection system |
| CN105306919A (en) * | 2014-06-03 | 2016-02-03 | 宏碁股份有限公司 | Stereo image synthesis method and device |
| CN107105193A (en) * | 2016-02-23 | 2017-08-29 | 芋头科技（杭州）有限公司 | Robot monitoring system based on human body information |
| CN112152567A (en) * | 2020-08-21 | 2020-12-29 | 杭州伍暨通信设备有限公司 | High-efficiency high-harmonic suppression power amplification device and method |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TWI845319B (en) * | 2023-05-25 | 2024-06-11 | 友達光電股份有限公司 | Graphics synthesis method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Fischer et al. | Rt-gene: Real-time eye gaze estimation in natural environments | |
| WO2021139484A1 (en) | Target tracking method and apparatus, electronic device, and storage medium | |
| KR100530812B1 (en) | Wavelet-based facial motion capture for avatar animation | |
| EP2192549B1 (en) | Target tracking device and target tracking method | |
| US20170045950A1 (en) | Gesture Recognition Systems | |
| JP5438601B2 (en) | Human motion determination device and program thereof | |
| US11159717B2 (en) | Systems and methods for real time screen display coordinate and shape detection | |
| JP2009070314A (en) | Image processor and image processing method, and program | |
| KR960035351A (en) | 3D display image generation method, image processing method using depth information, depth information generation method | |
| JP2015527625A (en) | physical measurement | |
| CN110941996A (en) | Target and track augmented reality method and system based on generation of countermeasure network | |
| US20190066311A1 (en) | Object tracking | |
| CN111898571A (en) | Action recognition system and method | |
| CN113688680A (en) | An intelligent identification and tracking system | |
| US11822620B2 (en) | Personalized local image features using bilevel optimization | |
| CN112822470A (en) | Projection interaction system and method based on human body image tracking | |
| Zimmer et al. | Imposing temporal consistency on deep monocular body shape and pose estimation | |
| JP2015153321A (en) | Image processor, image processing method and program | |
| CN113065506A (en) | Human body posture recognition method and system | |
| JP5051671B2 (en) | Information processing apparatus, information processing method, and program | |
| CN116386118A (en) | Drama matching cosmetic system and method based on human image recognition | |
| CN106934818B (en) | Hand motion tracking method and system | |
| Hamlaoui et al. | Facial action tracking using particle filters and active appearance models | |
| CN112132864A (en) | Robot following method based on vision and following robot | |
| Jian et al. | Realistic face animation generation from videos |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210518 |
|
| RJ01 | Rejection of invention patent application after publication |