CN117793442A - Image video masking method, device, equipment and medium based on point set - Google Patents
Image video masking method, device, equipment and medium based on point set Download PDFInfo
- Publication number
- CN117793442A CN117793442A CN202311856747.2A CN202311856747A CN117793442A CN 117793442 A CN117793442 A CN 117793442A CN 202311856747 A CN202311856747 A CN 202311856747A CN 117793442 A CN117793442 A CN 117793442A
- Authority
- CN
- China
- Prior art keywords
- coordinates
- target
- point
- polygon
- texture
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 55
- 230000000873 masking effect Effects 0.000 title claims abstract description 48
- 238000009499 grossing Methods 0.000 claims abstract description 48
- 238000012545 processing Methods 0.000 claims abstract description 37
- 239000011159 matrix material Substances 0.000 claims description 97
- 230000006870 function Effects 0.000 claims description 17
- 238000004364 calculation method Methods 0.000 claims description 14
- 238000006073 displacement reaction Methods 0.000 claims description 10
- 238000004590 computer program Methods 0.000 claims description 5
- 230000008569 process Effects 0.000 claims description 4
- 230000001131 transforming effect Effects 0.000 claims description 2
- 230000000694 effects Effects 0.000 abstract description 8
- 230000009286 beneficial effect Effects 0.000 abstract description 4
- 238000010586 diagram Methods 0.000 description 14
- 238000006243 chemical reaction Methods 0.000 description 6
- 230000009466 transformation Effects 0.000 description 4
- 230000004048 modification Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 239000013598 vector Substances 0.000 description 3
- 238000012790 confirmation Methods 0.000 description 2
- 230000005669 field effect Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 235000019800 disodium phosphate Nutrition 0.000 description 1
- 230000032669 eclosion Effects 0.000 description 1
- 230000008030 elimination Effects 0.000 description 1
- 238000003379 elimination reaction Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000013604 expression vector Substances 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
Landscapes
- Image Processing (AREA)
- Image Generation (AREA)
Abstract
The application relates to an image video masking method, device, equipment and medium based on a point set, wherein the method comprises the following steps: recording a point set of a mask drawn by a user on a U I control, and generating a polygon of the mask based on the point set; converting the point coordinates in the point set into a coordinate system to obtain texture coordinates; recognizing texture coordinates in the interior and the exterior of the polygon to obtain target point coordinates, wherein the target point coordinates comprise target interior point coordinates and target exterior point coordinates, performing display processing on the target interior point coordinates, and performing hiding processing on the target exterior point coordinates; and calculating the transparency of the mask based on the coordinates of the target point, smoothing the mask, and modifying the transparent channel and the color value of the original image to obtain the target image. The method and the device utilize the point set of the mask drawn by the user on the U I control to carry out image video masking, can adapt to the masking range of different scales, and are beneficial to improving the effect and efficiency of the image video masking.
Description
Technical Field
The present disclosure relates to the field of image processing technologies, and in particular, to a method, an apparatus, a device, and a medium for masking an image and video based on a point set.
Background
Masking and feathering are a common technique for clipping, masking, and smoothing an image, and are widely used in the fields of image editing, compositing, enhancement, and the like. They allow image processing professionals to precisely control the region selection and transitional effects of images to achieve more natural, realistic and artistic visual effects.
Currently, masking is accomplished by creating a mask (binary mask) map of the same size as the image pixels. This binary mask marks certain areas in the image as either "selected" (typically white) or "unselected" (typically black). The "selected" areas will then set the transparency of the image to opaque, and the "unselected" areas will then set the transparency of the image to transparent. However, when the mask is out of the original size, it is difficult to calculate by entering an appropriate mask (binary mask), resulting in poor image video masking, and it requires mask creation to be implemented, resulting in low video masking efficiency.
Disclosure of Invention
An embodiment of the application aims to provide an image video masking method, device, equipment and medium based on a point set so as to improve the effect and efficiency of the image video masking.
In order to solve the above technical problems, an embodiment of the present application provides an image video masking method based on a point set, including:
recording a point set of a mask drawn by a user on a UI control, and generating a polygon of the mask based on the point set;
converting the point coordinates in the point set into a coordinate system to obtain texture coordinates;
recognizing texture coordinates inside and outside the polygon to obtain target point coordinates, wherein the target point coordinates comprise target internal point coordinates and target external point coordinates, performing display processing on the target internal point coordinates, and performing hiding processing on the target external point coordinates;
and calculating the transparency of the shade based on the target point coordinates, smoothing the shade according to the transparency, and modifying the transparent channel and the color value of the original image according to the transparency to obtain a target image.
Further, the transforming the point coordinates in the point set to a coordinate system to obtain texture coordinates includes:
converting each point coordinate in the point set into a screen coordinate of OpenGL according to a preset mode, wherein the preset mode is to take an abscissa of the point coordinate as an abscissa of the screen coordinate, and take a value obtained by subtracting an ordinate of the point coordinate from the height of the original image as an ordinate of the screen coordinate;
and calculating a model matrix, a view matrix and a projection matrix in the OpenGL, and converting the screen coordinates into a coordinate system based on the model matrix, the view matrix and the projection matrix to obtain the texture coordinates.
Further, the calculating the model matrix, the view matrix, and the projection matrix in OpenGL, and converting the screen coordinates based on the model matrix, the view matrix, and the projection matrix to perform coordinate system conversion to render textures to the mask, to obtain texture coordinates, includes:
calculating the model matrix according to the model scaling value, the model rotation value and the model displacement in the OpenGL;
calculating the view matrix according to the view rotation value and the view displacement in the OpenGL;
taking a preset matrix as the projection matrix;
calculating a result matrix based on the model matrix, the view matrix and the projection matrix;
and converting the screen coordinates into a coordinate system according to the inverse matrix of the result matrix to obtain the texture coordinates.
Further, the identifying texture coordinates inside and outside the polygon to obtain target point coordinates, where the target point coordinates include target internal point coordinates and target external point coordinates, performing display processing on the target internal point coordinates, and performing hiding processing on the target external point coordinates, including:
judging whether a ray corresponding to the texture coordinate intersects the polygon or not by adopting a ray method to obtain a judging result;
calculating the number of intersections of each texture coordinate corresponding ray and the polygon based on the judging result to obtain the number of target intersections;
and if the number of the target intersections is an odd number, judging that the texture coordinates are inside the polygon, and taking the texture coordinates inside the polygon as the target internal point coordinates.
Further, the determining, by using a ray method, whether a ray corresponding to the texture coordinate intersects the polygon, to obtain a determination result includes:
taking each texture coordinate as a starting point to send out a ray to the right to obtain a target ray;
eliminating target rays which are not intersected with the preset polygon;
calculating the intersecting coordinates of the target ray and the polygon to obtain intersecting coordinates;
and if the abscissa of the intersection coordinate is larger than the abscissa of the texture coordinate, judging whether the ray corresponding to the texture coordinate intersects the polygon or not, and obtaining the judging result.
Further, the calculating the transparency of the mask based on the target point coordinates, smoothing the mask according to the transparency, and modifying the transparent channel and the color value of the original image according to the transparency to obtain a target image includes:
calculating the shortest distance from the coordinates of the target point to the polygon;
performing stage smoothing processing on the mask based on the shortest distance by adopting a smoothing stage function, and calculating the transparency;
and modifying the transparent channel of the original image based on the transparency, and multiplying the color value of each pixel point on the original image with the transparency to obtain the target image.
Further, the performing a stage smoothing process on the mask based on the shortest distance using a smoothing stage function, and calculating the transparency includes:
inputting any shortest distance and two preset boundary values into a smoothing stage function to carry out stage smoothing treatment, so as to obtain transparency after the stage smoothing treatment;
the calculation mode of the smoothing stage function is as follows:
y＝k*k*(3-2k)；
wherein t is 1 、t 2 And for the two preset boundary values, x is the shortest distance, k is a smoothing coefficient, and y is the transparency.
In order to solve the above technical problem, an embodiment of the present application provides an image video mask device based on a point set, including:
a point set recording unit, configured to record a point set of a mask drawn by a user on a UI control, and generate a polygon of the mask based on the point set;
the texture coordinate generation unit is used for converting the point coordinates in the point set into a coordinate system to obtain texture coordinates;
the target point coordinate confirming unit is used for identifying texture coordinates inside and outside the polygon to obtain target point coordinates, wherein the target point coordinates comprise target inner point coordinates and target outer point coordinates, the target inner point coordinates are subjected to display processing, and the target outer point coordinates are subjected to hiding processing;
and the target image generation unit is used for calculating the transparency of the shade based on the target point coordinates, smoothing the shade according to the transparency, and modifying the transparent channel and the color value of the original image according to the transparency to obtain a target image.
In order to solve the technical problems, the invention adopts a technical scheme that: a computer device is provided comprising one or more processors; a memory for storing one or more programs to cause the one or more processors to implement the point set based image video masking method of any of the above.
In order to solve the technical problems, the invention adopts a technical scheme that: a computer readable storage medium having stored thereon a computer program which when executed by a processor implements the point set based image video masking method of any of the above.
The embodiment of the invention provides an image video masking method, device, equipment and medium based on a point set. The method comprises the following steps: recording a point set of a mask drawn by a user on a UI control, and generating a polygon of the mask based on the point set; converting the point coordinates in the point set into a coordinate system to obtain texture coordinates; recognizing texture coordinates inside and outside the polygon to obtain target point coordinates, wherein the target point coordinates comprise target internal point coordinates and target external point coordinates, performing display processing on the target internal point coordinates, and performing hiding processing on the target external point coordinates; and calculating the transparency of the shade based on the target point coordinates, smoothing the shade according to the transparency, and modifying the transparent channel and the color value of the original image according to the transparency to obtain a target image. The embodiment of the invention utilizes the point set of the mask drawn by the user on the UI control to carry out the image video masking without constructing a mask, can adapt to the masking ranges of different scales, and is beneficial to improving the effect and efficiency of the image video masking.
Drawings
For a clearer description of the solution in the present application, a brief description will be given below of the drawings that are needed in the description of the embodiments of the present application, it being obvious that the drawings in the following description are some embodiments of the present application, and that other drawings may be obtained from these drawings without inventive effort for a person of ordinary skill in the art.
FIG. 1 is a flowchart of an implementation of a method for masking image video based on a point set according to an embodiment of the present application;
FIG. 2 is a flowchart of an implementation of a sub-process in a point set-based image video masking method provided in an embodiment of the present application;
FIG. 3 is a flowchart of an implementation of a sub-process in a point set-based image video masking method provided in an embodiment of the present application;
FIG. 4 is a schematic diagram of coordinate system conversion provided in an embodiment of the present application;
FIG. 5 is a flowchart of an implementation of a sub-process in a point set-based image video masking method provided in an embodiment of the present application;
FIG. 6 is a flowchart of an implementation of a sub-process in a point set based image video masking method provided in an embodiment of the present application;
FIG. 7 is a schematic diagram illustrating a situation where a target ray and a polygon are not intersected according to an embodiment of the present application;
FIG. 8 is a schematic diagram of the location of intersecting coordinates provided by an embodiment of the present application;
FIG. 9 is a schematic diagram of an intersection coordinate calculation provided in an embodiment of the present application;
FIG. 10 is a flowchart of an implementation of a sub-process in a point set based image video masking method provided in an embodiment of the present application;
FIG. 11 is a schematic diagram of a distance field effect provided by an embodiment of the present application;
FIG. 12 is a schematic diagram of a smooth step function provided by an embodiment of the present application;
FIG. 13 is a schematic view of an image video masking apparatus based on a set of points provided in an embodiment of the present application;
fig. 14 is a schematic diagram of a computer device provided in an embodiment of the present application.
Detailed Description
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs; the terminology used in the description of the applications herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "comprising" and "having" and any variations thereof in the description and claims of the present application and in the description of the figures above are intended to cover non-exclusive inclusions. The terms first, second and the like in the description and in the claims or in the above-described figures, are used for distinguishing between different objects and not necessarily for describing a sequential or chronological order.
Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the present application. The appearances of such phrases in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Those of skill in the art will explicitly and implicitly appreciate that the embodiments described herein may be combined with other embodiments.
In order to better understand the technical solutions of the present application, the following description will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the accompanying drawings.
The present invention will be described in detail with reference to the drawings and embodiments.
It should be noted that, the image and video masking method based on the point set provided in the embodiments of the present application is generally executed by a server, and accordingly, the image and video masking device based on the point set is generally configured in the server.
Referring to fig. 1, fig. 1 illustrates one embodiment of a point set based image video masking method.
It should be noted that, if there are substantially the same results, the method of the present invention is not limited to the flow sequence shown in fig. 1, and the method includes the following steps:
s1: a set of points of a mask drawn by a user on a UI control is recorded, and a polygon of the mask is generated based on the set of points.
Specifically, in the embodiment of the application, a Widget (component) of an application development framework records a point set of a mask drawn by a user on a UI control, wherein a coordinate system origin of point coordinates in the point set is at the upper left, and an OpenGL screen coordinate origin is at the lower left. After all the point sets are recorded, the polygon corresponding to the mask is formed by the outermost peripheral point coordinates in the point sets.
S2: and converting the point coordinates in the point set into a coordinate system to obtain texture coordinates.
Specifically, in the embodiment of the present application, the point coordinates in the point set need to be converted into the screen coordinates and then converted into the model coordinates, where the model coordinates are texture coordinates.
Referring to fig. 2, fig. 2 shows a specific embodiment of step S2, which is described in detail as follows:
s21: and converting each point coordinate in the point set into a screen coordinate of OpenGL according to a preset mode, wherein the preset mode is to take an abscissa of the point coordinate as an abscissa of the screen coordinate, and take a value obtained by subtracting an ordinate of the point coordinate from the height of the original image as an ordinate of the screen coordinate.
Specifically, the point coordinates are converted into the screen coordinates of OpenGL by taking the abscissa of the point coordinates as the abscissa of the screen coordinates and the value of the height of the original minus the ordinate of the point coordinates as the ordinate of the screen coordinates. Among them, openGL (english: open Graphics Library, translated: open graphics library or "open graphics library") is a cross-language, cross-platform application programming interface for rendering 2D, 3D vector graphics.
S22: and calculating a model matrix, a view matrix and a projection matrix in the OpenGL, and converting the screen coordinates into a coordinate system based on the model matrix, the view matrix and the projection matrix to obtain the texture coordinates.
Specifically, an object in OpenGL needs to be mapped to a screen through model transformation, view transformation, projection transformation and then through viewport transformation to obtain pixel positions in a screen space, that is, in this example, a model matrix, a view matrix and a projection matrix in OpenGL need to be calculated, and screen coordinates are transformed into a coordinate system based on the model matrix, the view matrix and the projection matrix, so as to render textures on a mask, and texture coordinates are obtained
Referring to fig. 3 and fig. 4, fig. 3 shows a specific implementation manner of step S22, and fig. 4 is a schematic diagram of coordinate system conversion provided in the embodiment of the present application; the details are as follows:
s221: and calculating the model matrix according to the model scaling value, the model rotation value and the model displacement in the OpenGL.
Specifically, M represents a model matrix, model is a scaling of the model, model r is a rotation of the model, and model t is a displacement of the model. The model matrix is calculated as:
M＝ModelS*ModelR*ModelT。
s222: and calculating the view matrix according to the view rotation value and the view displacement in the OpenGL.
Specifically, V represents a view matrix, viewR is rotation of the view, and ViewT is displacement of the view. The view matrix is calculated in the following manner:
V＝ViewR*ViewT。
s223: and taking a preset matrix as the projection matrix.
Specifically, P is defined as the current projection matrix in the embodiment of the present application.
S224: a result matrix is calculated based on the model matrix, the view matrix and the projection matrix.
S225: and converting the screen coordinates into a coordinate system according to the inverse matrix of the result matrix to obtain the texture coordinates.
Specifically, the model matrix, the view matrix and the projection matrix are calculated and multiplied (m×v×p) to obtain a result matrix, in this embodiment, T is defined as the result matrix of m×v×p, and T-1 is represented as the inverse matrix of T. The Model coordinates are T -1 And multiplying the result of screen coordinates, so that the coordinates of the model can be obtained through the screen coordinates, and the texture coordinates can be obtained.
The calculation mode is as follows:
T＝M*V*P；
s3: and identifying texture coordinates inside and outside the polygon to obtain target point coordinates, wherein the target point coordinates comprise target internal point coordinates and target external point coordinates, displaying the target internal point coordinates, and hiding the target external point coordinates.
Specifically, the above steps have transformed the coordinate system of the point coordinates, rendered the mask, and obtained the texture coordinates, so in the embodiment of the present application, it is necessary to identify the texture coordinates inside or outside the polygon of the mask, so as to determine the transparency value of the mask. The target point coordinates comprise target internal point coordinates and target external point coordinates, the target internal point coordinates are displayed, and the target external point coordinates are hidden.
Referring to fig. 5, fig. 5 shows a specific embodiment of step S3, which is described in detail as follows:
s31: and judging whether the ray corresponding to the texture coordinate intersects the polygon or not by adopting a ray method, and obtaining a judging result.
Referring to fig. 6 to 9, fig. 5 shows a specific embodiment of step S31; FIG. 7 is a schematic diagram illustrating a situation where a target ray and a polygon are not intersected according to an embodiment of the present application; FIG. 8 is a schematic diagram of the location of intersecting coordinates provided by an embodiment of the present application; FIG. 9 is a schematic diagram of an intersection coordinate calculation provided in an embodiment of the present application; the details are as follows:
s311: and sending a ray to the right by taking each texture coordinate as a starting point to obtain a target ray.
S312: and eliminating the target ray which is not intersected with the preset polygon.
As shown in fig. 7, a ray is issued to the right for any texture coordinate a, resulting in a target ray AF. The preset non-intersecting condition comprises that the target ray is parallel to and overlapped with the edge of the polygon, the line segment (edge of the polygon) is above the target ray, the line segment is issued under the target ray, the line segment end point is on the target ray and the line segment start point is on the target ray.
S313: and calculating the intersecting coordinates of the target ray and the polygon to obtain intersecting coordinates.
As shown in fig. 9, mx coordinates of an intersection point of AF and BC (sides of a polygon) are calculated in the present application using a principle of triangle similarity; the calculation mode is as follows:
where (Ax, ay) is the texture coordinates, (Bx, by) and (Cx, cy) are the two end coordinates of the side BC of the polygon. It should be noted that, only the abscissa of intersection of the target ray and the polygon needs to be calculated, and then the intersection of the image capture master corresponding to the texture coordinate and the polygon can be judged only by comparing the abscissa.
S314: and if the abscissa of the intersection coordinate is larger than the abscissa of the texture coordinate, judging whether the ray corresponding to the texture coordinate intersects the polygon or not, and obtaining the judging result.
As shown in fig. 8, if the abscissa of the intersection coordinate is greater than the abscissa of the texture coordinate, that is, the intersection point is on the right side of the starting point a, it is determined that the ray corresponding to the texture coordinate intersects the polygon; if not, judging that the ray corresponding to the texture coordinate is not intersected with the polygon.
S32: and calculating the number of intersections of each texture coordinate corresponding ray and the polygon based on the judging result to obtain the target intersection number.
S33: and if the number of the target intersections is an odd number, judging that the texture coordinates are inside the polygon, and taking the texture coordinates inside the polygon as the target internal point coordinates.
S34: and if the number of the target intersections is even, judging that the texture coordinates are outside the polygon, and taking the texture coordinates outside the polygon as the target external point coordinates.
Specifically, the intersection quantity of the corresponding rays of each texture coordinate and the polygon is accumulated to obtain the object intersection quantity, if the object intersection quantity is odd, the texture coordinate is judged to be positioned in the polygon, and the texture coordinate positioned in the polygon is taken as the object external point coordinate; if the number of the intersected objects is even, determining that the texture coordinates are outside the polygon, and obtaining the coordinates of points outside the objects. Further, the coordinates of the points inside the object are subjected to display processing, the coordinates of the points outside the object are subjected to hiding processing, and thus a mask is displayed, and the coordinates of the points at the edges of the polygon are used for eclosion processing.
S4: and calculating the transparency of the shade based on the target point coordinates, smoothing the shade according to the transparency, and modifying the transparent channel and the color value of the original image according to the transparency to obtain a target image.
Referring to fig. 10 to 12, fig. 6 shows a specific embodiment of step S4; FIG. 11 is a schematic diagram of a distance field effect provided by an embodiment of the present application; FIG. 12 is a schematic diagram of a smooth step function provided by an embodiment of the present application; the details are as follows:
s41: and calculating the shortest distance from the coordinates of the target point to the polygon.
Specifically, distances from points to different sides of the polygon are sequentially calculated, the shortest distance minL from the points to the sides of the polygon is taken as the shortest distance minL from the points to the sides, and the formula of the distance from the points to the sides is as follows:
wherein, distanceToedge (P, v0, v 1) is the distance of the point to the edge, < P-v0, v1-v0>: represents the point multiplication operation of the vector, < P-v0, v1-v0>: the point multiplication operation of the expression vector, |x|: representing the length of the vector, P is the target point coordinates and v0 and v1 are sides of the polygon.
S42: and carrying out stage smoothing on the mask based on the shortest distance by adopting a smoothing stage function, and calculating the transparency.
Further, a specific embodiment of step S42 is provided: and inputting any one shortest distance and two preset boundary values into a smoothing stage function to carry out stage smoothing treatment, so as to obtain transparency after the stage smoothing treatment.
The calculation mode of the smoothing stage function is as follows:
y＝k*k*(3-2k)；
wherein t is 1 、t 2 And for the two preset boundary values, x is the shortest distance, k is a smoothing coefficient, and y is the transparency.
In particular, a stage smoothing transition is required because the effect of the distance field, which is a visual effect like feathering, can be produced by smoothing the step function, which produces the effect shown in fig. 11. In the embodiment of the application, the alpha value (transparency) is obtained by performing step smoothing by using smoothstep of OpenGL.
S43: and modifying the transparent channel of the original image based on the transparency, and multiplying the color value of each pixel point on the original image with the transparency to obtain the target image.
In particular, in the embodiment of the present application, in order to avoid the occurrence of the boundary line of the feathering, modification is required to the transparent channel and the color value of the original. And modifying the transparent channel of the original image into a transparency value, and multiplying the transparency by the color value of each pixel point on the original image to modify the color value of the original image so as to obtain the target image. The calculation mode is as follows:
Color＝(r*alpha,g*alpha,b*alpha,alpha)
wherein r, g and b are values of three color channels of the original image, and alpha is transparency.
In the embodiment of the application, a point set of a mask drawn by a user on a UI control is recorded, and a polygon of the mask is generated based on the point set; converting the point coordinates in the point set into a coordinate system to obtain texture coordinates; recognizing texture coordinates inside and outside the polygon to obtain target point coordinates, wherein the target point coordinates comprise target internal point coordinates and target external point coordinates, performing display processing on the target internal point coordinates, and performing hiding processing on the target external point coordinates; and calculating the transparency of the shade based on the target point coordinates, smoothing the shade according to the transparency, and modifying the transparent channel and the color value of the original image according to the transparency to obtain a target image. The embodiment of the invention utilizes the point set of the mask drawn by the user on the UI control to carry out the image video masking without constructing a mask, can adapt to the masking ranges of different scales, and is beneficial to improving the effect and efficiency of the image video masking.
Referring to fig. 13, as an implementation of the method shown in fig. 1, the present application provides an embodiment of an image video masking apparatus based on a point set, where the embodiment of the apparatus corresponds to the embodiment of the method shown in fig. 1, and the apparatus may be specifically applied to various electronic devices.
As shown in fig. 13, the image video masking apparatus based on the point set of the present embodiment includes: a point set recording unit 51, a texture coordinate generating unit 52, a target point coordinate confirming unit 53, and a target image generating unit 54, wherein:
a point set recording unit 51 for recording a point set of a mask drawn by a user on a UI control, and generating a polygon of the mask based on the point set;
a texture coordinate generating unit 52, configured to convert the point coordinates in the point set into a coordinate system, so as to obtain texture coordinates;
a target point coordinate confirmation unit 53, configured to identify texture coordinates inside and outside the polygon, and obtain target point coordinates, where the target point coordinates include a target internal point coordinate and a target external point coordinate, perform display processing on the target internal point coordinate, and perform hiding processing on the target external point coordinate;
and a target image generating unit 54, configured to calculate the transparency of the mask based on the target point coordinates, smooth the mask according to the transparency, and modify the transparent channel and the color value of the original image according to the transparency, so as to obtain a target image.
Further, the texture coordinate generation unit 52 includes:
a screen coordinate conversion unit, configured to convert each point coordinate in the point set into a screen coordinate of OpenGL according to a preset manner, where the preset manner is to use an abscissa of the point coordinate as an abscissa of the screen coordinate, and use a value obtained by subtracting an ordinate of the point coordinate from an altitude of the original image as an ordinate of the screen coordinate;
and the coordinate system conversion unit is used for calculating a model matrix, a view matrix and a projection matrix in the OpenGL, and converting the screen coordinate into a coordinate system based on the model matrix, the view matrix and the projection matrix to obtain the texture coordinate.
Further, the coordinate system conversion unit includes:
a model matrix calculation unit, configured to calculate the model matrix according to a model scaling value, a model rotation value, and a model displacement in OpenGL;
a view matrix calculating unit, configured to calculate the view matrix according to a view rotation value and a view displacement in OpenGL;
the projection matrix generation unit is used for taking a preset matrix as the projection matrix;
a result matrix calculation unit configured to calculate a result matrix based on the model matrix, the view matrix, and the projection matrix;
and the texture coordinate acquisition unit is used for converting the screen coordinates into a coordinate system according to the inverse matrix of the result matrix to obtain the texture coordinates.
Further, the target point coordinate confirmation unit 53 includes:
the judging result generating unit is used for judging whether the ray corresponding to the texture coordinate intersects the polygon or not by adopting a ray method to obtain a judging result;
the target intersection number generating unit is used for calculating the number of intersections between each texture coordinate corresponding ray and the polygon based on the judging result to obtain the target intersection number;
a target external point coordinate identifying unit, configured to determine that the texture coordinate is inside the polygon if the number of target intersections is an odd number, and take the texture coordinate inside the polygon as the target internal point coordinate;
and the target external and internal point coordinate recognition unit is used for judging that the texture coordinates are outside the polygon if the number of the target intersections is even, and taking the texture coordinates outside the polygon as the target external point coordinates, wherein the target internal point coordinates are subjected to display processing and the target external point coordinates are subjected to hiding processing.
Further, the judgment result generation unit includes:
the target ray generation unit is used for sending a ray to the right by taking each texture coordinate as a starting point to obtain a target ray;
a ray elimination unit, configured to eliminate a target ray that is not intersected with the preset polygon;
the intersecting coordinate calculating unit is used for calculating the intersecting coordinate of the target ray and the polygon to obtain an intersecting coordinate;
and the coordinate judging unit is used for judging whether the ray corresponding to the texture coordinate intersects the polygon or not if the abscissa of the intersecting coordinate is larger than the abscissa of the texture coordinate, so as to obtain the judging result.
Further, the target image generation unit 54 includes:
a shortest distance calculating unit for calculating the shortest distance from the coordinates of the target point to the polygon;
a transparency calculation unit for performing a stage smoothing process on the mask based on the shortest distance using a smoothing stage function, and calculating the transparency;
and the image modification unit is used for modifying the transparent channel of the original image based on the transparency, multiplying the color value of each pixel point on the original image with the transparency, and modifying the color value of the original image to obtain the target image.
Further, the transparency calculation unit includes:
the smoothing processing unit is used for inputting any shortest distance and two preset boundary values into a smoothing stage function to carry out stage smoothing processing, so as to obtain transparency after the stage smoothing processing;
the calculation mode of the smoothing stage function is as follows:
y＝k*k*(3-2k)；
wherein t is 1 、t 2 And for the two preset boundary values, x is the shortest distance, k is a smoothing coefficient, and y is the transparency.
In the embodiment of the application, a point set of a mask drawn by a user on a UI control is recorded, and a polygon of the mask is generated based on the point set; converting the point coordinates in the point set into a coordinate system to obtain texture coordinates; recognizing texture coordinates inside and outside the polygon to obtain target point coordinates, wherein the target point coordinates comprise target internal point coordinates and target external point coordinates, performing display processing on the target internal point coordinates, and performing hiding processing on the target external point coordinates; and calculating the transparency of the shade based on the target point coordinates, smoothing the shade according to the transparency, and modifying the transparent channel and the color value of the original image according to the transparency to obtain a target image. The embodiment of the invention utilizes the point set of the mask drawn by the user on the UI control to carry out the image video masking without constructing a mask, can adapt to the masking ranges of different scales, and is beneficial to improving the effect and efficiency of the image video masking.
In order to solve the technical problems, the embodiment of the application also provides computer equipment. Referring specifically to fig. 14, fig. 14 is a basic structural block diagram of a computer device according to the present embodiment.
The computer device 6 comprises a memory 61, a processor 62, a network interface 63 communicatively connected to each other via a system bus. It should be noted that only a computer device 6 having three components, a memory 61, a processor 62, and a network interface 63, is shown in the figures, but it should be understood that not all of the illustrated components are required to be implemented and that more or fewer components may be implemented instead. It will be appreciated by those skilled in the art that the computer device herein is a device capable of automatically performing numerical calculations and/or information processing in accordance with predetermined or stored instructions, the hardware of which includes, but is not limited to, microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASICs), programmable gate arrays (fields-Programmable Gate Array, FPGAs), digital processors (Digital Signal Processor, DSPs), embedded devices, etc.
The computer device may be a desktop computer, a notebook computer, a palm computer, a cloud server, or the like. The computer device can perform man-machine interaction with a user through a keyboard, a mouse, a remote controller, a touch pad or voice control equipment and the like.
The memory 61 includes at least one type of readable storage media including flash memory, hard disk, multimedia card, card memory (e.g., SD or DX memory, etc.), random Access Memory (RAM), static Random Access Memory (SRAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), programmable Read Only Memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 61 may be an internal storage unit of the computer device 6, such as a hard disk or memory of the computer device 6. In other embodiments, the memory 61 may also be an external storage device of the computer device 6, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card) or the like, which are provided on the computer device 6. Of course, the memory 61 may also include both internal storage units of the computer device 6 and external storage devices. In the present embodiment, the memory 61 is typically used for storing an operating system installed on the computer device 6 and various kinds of application software, such as program codes of an image video masking method based on a point set. Further, the memory 61 may also be used to temporarily store various types of data that have been output or are to be output.
The processor 62 may be a central processing unit (Central Processing Unit, CPU), controller, microcontroller, microprocessor, or other data processing chip in some embodiments. The processor 62 is typically used to control the overall operation of the computer device 6. In this embodiment, the processor 62 is configured to execute the program code stored in the memory 61 or process data, for example, the program code of the above-mentioned point set-based image video masking method, to implement various embodiments of the point set-based image video masking method.
The network interface 63 may comprise a wireless network interface or a wired network interface, which network interface 63 is typically used for establishing a communication connection between the computer device 6 and other electronic devices.
The present application also provides another embodiment, namely, a computer readable storage medium storing a computer program executable by at least one processor to cause the at least one processor to perform the steps of a point set-based image video masking method as described above.
From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk), comprising several instructions for causing a terminal device (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to perform the method of the embodiments of the present application.
It is apparent that the embodiments described above are only some embodiments of the present application, but not all embodiments, the preferred embodiments of the present application are given in the drawings, but not limiting the patent scope of the present application. This application may be embodied in many different forms, but rather, embodiments are provided in order to provide a more thorough understanding of the present disclosure. Although the present application has been described in detail with reference to the foregoing embodiments, it will be apparent to those skilled in the art that modifications may be made to the embodiments described in the foregoing, or equivalents may be substituted for elements thereof. All equivalent structures made by the specification and the drawings of the application are directly or indirectly applied to other related technical fields, and are also within the protection scope of the application.
Claims (10)
1. A method of masking an image video based on a set of points, comprising:
recording a point set of a mask drawn by a user on a UI control, and generating a polygon of the mask based on the point set;
converting the point coordinates in the point set into a coordinate system to obtain texture coordinates;
recognizing texture coordinates inside and outside the polygon to obtain target point coordinates, wherein the target point coordinates comprise target internal point coordinates and target external point coordinates, performing display processing on the target internal point coordinates, and performing hiding processing on the target external point coordinates;
and calculating the transparency of the shade based on the target point coordinates, smoothing the shade according to the transparency, and modifying the transparent channel and the color value of the original image according to the transparency to obtain a target image.
2. The method of claim 1, wherein transforming the coordinates of the points in the set of points to obtain texture coordinates comprises:
converting each point coordinate in the point set into a screen coordinate of OpenGL according to a preset mode, wherein the preset mode is to take an abscissa of the point coordinate as an abscissa of the screen coordinate, and take a value obtained by subtracting an ordinate of the point coordinate from the height of the original image as an ordinate of the screen coordinate;
and calculating a model matrix, a view matrix and a projection matrix in the OpenGL, and converting the screen coordinates into a coordinate system based on the model matrix, the view matrix and the projection matrix to obtain the texture coordinates.
3. The point set-based image video mask method of claim 2, wherein the computing the model matrix, the view matrix, and the projection matrix in OpenGL and converting the screen coordinates into a coordinate system based on the model matrix, the view matrix, and the projection matrix to render a texture to the mask, resulting in texture coordinates, comprises:
calculating the model matrix according to the model scaling value, the model rotation value and the model displacement in the OpenGL;
calculating the view matrix according to the view rotation value and the view displacement in the OpenGL;
taking a preset matrix as the projection matrix;
calculating a result matrix based on the model matrix, the view matrix and the projection matrix;
and converting the screen coordinates into a coordinate system according to the inverse matrix of the result matrix to obtain the texture coordinates.
4. The point set-based image video masking method of claim 1, wherein the identifying texture coordinates inside and outside the polygon, resulting in target point coordinates, comprises:
judging whether a ray corresponding to the texture coordinate intersects the polygon or not by adopting a ray method to obtain a judging result;
calculating the number of intersections of each texture coordinate corresponding ray and the polygon based on the judging result to obtain the number of target intersections;
if the number of the target intersections is an odd number, judging that the texture coordinates are inside the polygon, and taking the texture coordinates inside the polygon as the target internal point coordinates;
if the number of the target intersections is even, determining that the texture coordinates are outside the polygon, and taking the texture coordinates outside the polygon as the target external point coordinates, wherein the target internal point coordinates are subjected to display processing, and the target external point coordinates are subjected to hiding processing.
5. The method of claim 4, wherein determining whether the ray corresponding to the texture coordinate intersects the polygon by using a ray method to obtain a determination result comprises:
taking each texture coordinate as a starting point to send out a ray to the right to obtain a target ray;
eliminating target rays which are not intersected with the preset polygon;
calculating the intersecting coordinates of the target ray and the polygon to obtain intersecting coordinates;
and if the abscissa of the intersection coordinate is larger than the abscissa of the texture coordinate, judging whether the ray corresponding to the texture coordinate intersects the polygon or not, and obtaining the judging result.
6. The method of any one of claims 1 to 5, wherein calculating the transparency of the mask based on the target point coordinates, smoothing the mask according to the transparency, and modifying the transparency channel and color values of the original image according to the transparency to obtain a target image comprises:
calculating the shortest distance from the coordinates of the target point to the polygon;
performing stage smoothing processing on the mask based on the shortest distance by adopting a smoothing stage function, and calculating the transparency;
and modifying the transparent channel of the original image based on the transparency, and multiplying the color value of each pixel point on the original image with the transparency to obtain the target image.
7. The point set based image video mask method according to claim 6, wherein the performing a stage smoothing process on the mask based on the shortest distance using a smoothing stage function and calculating the transparency comprises:
inputting any shortest distance and two preset boundary values into a smoothing stage function to carry out stage smoothing treatment, so as to obtain transparency after the stage smoothing treatment;
the calculation mode of the smoothing stage function is as follows:
y＝k*k*(3-2k)；
wherein t is 1 、t 2 And for the two preset boundary values, x is the shortest distance, k is a smoothing coefficient, and y is the transparency.
8. An image video masking apparatus based on a set of points, comprising:
a point set recording unit, configured to record a point set of a mask drawn by a user on a UI control, and generate a polygon of the mask based on the point set;
the texture coordinate generation unit is used for converting the point coordinates in the point set into a coordinate system to obtain texture coordinates;
the target point coordinate confirming unit is used for identifying texture coordinates inside and outside the polygon to obtain target point coordinates, wherein the target point coordinates comprise target inner point coordinates and target outer point coordinates, the target inner point coordinates are subjected to display processing, and the target outer point coordinates are subjected to hiding processing;
and the target image generation unit is used for calculating the transparency of the shade based on the target point coordinates, smoothing the shade according to the transparency, and modifying the transparent channel and the color value of the original image according to the transparency to obtain a target image.
9. A computer device comprising a memory and a processor, the memory having stored therein a computer program, the processor implementing the point set based image video masking method of any one of claims 1 to 7 when the computer program is executed.
10. A computer readable storage medium, having stored thereon a computer program which, when executed by a processor, implements the point set based image video masking method of any of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311856747.2A CN117793442B (en) | 2023-12-29 | 2023-12-29 | Image video masking method, device, equipment and medium based on point set |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311856747.2A CN117793442B (en) | 2023-12-29 | 2023-12-29 | Image video masking method, device, equipment and medium based on point set |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117793442A true CN117793442A (en) | 2024-03-29 |
| CN117793442B CN117793442B (en) | 2024-07-09 |
Family
ID=90396073
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311856747.2A Active CN117793442B (en) | 2023-12-29 | 2023-12-29 | Image video masking method, device, equipment and medium based on point set |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117793442B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2005293616A (en) * | 2005-06-06 | 2005-10-20 | Dainippon Printing Co Ltd | Image synthesis device |
| CN106780642A (en) * | 2016-11-15 | 2017-05-31 | 网易（杭州）网络有限公司 | The generation method and device of dense fog shade textures |
| CN110276791A (en) * | 2019-06-28 | 2019-09-24 | 北京航空航天大学 | A Depth Camera Simulation Method with Configurable Parameters |
| CN111045582A (en) * | 2019-11-28 | 2020-04-21 | 深圳市木愚科技有限公司 | Personalized virtual portrait activation interaction system and method |
| CN112734896A (en) * | 2021-01-08 | 2021-04-30 | 网易（杭州）网络有限公司 | Environment shielding rendering method and device, storage medium and electronic equipment |
-
2023
- 2023-12-29 CN CN202311856747.2A patent/CN117793442B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2005293616A (en) * | 2005-06-06 | 2005-10-20 | Dainippon Printing Co Ltd | Image synthesis device |
| CN106780642A (en) * | 2016-11-15 | 2017-05-31 | 网易（杭州）网络有限公司 | The generation method and device of dense fog shade textures |
| CN110276791A (en) * | 2019-06-28 | 2019-09-24 | 北京航空航天大学 | A Depth Camera Simulation Method with Configurable Parameters |
| CN111045582A (en) * | 2019-11-28 | 2020-04-21 | 深圳市木愚科技有限公司 | Personalized virtual portrait activation interaction system and method |
| CN112734896A (en) * | 2021-01-08 | 2021-04-30 | 网易（杭州）网络有限公司 | Environment shielding rendering method and device, storage medium and electronic equipment |
Non-Patent Citations (2)
| Title |
|---|
| A. GRIGOREV 等: "Coordinate-Based Texture Inpainting for Pose-Guided Human Image Generation", 2019 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 9 January 2020 (2020-01-09), pages 12127 - 12136 * |
| 郭璠: "图像去雾方法和评价及其应用研究", 中国博士学位论文全文数据库, no. 02, 15 February 2013 (2013-02-15) * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117793442B (en) | 2024-07-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111583381B (en) | Game resource map rendering method and device and electronic equipment | |
| CN112734896B (en) | Environment shielding rendering method and device, storage medium and electronic equipment | |
| US20090303242A1 (en) | Methods and apparatuses to arbitrarily transform windows | |
| CN112766027A (en) | Image processing method, device, equipment and storage medium | |
| CN110428504B (en) | Text image synthesis method, apparatus, computer device and storage medium | |
| CN116802683A (en) | Image processing method and system | |
| CN112580213B (en) | Method and device for generating display image of electric field lines and storage medium | |
| CN116385859A (en) | Primitive identification method, device, computer equipment and medium thereof | |
| KR20060052042A (en) | Hardware accelerated anti-aliasing method and system in 3D | |
| CN116070687A (en) | Neural network light field representation method based on global ray space affine transformation | |
| CN112528707A (en) | Image processing method, device, equipment and storage medium | |
| CN113407080B (en) | Method, device, computer equipment and storage medium for generating component labels | |
| CN117793442B (en) | Image video masking method, device, equipment and medium based on point set | |
| CN114693515A (en) | Image deformation method and device, electronic device and storage medium | |
| CN112149745B (en) | Method, device, device and storage medium for determining difficult case samples | |
| CN111552755B (en) | Drawing method, device and equipment for three-dimensional place name label and storage medium | |
| CN116309274B (en) | Method and device for detecting small target in image, computer equipment and storage medium | |
| CN112465692A (en) | Image processing method, device, equipment and storage medium | |
| Ruwanthika et al. | Dynamic 3D model construction using architectural house plans | |
| CN112767518A (en) | Virtual animation special effect making method and device and electronic equipment | |
| US20230316597A1 (en) | Method and apparatus for rendering hair, computer storage medium, electronic device | |
| CN112464696A (en) | Image processing method, device, equipment and storage medium | |
| CN116962817B (en) | Video processing method, device, electronic equipment and storage medium | |
| CN114546555B (en) | Graph pickup method, device and medium based on space geometry | |
| CN117789211A (en) | Simulation data labeling method, device, equipment and medium based on simulation platform |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |