CN117333586A - Image processing method and device, storage medium and electronic equipment - Google Patents
Image processing method and device, storage medium and electronic equipment Download PDFInfo
- Publication number
- CN117333586A CN117333586A CN202311628974.XA CN202311628974A CN117333586A CN 117333586 A CN117333586 A CN 117333586A CN 202311628974 A CN202311628974 A CN 202311628974A CN 117333586 A CN117333586 A CN 117333586A
- Authority
- CN
- China
- Prior art keywords
- image
- dynamic
- determining
- transformation
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T11/00—2D [Two Dimensional] image generation
- G06T11/60—Editing figures and text; Combining figures or text
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Biophysics (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Image Processing (AREA)
Abstract
The application discloses an image processing method, a storage medium and an electronic device, wherein the method comprises the following steps: according to the extraction operation of the main body image in the target image, determining the extracted main body image as a first image; performing a repair process on an image having a missing region formed in the target image based on the extraction operation, determining the repair-processed image as a second image; wherein the edge of the missing region corresponds to the contour edge of the subject image; performing image dynamic processing on the first image according to the set dynamic transformation period, and determining the processed sequence frame image as a dynamic sequence frame image which is dynamically transformed in the dynamic transformation period; the image after the fusion processing of the dynamic sequence frame image and the second image is determined to be a dynamic composite image; therefore, the visual perception and impact can be improved, and the expression mode of commodity images in the application software platform is enriched.
Description
Technical Field
The present disclosure relates to the field of electronic information technology of computers, and in particular, to an image processing method and apparatus. The application also relates to a method and a device for processing the food image, a computer storage medium and an electronic device.
Background
With the continuous development of computer and internet technologies, life service applications are increasingly diversified and comprehensive, and are also called as one of important life components. The creation of life service class applications improves the convenience of life services, such as: application software for dining, shopping, distribution and the like. In the life service application program, the embodiment of the commodity object can comprise text information description and image information description of the commodity object so as to transmit related information of the commodity object to a user, thereby improving the perception and experience of the user on the commodity object in an on-line scene.
In general, the image information description of the commodity object is mainly expressed in the form of a still image, such as: for commodity objects of the food class, photos of the food can be output as image information, text information and the like in an application platform interface, so that a user can intuitively know commodity information.
Disclosure of Invention
The application provides an image processing method to solve the technical problems of high dynamic processing cost and complex processing of commodity pictures in the prior art.
The application provides an image processing method, which comprises the following steps:
according to the extraction operation of the main body image in the target image, determining the extracted main body image as a first image;
performing a repair process on an image having a missing region formed in the target image based on the extraction operation, determining the repair-processed image as a second image; wherein the edge of the missing region corresponds to the contour edge of the subject image;
performing image dynamic processing on the first image according to the set dynamic transformation period, and determining the processed sequence frame image as a dynamic sequence frame image which is dynamically transformed in the dynamic transformation period;
and determining the image after the fusion processing of the dynamic sequence frame image and the second image as a dynamic composite image.
In some embodiments, the determining the extracted subject image as the first image according to the extracting operation of the subject image in the target image includes:
and determining the extracted subject image as a first image according to the extraction interaction operation of the subject image in the target image.
In some embodiments, the determining the extracted subject image as a first image according to an extraction interaction of the subject image in the target image includes:
determining coordinate information of the subject image in the target image in response to a selection operation of the subject image in the target image;
determining a coordinate feature vector according to the coding of the coordinate information;
taking the coordinate vector features and the image feature vectors of the target image as extraction requirement information, and inputting the extraction requirement information into a large model;
and determining an image output by the large model as the first image.
In some embodiments, the performing a repair process on an image having a missing region formed in the target image based on the extracting operation, determining the repair processed image as a second image, includes:
determining a binarized image according to binarization processing of an image including the missing region;
determining an image to be repaired according to channel superposition of the binarized image and the target image;
and inputting the image to be repaired into a convolutional neural network for learning, and acquiring the second image.
In some embodiments, the performing image dynamic processing on the first image according to the set cycle period, determining the processed sequence frame image as a dynamic sequence frame image that is dynamically transformed in the cycle period, including:
performing image dynamic transformation processing on the first image according to a transformation mode selected in a transformation template library, and determining a dynamic frame image;
sequencing the dynamic frame images according to a dynamic playing order to obtain the sequence frame images;
and determining the sequence frame images played according to the cycle period as the dynamic sequence frame images.
In some embodiments, the performing image dynamic transformation processing on the first image according to the transformation mode selected in the transformation template library, and determining a dynamic frame image includes:
determining a target transformation mode corresponding to the attribute in the transformation template library according to the attribute of the main image;
and according to the target transformation mode, determining the frame image subjected to image dynamic transformation processing on the first image as the dynamic frame image.
In some embodiments, the determining, according to the target transformation mode, a frame image that performs image dynamic transformation processing on the first image as the dynamic frame image includes:
when the target transformation mode is out-of-frame transformation, centering the first image in a transparent base map, and setting the image size of the transparent base map to be larger than that of the first image;
and carrying out affine transformation on the first image from the center of the transparent base image to the edge in sequence to obtain the dynamic frame image.
In some embodiments, the determining, according to the target transformation mode, a frame image that performs image dynamic transformation processing on the first image as the dynamic frame image includes:
when the target transformation mode is twisting transformation, a plurality of transformation sequence frame images in a preset period are obtained through transformation adjustment in the height direction and/or the width direction of the first image;
the plurality of transform sequence image frames is determined as the dynamic frame image.
In some embodiments, the determining the fused image of the dynamic sequence frame image and the second image as a dynamic composite image includes:
and respectively carrying out transparency mixing on the dynamic sequence frame images and the second image to obtain the dynamic synthetic image.
In some embodiments, the transparency mixing the dynamic sequence frame image with the second image respectively to obtain the dynamic composite image includes:
determining a first alpha channel matrix of the dynamic sequence frame image;
determining a second alpha channel matrix of the second image;
and acquiring the dynamic synthesized image according to the first alpha channel matrix and the second alpha channel matrix.
The present application also provides an image processing apparatus including:
an extraction unit configured to determine an extracted subject image as a first image according to an extraction operation of the subject image in a target image;
a restoration unit configured to perform restoration processing on an image having a missing region formed in the target image based on the extraction operation, the restoration-processed image being determined as a second image; wherein the edge of the missing region corresponds to the contour edge of the subject image;
the dynamic processing unit is used for carrying out image dynamic processing on the first image according to a set dynamic transformation period, and determining the processed sequence frame image as a dynamic sequence frame image which is dynamically transformed in the dynamic transformation period;
and the fusion unit is used for determining the image after the fusion processing of the dynamic sequence frame image and the second image as a dynamic synthetic image.
The application also discloses a meal image processing method, which comprises the following steps:
according to the extraction operation of the main meal image in the target meal image, determining the extracted main meal image as a first image;
performing repair processing on an image having a missing region formed in the target meal image based on the extraction operation, and determining the repair processed image as a second image; wherein the edge of the missing region corresponds to the contour edge of the main meal image;
performing image dynamic processing on the first image according to the set dynamic transformation period, and determining the processed sequence frame image as a dynamic sequence frame image which is dynamically transformed in the dynamic transformation period;
and determining the image after the fusion processing of the dynamic sequence frame image and the second image as a dynamic synthesized meal image.
The application also provides a meal image processing device, including:
the extraction unit is used for determining the extracted main meal image as a first image according to the extraction operation of the main meal image in the target meal image;
a restoration unit configured to perform restoration processing on an image having a missing region formed in the target meal image based on the extraction operation, the restoration-processed image being determined as a second image; wherein the edge of the missing region corresponds to the contour edge of the main meal image;
the dynamic processing unit is used for carrying out image dynamic processing on the first image according to a set dynamic transformation period, and determining the processed sequence frame image as a dynamic sequence frame image which is dynamically transformed in the dynamic transformation period;
and the fusion unit is used for determining the image after the fusion processing of the dynamic sequence frame image and the second image as a dynamic synthesized meal image.
The present application also provides a computer storage medium for storing a computer program;
the program performs the image processing method as described above, or performs the meal image processing method as described above.
The application also provides an electronic device comprising:
a processor;
and a memory for storing a computer program for executing the image processing method as described above or for executing the meal image processing method as described above.
Compared with the prior art, the application has the following advantages:
according to the image processing method, after the main image is extracted from the target image, the restoration effect is close to that of the original image, so that after the first image and the restored second image which are obtained by performing effective processing on the main image are fused, the restoration edge is increased smoothly and truly, and a dynamic composite image can be formed after the first image and the second image are fused, so that the visual perception and impact can be improved, and the expression mode of commodity images in an application software platform is enriched. Compared with the dynamic processing process of commodity images in the prior art, the processing is simpler and more convenient, and the processing cost is lower.
Drawings
Fig. 1 is a flowchart of an image processing method provided in the present application.
Fig. 2 is a flowchart of an embodiment of combining application scenarios in an image processing method provided in the present application.
Fig. 3 is a schematic view of a scene of an embodiment of frame out transformation in an image processing method provided in the present application.
Fig. 4 is a schematic view of a scene of a twist transform embodiment in an image processing method provided in the present application.
Fig. 5 is a schematic structural diagram of an image processing apparatus provided in the present application.
Fig. 6 is a flowchart of a method for processing a meal image provided in the present application.
Fig. 7 is a schematic structural diagram of a food image processing device provided in the present application.
Fig. 8 is a schematic structural diagram of an electronic device provided in the present application.
Detailed Description
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present application. This application is, however, susceptible of embodiment in many other ways than those herein described and similar generalizations can be made by those skilled in the art without departing from the spirit of the application and the application is therefore not limited to the specific embodiments disclosed below.
The terminology used in the present application is for the purpose of describing particular embodiments only and is not intended to be limiting of the present application. The manner of description used in this application and in the appended claims is for example: "a", "a" and "a" etc. are not limited in number or order, but are used to distinguish the same type of information from each other.
Based on the background technology, the invention concept of the image processing method provided by the application is derived from the fact that the expression of the meal in the existing ordering application platform adopts a static image form, and better visual perceptibility cannot be achieved. Other application platforms exist, such as: the shopping application platform comprises commodities such as clothes and shoes, and the diversified expression mode of commodity images is realized by video recording and the like of the commodities, however, the mode can lead to higher manufacturing cost on one hand, and visual perceptibility or impact force cannot be realized by dynamic difference between the commodity core part images and background images on the other hand. Accordingly, the present application provides an image processing method capable of highlighting the visual impact of a commodity core portion while reducing processing costs, as will be described in detail below.
As shown in fig. 1, fig. 1 is a flowchart of an image processing method provided in the present application, where the method may include:
step S101: according to the extraction operation of the main body image in the target image, the extracted main body image is determined to be a first image.
Step S102: performing a repair process on an image having a missing region formed in the target image based on the extraction operation, determining the repair-processed image as a second image; wherein the edges of the missing region correspond to contour edges of the subject image.
Step S103: and carrying out image dynamic processing on the first image according to the set dynamic transformation period, and determining the processed sequence frame image as a dynamic sequence frame image which is dynamically transformed in the dynamic transformation period.
Step S104: and determining the image after the fusion processing of the dynamic sequence frame image and the second image as a dynamic composite image.
The following describes the steps S101 to S104 in detail with reference to the specific embodiments, and please refer to fig. 1 with reference to fig. 2, fig. 2 is a flowchart of an embodiment of an image processing method with reference to an application scenario provided in the present application.
The purpose of the step S101 is to: and extracting a main body image from the target image. The target image is an image including a subject element and a background image, and the extraction of the subject image is to divide the subject image and the background image. In this embodiment, the subject element is an element that characterizes the core content of the target image, for example: the dish elements in the dish image are main elements; the clothing in the clothing image is a main body element and the like. The extracted image including the subject element is taken as a first image.
The specific implementation process of step S101 may include:
step S101-1: and determining the extracted subject image as a first image according to the extraction interaction operation of the subject image in the target image.
The extraction interaction operation may be an interactive image extraction operation implemented based on a CV large model, where the CV large model refers to a large deep learning model for a computer vision task, and is typically implemented by using a deep learning algorithm such as a convolutional neural network (Convolutional Neural Network, CNN). The interactive image extraction can also be realized by combining different image video processing models, and the method specifically can comprise the following steps:
step S101-11: determining coordinate information of the subject image in the target image in response to a selection operation of the subject image in the target image; the selection operation may be a click operation on the subject image, and coordinate information of the click operation is obtained. By clicking the main body image in the target image, the main body image can be positioned more accurately.
Step S101-12: determining a coordinate feature vector according to the coding of the coordinate information; in this embodiment, the coordinate points may be encoded into coordinate feature vectors (empedding) by position encoding (position encoding). The position encoder is an important concept in a transducer model, and a coordinate feature vector is determined by a positional relationship between coordinate points.
Step S101-13: taking the coordinate vector features and the image feature vectors of the target image as extraction requirement information, and inputting the extraction requirement information into a CV large model;
step S101-14: and determining an image output by the CV large model as the first image.
In the embodiment, the interactive image extraction or segmentation is realized based on the CV large model, so that the selection of the dimension of a single main body can be realized when the object image with a complex background is faced, and the random combination is more accurate.
Regarding step S102: performing a repair process on an image having a missing region formed in the target image based on the extraction operation, determining the repair-processed image as a second image; wherein the edges of the missing region correspond to contour edges of the subject image.
The purpose of step S102 is to repair the extracted target image, and based on the knowledge in step S101, after the subject image is extracted from the target image, the target image generates a missing region corresponding to the outline of the subject image, and the missing region needs to be modified for the smoothness of the subsequent dynamic composite image, so this step may be performed by using a convolutional neural network, for example: the repair can be achieved using a fast fourier convolutional repair network architecture with a wider image receptive field, high receptive field perceived loss, larger training mask, with lower parameters and computational cost to achieve fine, high quality repair effects, excellent performance even in challenging situations, and generalization to higher resolution images than during training. The specific implementation process of step S102 may include:
step S102-1: determining a binarized image according to binarization processing of an image including the missing region; specifically, the method includes the steps that an image with a missing area is subjected to alpha channel binarization processing, and a binarized image comprising a background part and a part to be repaired is distinguished;
step S102-2: determining an image to be repaired according to channel superposition of the binarized image and the target image;
step S102-3: and inputting the image to be repaired into a convolutional neural network for learning, and acquiring the second image.
The step S102 repairs the target image including the missing region by using the convolutional neural network, so that the edge of the missing region is combined with the background of the target image after repair more precisely and accurately.
Regarding step S103: and carrying out image dynamic processing on the first image according to the set cycle period, and determining the processed sequence frame image as a dynamic sequence frame image which is dynamically transformed in the cycle period.
The purpose of step S103 is to: the first image is dynamically processed, namely: and dynamically displaying the extracted main body image. In this embodiment, the image dynamic processing may employ a conventional computer vision algorithm (VC), such as: linear transformation, cosine transformation, etc., are mapped to changes in the morphology of the body by the principle of the function, the dynamic effect of the first image. Of course, physical characteristics such as deformation, elasticity, distortion (even particles, 3D special effects) and the like with different magnitudes can be simulated through template functions and the like and parameter value updating, so that the dynamic effect of the first image is more diversified, and the limitation of the existing dynamic effect is broken through. Thus, the specific implementation procedure of step S103 may include:
step S103-1: performing image dynamic transformation processing on the first image according to a transformation mode selected in a transformation template library, and determining a dynamic frame image;
step S103-2: sequencing the dynamic frame images according to a dynamic playing order to obtain the sequence frame images;
step S103-3: and determining the sequence frame images played according to the cycle period as the dynamic sequence frame images. A step of
The specific implementation process of the step S103-1 may include:
step S103-11: determining a target transformation mode corresponding to the attribute in the transformation template library according to the main body image; in this embodiment, when the main image is a meal image, the target transformation mode may be determined according to the attribute of the meal, for example: when the meal is a hot pot, a frame-out conversion mode can be adopted, namely, the hot pot representing the main image is converted from the current image size state to multiple-level increase conversion, and the hot pot can be gradually increased and converted, so that the hot air effect can be increased. When the food is cold drink, a twisting transformation mode can be adopted, namely, the cold drink representing the main image is twisted or rotated from left to right and/or up and down from the current image, and the cooling (such as cold air, ice crystal stars and ice breaking, namely, the dynamic effect display representing the cold drink) micro-motion effect can be further increased. That is, the attribute of the meaning expressed by the main image may correspond to the corresponding target transformation mode, and the target transformation mode may be selected by matching the attribute value with the template in the transformation template library, which may be, of course, manually selected.
Step S103-12: and according to the target transformation mode, determining the frame image subjected to image dynamic transformation processing on the first image as the dynamic frame image.
As shown in fig. 3 and 4, based on the target transformation method determined in the step S103-11, the specific implementation procedure of the step S103-12 may include at least two modes as follows.
One way may be understood with reference to fig. 3, and fig. 3 is a schematic view of a frame transformation embodiment in an image processing method provided in the present application, where the one way may specifically include:
steps S103-1211: when the target transformation mode is out-of-frame transformation, centering the first image in a transparent base map, wherein the image size of the transparent base map is larger than that of the first image;
step S103-1212: and carrying out affine transformation on the first image from the center of the transparent base image to the edge in sequence to obtain the dynamic frame image.
A second mode may be understood with reference to fig. 4, and fig. 4 is a schematic view of a case of a twist transformation embodiment in an image processing method provided in the present application, where the second mode may specifically include:
step S103-1221: when the target transformation mode is twisting transformation, a plurality of transformation sequence frame images in a preset period are obtained by carrying out transformation adjustment in the height direction and/or the width direction on the first image; for example: the speed of the change in height and/or width can be numerically controlled using a cosine function, namely: the bulk bbox area remains unchanged, w=s/h, h=h×cos (2pi i/F). Wherein, bbox is an abbreviation of BoundingBox, and refers to a directional rectangular box used for representing the position and the size of an object in target detection. w represents the width of the first image, S represents the area of the first image, and h represents the height of the first image; i represents a frame number; f represents the total number of frames in one cycle. Of course, the three-dimensional space is not limited to the transformation in the height and width directions, and the x, y and z directions in the three-dimensional space can be transformed and adjusted according to the requirements. In the present embodiment, the twist conversion may be performed by a method of changing the middle region of the first image at a high speed, or changing the middle region at a low speed at both sides or the periphery, or the like, and is not particularly limited.
Regarding step S104: and determining the image after the fusion processing of the dynamic sequence frame image and the second image as a dynamic composite image.
The purpose of the step S104 is to: the fusion processing is performed on the dynamic sequence frame image and the second image, so as to realize the superposition dynamic composite image at the pixel level, and the specific implementation process can comprise the following steps:
step S104-1: and respectively carrying out transparency mixing on the dynamic sequence frame images and the second image to obtain the dynamic synthetic image. The transparency blending (Alpha blending) is to blend transparency into color. The specific implementation process can comprise the following steps:
step S104-11: determining a first alpha channel matrix of the dynamic sequence frame image; the alpha channel matrix may be obtained by obtaining an alpha matrix of alpha values of the image to be overlaid, dividing the alpha values in the alpha matrix by 255, such that the values remain between 0-1, alpha_png=img [: 3]/255.0.
Step S104-12: determining a second alpha channel matrix of the second image;
step S104-13: and acquiring the dynamic synthesized image according to the first alpha channel matrix and the second alpha channel matrix.
The most common pixel representation format is RGBA8888, i.e., (r, g, b, a), 8 bits per channel. For convenience of representation, the alpha channel is generally written as a floating point number normalized to 0-1: for example, 50% transparency for red is (255, 0, 127), normalized to (255,0,0,0.5). Thus, the step S104-13 may be calculated using the following formula:
DestinationColor.rgb = (SourceColor.rgb * SourceColor.a) + (DestinationColor.rgb * (1 - SourceColor.a))；
wherein DestinationColor. RGB is the RGB channel value (target color) of the dynamic sequence frame image, sourceColor. RGB is the RGB channel value (primary color) of the second image, sourceColor. A is the alpha channel value (primary color) of the second image.
And multiplying the three channel matrixes of RGB by the alpha matrix respectively according to the formula, and combining the three channel matrixes into a new picture to obtain single-frame dynamic element superposition, namely a dynamic frame image. And then according to the sequence frame, mixing the other single frame images with the transparency of the second image one by one until the fusion of all frame images and the second image is completed, and determining the fusion result frame sequence as a dynamic composite image, namely: may be a target gif file.
In consideration of pixel optimization, unnecessary pixel value residues are removed, only necessary parts are reserved, so that the rendering speed is improved, the waste of rendering resources is avoided, and in the embodiment, software such as ps/camera raw and the like can be used for preliminary compression. Meanwhile, only the difference part between different frames is processed through an open source tool gifsicle, and the same pixel part is not stored redundantly, so that the later appearing frame is less in information and smaller in size. The compression is approximately lossless lossy compression, and can ensure the rendering effect to the maximum extent, wherein the higher the compression rate is, the larger the effect distortion is. Therefore, compression of the gif file (target gif file) can be achieved by selecting a compression mode of the provided multiple-level compression options when compressing the gif file, for example: only the information part of the difference between each picture is kept, the same information part being ignored. Meanwhile, the similarity of adjacent pixels is utilized in the single graph, so that the compression rate is greatly improved; only the information part of the difference between each picture is reserved, and meanwhile, the transparency configuration value is used for reducing the size of a single pixel value; the image pixel values are changed to reduce the file size, the larger the value the smaller the file.
The method can enable the restoration effect of the target image on the missing part generated by extraction to be close to that of the original image after the main image is extracted, further enable the restoration edge to be smooth and real after the first image and the restored second image which are obtained by carrying out effective processing on the main image are fused, enable dynamic synthetic images to be formed after the first image and the second image are fused, improve visual perception and impact, and enrich the expression mode of commodity images in an application software platform.
The foregoing is a specific description of an embodiment of an image processing method, corresponding to the foregoing embodiment of an image processing method, and further discloses an embodiment of an image processing apparatus, please refer to fig. 5, and since the embodiment of the apparatus is substantially similar to the embodiment of the method, the description is relatively simple, and the relevant points refer to the part of the description of the embodiment of the method. The device embodiments described below are merely illustrative.
As shown in fig. 5, fig. 5 is a schematic structural diagram of an image processing apparatus provided in the present application, and the apparatus may include:
an extraction unit 501 configured to determine an extracted subject image as a first image according to an extraction operation of the subject image in a target image;
a restoration unit 502 for performing restoration processing on an image having a missing region formed in the target image based on the extraction operation, the restoration processed image being determined as a second image; wherein the edge of the missing region corresponds to the contour edge of the subject image;
a dynamic processing unit 503, configured to perform image dynamic processing on the first image according to a set dynamic transformation period, and determine a processed sequence frame image as a dynamic sequence frame image that is dynamically transformed in the dynamic transformation period;
and a fusion unit 504, configured to determine the image after the fusion processing of the dynamic sequence frame image and the second image as a dynamic composite image.
Wherein the extracting unit 501 may specifically be configured to determine the extracted subject image as the first image according to an extraction interaction operation on the subject image in the target image. The method specifically comprises the following steps: the device comprises a first determining subunit, a second determining subunit, a transmitting subunit and a third determining subunit.
The first determination subunit is configured to determine coordinate information of the subject image in the target image in response to a selection operation of the subject image in the target image.
The second determining subunit is configured to determine a coordinate feature vector according to the encoding of the coordinate information.
The sending subunit is configured to send an extraction request to a large language model by using the coordinate vector feature and the image feature vector of the target image as extraction requirement information.
The third determining subunit is configured to determine, as the first image, an image fed back by the large language model based on the extraction request.
The repair unit 502 includes: the device comprises a first determining subunit, a second determining subunit and an acquiring subunit.
The first determination subunit is configured to determine a binarized image according to binarization processing of an image including the missing region.
The second determining subunit is configured to determine an image to be repaired according to the channel superposition of the binarized image and the target image.
The acquisition subunit is used for inputting the image to be repaired into a convolutional neural network for learning, and acquiring the second image.
The dynamic processing unit 503 may include: a transform subunit, a sort subunit, and a determine subunit.
The transformation subunit is used for carrying out image dynamic transformation processing on the first image according to the transformation mode selected in the transformation template library and determining a dynamic frame image;
the sequencing subunit is used for sequencing the dynamic frame images according to a dynamic playing sequence to obtain the sequence frame images;
the determining subunit is configured to determine the sequence frame image played according to the cycle period as the dynamic sequence frame image.
The transformation subunit may include: a mode determination subunit and an image determination subunit; the mode determining subunit is used for determining a target transformation mode corresponding to the attribute in the transformation template library according to the attribute of the main image; the image determining subunit is configured to determine, according to the target transformation mode, a frame image that performs image dynamic transformation processing on the first image as the dynamic frame image.
The image determination subunit may specifically include:
mode one:
and the setting subunit is used for centering the first image in the transparent base map when the target transformation mode is out-of-frame transformation, and setting the image size of the transparent base map to be larger than that of the first image.
And the acquisition subunit is used for sequentially carrying out affine transformation on the first image from the center of the transparent base map to the edge to acquire the dynamic frame image.
Mode two:
obtaining a subunit: and the method is used for obtaining a plurality of transformation sequence frame images in a preset period by carrying out transformation adjustment on the first image in the height direction and/or the width direction when the target transformation mode is torsion transformation.
An image determination subunit configured to determine the plurality of transform sequence image frames as the dynamic frame image.
The fusion unit 504 is specifically configured to perform transparency mixing on the dynamic sequence frame images and the second image, so as to obtain the dynamic composite image. The method specifically comprises the following steps: the first matrix determining subunit, the second matrix determining subunit, and the acquiring subunit.
The first matrix determining subunit is configured to determine a first alpha channel matrix of the dynamic sequence frame image.
The second matrix determining subunit is configured to determine a second alpha channel matrix of the second image.
The acquisition subunit is configured to acquire the dynamic synthesized image according to the first alpha channel matrix and the second alpha channel matrix.
The foregoing is a description of an embodiment of an image processing apparatus provided in the present application, and reference may be made to the foregoing method embodiment for details of the apparatus embodiment, which are not described in detail herein.
Based on the foregoing, the present application further provides a method for processing an image of a meal, as shown in fig. 6, fig. 6 is a flowchart of the method for processing an image of a meal, where the method includes:
step S601: according to the extraction operation of the main meal image in the target meal image, determining the extracted main meal image as a first image;
step S602: performing repair processing on an image having a missing region formed in the target meal image based on the extraction operation, and determining the repair processed image as a second image; wherein the edge of the missing region corresponds to the contour edge of the main meal image;
step S603: performing image dynamic processing on the first image according to the set dynamic transformation period, and determining the processed sequence frame image as a dynamic sequence frame image which is dynamically transformed in the dynamic transformation period;
step S604: and determining the image after the fusion processing of the dynamic sequence frame image and the second image as a dynamic synthesized meal image.
Regarding the steps S601 to S604, reference may be made to the contents of the steps S101 to S104, and the detailed description will not be repeated here.
Accordingly, the present application further provides a meal image processing apparatus, as shown in fig. 7, fig. 7 is a schematic structural diagram of the meal image processing apparatus provided in the present application, where the apparatus may include:
an extraction unit 701, configured to determine, as a first image, a main meal image in a target meal image according to an extraction operation of the main meal image;
a repair unit 702 for performing repair processing on an image having a missing region formed in the target meal image based on the extraction operation, the repair processed image being determined as a second image; wherein the edge of the missing region corresponds to the contour edge of the main meal image;
a dynamic processing unit 703, configured to perform image dynamic processing on the first image according to a set dynamic transformation period, and determine a processed sequence frame image as a dynamic sequence frame image that is dynamically transformed in the dynamic transformation period;
and a fusion unit 704, configured to determine the image after the fusion processing of the dynamic sequence frame image and the second image as a dynamic synthesized food image.
Similarly, for the specific implementation procedure of the meal image processing apparatus, reference may be made to the steps S601 to S604, and the contents of the steps S101 to S104, which will not be described in detail herein.
Based on the foregoing, the present application further provides a computer storage medium for storing a computer program;
the program performs the steps as referred to in the image processing method described above, or performs the steps as referred to in the meal image processing method described above.
Based on the foregoing, the present application further provides an electronic device, as shown in fig. 8, fig. 8 is a schematic structural diagram of the electronic device provided in the present application, where the electronic device may include: a processor 801 and a memory 802.
The memory 802 is used for storing a computer program that performs steps as referred to in the above-described image processing method or performs steps as referred to in the above-described meal image processing method.
It should be noted that, the user information (including but not limited to user equipment information, user personal information, etc.) and the data (including but not limited to data for analysis, stored data, presented data, etc.) related to the present application are information and data authorized by the user or fully authorized by each party, and the collection, use and processing of the related data need to comply with the related laws and regulations and standards of the related country and region, and provide corresponding operation entries for the user to select authorization or rejection.
In one typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include volatile memory in a computer-readable medium, random Access Memory (RAM) and/or nonvolatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of computer-readable media.
1. Computer readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device. Computer readable media, as defined herein, does not include non-transitory computer readable media (transmission media), such as modulated data signals and carrier waves.
2. It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
While the preferred embodiment has been described, it is not intended to limit the invention thereto, and any person skilled in the art may make variations and modifications without departing from the spirit and scope of the present invention, so that the scope of the present invention shall be defined by the claims of the present application.
Claims (10)
1. An image processing method, comprising:
according to the extraction operation of the main body image in the target image, determining the extracted main body image as a first image;
performing a repair process on an image having a missing region formed in the target image based on the extraction operation, determining the repair-processed image as a second image; wherein the edge of the missing region corresponds to the contour edge of the subject image;
performing image dynamic processing on the first image according to the set dynamic transformation period, and determining the processed sequence frame image as a dynamic sequence frame image which is dynamically transformed in the dynamic transformation period;
and determining the image after the fusion processing of the dynamic sequence frame image and the second image as a dynamic composite image.
2. The image processing method according to claim 1, wherein the determining the extracted subject image as the first image according to the extracting operation of the subject image in the target image includes:
determining coordinate information of the subject image in the target image in response to a selection operation of the subject image in the target image;
determining a coordinate feature vector according to the coding of the coordinate information;
taking the coordinate vector features and the image feature vectors of the target image as extraction requirement information, and inputting the extraction requirement information into a large model;
and determining an image output by the large model as the first image.
3. The image processing method according to claim 1, wherein the performing a repair process on an image having a missing region formed in the target image based on the extraction operation, determining the repair-processed image as a second image, includes:
determining a binarized image according to binarization processing of an image including the missing region;
determining an image to be repaired according to channel superposition of the binarized image and the target image;
and inputting the image to be repaired into a convolutional neural network for learning, and acquiring the second image.
4. The image processing method according to claim 1, wherein the performing image dynamic processing on the first image according to a set cycle period, determining the processed sequence frame image as a dynamic sequence frame image that is dynamically transformed in the cycle period, comprises:
determining a target transformation mode corresponding to the attribute in the transformation template library according to the attribute of the main image;
according to the target transformation mode, determining a frame image which carries out image dynamic transformation processing on the first image as the dynamic frame image;
sequencing the dynamic frame images according to a dynamic playing order to obtain the sequence frame images;
and determining the sequence frame images played according to the cycle period as the dynamic sequence frame images.
5. The image processing method according to claim 4, wherein the determining, as the dynamic frame image, the frame image subjected to the image dynamic transformation processing on the first image according to the target transformation scheme, includes:
when the target transformation mode is out-of-frame transformation, centering the first image in a transparent base map, and setting the image size of the transparent base map to be larger than that of the first image;
carrying out affine transformation on the first image from the center of the transparent base image to the edge in sequence to obtain the dynamic frame image;
or,
when the target transformation mode is twisting transformation, a plurality of transformation sequence frame images in a preset period are obtained through transformation adjustment in the height direction and/or the width direction of the first image;
the plurality of transform sequence image frames is determined as the dynamic frame image.
6. The image processing method according to claim 1, wherein the determining the fusion-processed image of the dynamic sequence frame image and the second image as a dynamic composite image includes:
and respectively carrying out transparency mixing on the dynamic sequence frame images and the second image to obtain the dynamic synthetic image.
7. The image processing method according to claim 6, wherein the transparency mixing the dynamic sequence frame images with the second images, respectively, to obtain the dynamic composite image, comprises:
determining a first alpha channel matrix of the dynamic sequence frame image;
determining a second alpha channel matrix of the second image;
and acquiring the dynamic synthesized image according to the first alpha channel matrix and the second alpha channel matrix.
8. A method for processing an image of a meal, comprising:
according to the extraction operation of the main meal image in the target meal image, determining the extracted main meal image as a first image;
performing repair processing on an image having a missing region formed in the target meal image based on the extraction operation, and determining the repair processed image as a second image; wherein the edge of the missing region corresponds to the contour edge of the main meal image;
performing image dynamic processing on the first image according to the set dynamic transformation period, and determining the processed sequence frame image as a dynamic sequence frame image which is dynamically transformed in the dynamic transformation period;
and determining the image after the fusion processing of the dynamic sequence frame image and the second image as a dynamic synthesized meal image.
9. A computer storage medium storing a computer program;
the program performs the image processing method according to any one of the preceding claims 1 to 7, or performs the meal image processing method according to claim 8.
10. An electronic device, comprising:
a processor;
a memory for storing a computer program that performs the image processing method according to any one of the preceding claims 1 to 7 or performs the meal image processing method according to claim 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311628974.XA CN117333586A (en) | 2023-11-30 | 2023-11-30 | Image processing method and device, storage medium and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311628974.XA CN117333586A (en) | 2023-11-30 | 2023-11-30 | Image processing method and device, storage medium and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117333586A true CN117333586A (en) | 2024-01-02 |
Family
ID=89279685
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311628974.XA Pending CN117333586A (en) | 2023-11-30 | 2023-11-30 | Image processing method and device, storage medium and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117333586A (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109003282A (en) * | 2018-07-27 | 2018-12-14 | 京东方科技集团股份有限公司 | A kind of method, apparatus and computer storage medium of image procossing |
| CN110324663A (en) * | 2019-07-01 | 2019-10-11 | 北京奇艺世纪科技有限公司 | A kind of generation method of dynamic image, device, electronic equipment and storage medium |
| CN112689168A (en) * | 2020-12-09 | 2021-04-20 | 四川金熊猫新媒体有限公司 | Dynamic effect processing method, dynamic effect display method and dynamic effect processing device |
| CN114416260A (en) * | 2022-01-20 | 2022-04-29 | 北京字跳网络技术有限公司 | Image processing method, device, electronic device and storage medium |
| US20230023585A1 (en) * | 2020-11-02 | 2023-01-26 | Tencent Technology (Shenzhen) Company Limited | Artificial intelligence-based image processing method and apparatus, computer device and storage medium |
| CN116579963A (en) * | 2023-05-19 | 2023-08-11 | 上海芯赛云计算科技有限公司 | System and method for generating dynamic image by using static image |
| CN116958317A (en) * | 2022-03-29 | 2023-10-27 | 山东师范大学 | An image repair method and system that combines edge information and appearance flow operations |
| CN116958286A (en) * | 2023-07-25 | 2023-10-27 | 中南大学湘雅医院 | Image reconstruction method, device, electronic equipment and storage medium |
| CN117078809A (en) * | 2023-08-24 | 2023-11-17 | 蚂蚁区块链科技(上海)有限公司 | Dynamic effect generation method, device, equipment and storage medium based on image |
-
2023
- 2023-11-30 CN CN202311628974.XA patent/CN117333586A/en active Pending
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109003282A (en) * | 2018-07-27 | 2018-12-14 | 京东方科技集团股份有限公司 | A kind of method, apparatus and computer storage medium of image procossing |
| CN110324663A (en) * | 2019-07-01 | 2019-10-11 | 北京奇艺世纪科技有限公司 | A kind of generation method of dynamic image, device, electronic equipment and storage medium |
| US20230023585A1 (en) * | 2020-11-02 | 2023-01-26 | Tencent Technology (Shenzhen) Company Limited | Artificial intelligence-based image processing method and apparatus, computer device and storage medium |
| CN112689168A (en) * | 2020-12-09 | 2021-04-20 | 四川金熊猫新媒体有限公司 | Dynamic effect processing method, dynamic effect display method and dynamic effect processing device |
| CN114416260A (en) * | 2022-01-20 | 2022-04-29 | 北京字跳网络技术有限公司 | Image processing method, device, electronic device and storage medium |
| CN116958317A (en) * | 2022-03-29 | 2023-10-27 | 山东师范大学 | An image repair method and system that combines edge information and appearance flow operations |
| CN116579963A (en) * | 2023-05-19 | 2023-08-11 | 上海芯赛云计算科技有限公司 | System and method for generating dynamic image by using static image |
| CN116958286A (en) * | 2023-07-25 | 2023-10-27 | 中南大学湘雅医院 | Image reconstruction method, device, electronic equipment and storage medium |
| CN117078809A (en) * | 2023-08-24 | 2023-11-17 | 蚂蚁区块链科技(上海)有限公司 | Dynamic effect generation method, device, equipment and storage medium based on image |
Non-Patent Citations (3)
| Title |
|---|
| LI ZHANG ET AL: "Dynamic graph Message Passing Networks for Visual Recognition", 《COMPUTER AND PATTERN RECOGNITION》, 31 December 2022 (2022-12-31) * |
| 曹姝清等: "空间目标快速轮廓特征提取与跟踪技术", 《飞控与探测》, 25 September 2018 (2018-09-25) * |
| 杨文博: "动态场景下基于背景补偿的目标检测技术研究", 《中国优秀硕士学位论文全文数据库电子期刊 信息科技辑》, vol. 2022, no. 4, 15 April 2022 (2022-04-15) * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8947422B2 (en) | Gradient modeling toolkit for sculpting stereoscopic depth models for converting 2-D images into stereoscopic 3-D images | |
| US8121408B2 (en) | Image processing method and apparatus for face image | |
| US10885718B2 (en) | Methods and systems for representing a pre-modeled object within virtual reality data | |
| CN114025219B (en) | Rendering method, device, medium and equipment for augmented reality special effects | |
| Shao et al. | QoE-guided warping for stereoscopic image retargeting | |
| Zhou et al. | Weakly supervised reinforced multi-operator image retargeting | |
| CN110751649A (en) | Video quality evaluation method and device, electronic equipment and storage medium | |
| Rodriguez-Pardo et al. | Seamlessgan: Self-supervised synthesis of tileable texture maps | |
| CN112712487A (en) | Scene video fusion method and system, electronic equipment and storage medium | |
| CN118570054B (en) | Training method, related device and medium for image generation model | |
| Niu et al. | Image retargeting quality assessment based on registration confidence measure and noticeability-based pooling | |
| CN114972847A (en) | Image processing method and device | |
| WO2024222821A1 (en) | Three-dimensional modeling system and method based on hand-drawn sketch, intelligent association modeling method, sketch model editing method, and related device | |
| Tous | Pictonaut: movie cartoonization using 3D human pose estimation and GANs | |
| CN118172292B (en) | Image restoration method, device, equipment and storage medium | |
| US20070076978A1 (en) | Moving image generating apparatus, moving image generating method and program therefor | |
| CN117333586A (en) | Image processing method and device, storage medium and electronic equipment | |
| EP3846136A1 (en) | Augmenting a video flux of a real scene | |
| US20230353702A1 (en) | Processing device, system and method for board writing display | |
| CN117576292A (en) | Three-dimensional scene rendering method and device, electronic equipment and storage medium | |
| Lee et al. | An image-guided network for depth edge enhancement | |
| CN116980549A (en) | Video frame processing method, device, computer equipment and storage medium | |
| Jiang et al. | Deep fusion local-content and global-semantic for image inpainting | |
| CN117152805A (en) | Image processing method and device, computing device, storage medium and program product | |
| CN114387315A (en) | Image processing model training, image processing method, device, equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |