CN112070671B - Mosaic removing method, system, terminal and storage medium based on spectrum analysis - Google Patents
Mosaic removing method, system, terminal and storage medium based on spectrum analysis Download PDFInfo
- Publication number
- CN112070671B CN112070671B CN202010921922.1A CN202010921922A CN112070671B CN 112070671 B CN112070671 B CN 112070671B CN 202010921922 A CN202010921922 A CN 202010921922A CN 112070671 B CN112070671 B CN 112070671B
- Authority
- CN
- China
- Prior art keywords
- mosaic
- image
- peak
- area
- spectrum analysis
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000010183 spectrum analysis Methods 0.000 title claims abstract description 66
- 238000000034 method Methods 0.000 title claims abstract description 59
- 238000001514 detection method Methods 0.000 claims abstract description 33
- 238000012216 screening Methods 0.000 claims abstract description 24
- 238000010586 diagram Methods 0.000 claims description 24
- 230000001131 transforming effect Effects 0.000 claims description 3
- 238000001228 spectrum Methods 0.000 abstract description 9
- 238000010191 image analysis Methods 0.000 abstract description 6
- 230000009466 transformation Effects 0.000 abstract description 2
- 238000012545 processing Methods 0.000 description 6
- 238000003708 edge detection Methods 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000000638 solvent extraction Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 239000011521 glass Substances 0.000 description 1
- 210000003128 head Anatomy 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4015—Image demosaicing, e.g. colour filter arrays [CFA] or Bayer patterns
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/60—Rotation of whole images or parts thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20048—Transform domain processing
- G06T2207/20056—Discrete and fast Fourier transform, [DFT, FFT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20112—Image segmentation details
- G06T2207/20132—Image cropping
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
- G06T2207/30201—Face
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Geometry (AREA)
- Image Analysis (AREA)
Abstract
The application discloses a mosaic removing method, a mosaic removing system, a mosaic removing terminal and a mosaic removing storage medium based on spectrum analysis. The method comprises the following steps: performing Fourier transform on an original mosaic image to obtain a spectrogram of the mosaic image; respectively carrying out spectrum analysis on the spectrograms, carrying out peak point detection on the spectrum analysis result by using a peak detection method, screening out peak points meeting preset conditions, and positioning a mosaic area in the original mosaic image according to coordinates of the screened peak points; and cutting the mosaic area according to the positioning to obtain an image with the mosaic area removed. According to the embodiment of the application, the frequency spectrum difference between the mosaic image and other areas is detected by utilizing Fourier transformation to carry out frequency spectrum analysis on the mosaic image, so that the mosaic area in the image is automatically, quickly and accurately positioned and removed, the introduction of noise is avoided, and the accuracy of image analysis is improved.
Description
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to a method, a system, a terminal, and a storage medium for removing mosaics based on spectrum analysis.
Background
In the internet era, in order to show the privacy of information on pictures or videos, some specific parts need to be demosaiced; however, large-area mosaics also bring noise pollution, and influence the accuracy of technologies such as image analysis and the like. Taking the face video heart rate estimation as an example, because of personal privacy problems, glasses and mouth parts in a face image are usually demosaiced, but because the face video heart rate estimation needs to be subjected to signal detection based on tiny variation of the skin of the face, the large-area mosaic brings great challenges to the face video heart rate estimation. Demosaicing therefore appears to be critical in image analysis tasks.
Most researches on mosaic removal in the prior art use a generated countermeasure network to restore the mosaics in the pictures, for example PLUSE (Photo Upsamp L I NG V I A LATENT SPACE Exp l orat i on, generated model latent space exploration self-supervision photo upsampling) method, but the face restored by using a PLUSE method cannot improve the model precision, but introduces larger noise.
Disclosure of Invention
The invention provides a mosaic removing method, a mosaic removing system, a mosaic removing terminal and a mosaic removing storage medium based on spectrum analysis, which can solve the defects in the prior art to a certain extent.
In order to solve the technical problems, the invention adopts the following technical scheme:
A mosaic removing method based on spectrum analysis comprises the following steps:
performing Fourier transform on the mosaic image to obtain a spectrogram of the mosaic image;
Performing spectrum analysis on the spectrograms respectively, and performing peak point detection on the spectrum analysis result by using a peak detection method, and screening out peak points meeting preset conditions, wherein the preset conditions are as follows: the vertical axis of the peak point is larger than a first preset value, and the abscissa interval value of the peak point is larger than a second preset value;
positioning a mosaic area in the mosaic image according to the coordinates of the screened peak points;
and cutting the mosaic area according to the positioning to obtain an image with the mosaic area removed.
The technical scheme adopted by the embodiment of the invention further comprises the following steps: the method for carrying out Fourier transform on the mosaic image further comprises the following steps:
And detecting edge pixel points of the mosaic image by utilizing sobe l operators to obtain a boundary image of the mosaic image.
The technical scheme adopted by the embodiment of the invention further comprises the following steps: the fourier transforming the mosaic image includes:
And grouping the pixel values in the boundary image according to the rows, and respectively carrying out Fourier transform on each group of pixel values to obtain a spectrogram corresponding to each group of pixel values.
The technical scheme adopted by the embodiment of the invention further comprises the following steps: the positioning the mosaic area in the mosaic image according to the position of the screened peak point comprises the following steps:
calculating according to the set high-low frequency critical point value to obtain the frequency ratio of the low frequency to the high frequency of each spectrogram, and obtaining a high-low frequency ratio chart of each spectrogram;
Performing peak point detection on the high-low frequency ratio graph by using a peak detection method to obtain a peak point graph;
Screening all peak points with longitudinal axes larger than a first preset value from the peak point diagram, and recording the abscissa where the peak points are located;
the first preset value is set according to the position and the size of the mosaic area;
Sorting the abscissa of the peak points, counting the interval value of the abscissa of the peak points, screening out the peak points with the interval value larger than a second preset value, and recording the ordinate of the peak points to obtain the ordinate position information of the mosaic region in the mosaic image;
The second preset value is set according to the length of the mosaic area.
The technical scheme adopted by the embodiment of the invention further comprises the following steps: the positioning the mosaic area in the mosaic image according to the coordinates of the screened peak point further comprises:
and rotating the mosaic image by 90 degrees, and carrying out ordinate positioning on the mosaic region on the rotated mosaic image again, wherein the ordinate positioning result obtained after rotation is the abscissa of the mosaic region in the mosaic image.
The technical scheme adopted by the embodiment of the invention further comprises the following steps: the number of the mosaic areas is at least one.
The technical scheme adopted by the embodiment of the invention further comprises the following steps: when the number of the mosaic areas is two or more, the rotating the mosaic image by 90 ° and repositioning the rotated mosaic image on the ordinate of the mosaic area further includes:
Dividing the mosaic image into two or more images according to the position of the mosaic region, and respectively rotating the two or more images by 90 degrees and performing ordinate positioning operation.
The embodiment of the invention adopts another technical scheme that: a spectral analysis-based mosaic removal system, comprising:
and a Fourier transform module: the method comprises the steps of carrying out Fourier transform on a mosaic image to obtain a spectrogram of the mosaic image;
mosaic positioning module: the method comprises the steps of respectively carrying out spectrum analysis on the spectrograms, carrying out peak point detection on the spectrum analysis result by using a peak detection method, screening out peak points meeting preset conditions, and positioning a mosaic area in the mosaic image according to coordinates of the screened peak points;
Wherein, the preset conditions are as follows:
The vertical axis of the peak point is larger than a first preset value, and the abscissa interval value of the peak point is larger than a second preset value;
mosaic clipping module: and cutting the mosaic area according to the positioning to obtain an image with the mosaic area removed.
The embodiment of the invention adopts the following technical scheme: a terminal comprising a processor, a memory coupled to the processor, wherein,
The memory stores program instructions for implementing the above-described spectrum analysis-based mosaic removal method;
The processor is configured to execute the program instructions stored by the memory to perform the spectral analysis-based mosaic removal operation.
The embodiment of the invention adopts the following technical scheme: a storage medium storing program instructions executable by a processor for performing the above-described spectral analysis-based mosaic removal method.
The beneficial effects of the application are as follows: according to the embodiment of the application, the spectrum difference between the mosaic image and other areas is detected by utilizing the Fourier transform to carry out spectrum analysis on the mosaic image, so that the mosaic area in the image is automatically, quickly and accurately positioned and removed, noise is avoided from being introduced, the accuracy of image analysis is improved, and the method and the device are well embodied in high efficiency and practicability.
Drawings
Fig. 1 is a flowchart of a mosaic removal method based on spectrum analysis according to a first embodiment of the present invention;
Fig. 2 is a flow chart of a mosaic removal method based on spectrum analysis according to a second embodiment of the present invention;
FIG. 3 is a schematic view of a frame of face image taken from a video;
fig. 4 is a flowchart of a mosaic removal method based on spectrum analysis according to a third embodiment of the present invention;
FIG. 5 is a schematic diagram of a face boundary image detected by an embodiment of the present application;
FIG. 6 is a spectrum diagram obtained by performing Fourier transform on a face boundary image according to an embodiment of the present invention; wherein, (a), (b), (c) and (d) are spectrograms of 180 th, 184 th, 200 th and 294 th row section lines in the face boundary image respectively;
FIG. 7 is a graph of a ratio of high frequency to low frequency calculated based on a spectrogram in an embodiment of the present invention;
FIG. 8 is a diagram of the ratio of high frequency to low frequency detected based on the diagram of the ratio of high frequency to low frequency;
fig. 9 is a face image after removing a mosaic area according to a third embodiment of the present invention;
Fig. 10 is a flowchart of a mosaic removal method based on spectrum analysis according to a fourth embodiment of the present invention;
fig. 11 is a face image after removing a mosaic area according to a fourth embodiment of the present invention;
fig. 12 is a schematic structural diagram of a mosaic removal system based on spectrum analysis according to an embodiment of the present invention;
Fig. 13 is a schematic diagram of a terminal structure according to an embodiment of the present invention;
Fig. 14 is a schematic diagram of a storage medium structure according to an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are only some, but not all embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The terms "first," "second," "third," and the like in this disclosure are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first", "a second", and "a third" may explicitly or implicitly include at least one such feature. In the description of the present invention, the meaning of "plurality" means at least two, for example, two, three, etc., unless specifically defined otherwise. All directional indications (such as up, down, left, right, front, back … …) in the embodiments of the present invention are merely used to explain the relative positional relationship, movement, etc. between the components in a particular gesture (as shown in the drawings), and if the particular gesture changes, the directional indication changes accordingly. Furthermore, the terms "comprise" and "have," as well as any variations thereof, are intended to cover a non-exclusive inclusion. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those listed steps or elements but may include other steps or elements not listed or inherent to such process, method, article, or apparatus.
Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the invention. The appearances of such phrases in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Those of skill in the art will explicitly and implicitly appreciate that the embodiments described herein may be combined with other embodiments.
Fig. 1 is a flowchart of a mosaic removal method based on spectrum analysis according to a first embodiment of the present invention. The mosaic removing method based on spectrum analysis of the first embodiment of the invention comprises the following steps:
s10: performing Fourier transform on the mosaic image to obtain a spectrogram of the mosaic image;
the fourier transform of the mosaic image specifically includes:
Performing edge pixel point detection on the mosaic image by using sobe l operators to obtain a boundary image of the mosaic image;
And grouping the pixel values in the boundary image according to the rows, and respectively carrying out Fourier transform on each group of pixel values to obtain a spectrogram corresponding to each group of pixel values.
S11, respectively carrying out spectrum analysis on the spectrograms, carrying out peak point detection on the spectrum analysis result by using a peak detection method, and screening out peak points meeting preset conditions;
The spectrum of the area without mosaic is relatively smooth, and the spectrum of the mosaic area is relatively oscillating, so that the spectrum analysis is performed based on the rule. The preset conditions are as follows: the vertical axis of the peak point is greater than a first preset value, and the abscissa interval value of the peak point is greater than a second preset value.
S12: positioning a mosaic area in the mosaic image according to the coordinates of the screened peak points;
the mosaic area positioning specifically comprises the following steps:
calculating according to the set high-low frequency critical point value to obtain the frequency ratio of the low frequency to the high frequency of each spectrogram, and obtaining a high-low frequency ratio chart of each spectrogram;
Performing peak point detection on the high-low frequency ratio graph by using a peak detection method to obtain a peak point graph;
Screening all peak points with longitudinal axes larger than a first preset value from the peak point diagram, and recording the abscissa where the peak points are located;
sorting the abscissa of the peak points, counting the interval value of the abscissa of the peak points, screening out the peak points with the interval value larger than a second preset value, and recording the ordinate of the peak points to obtain the coordinate position information of the mosaic area in the mosaic image.
S13: and cutting the mosaic area according to the positioning to obtain an image with the mosaic area removed.
Based on the above, in the first embodiment of the present application, the spectrum difference between the mosaic image and other areas is detected by performing the spectrum analysis on the mosaic image using the fourier transform, so that the mosaic area in the image is automatically, quickly and accurately positioned, and the mosaic area obtained by positioning in the image is removed, thereby avoiding the introduction of noise, being beneficial to improving the accuracy of image analysis, and being well reflected in terms of efficiency and practicality.
Further, please refer to fig. 2, which is a flowchart illustrating a mosaic removing method based on spectrum analysis according to a second embodiment of the present invention. The mosaic removing method based on spectrum analysis according to the second embodiment of the present invention comprises the following steps:
s20: reading a mosaic image;
S21: performing edge pixel point detection on the mosaic image by utilizing sobe l operators to obtain a boundary image of the mosaic image;
In this step, sobe l operators are discrete difference operators, which are used to calculate the approximate gradient of the brightness value of the image pixel point. The edge detection algorithm specifically comprises the following steps:
the mosaic image is recorded as I, and the operators of sobe l operators along the horizontal direction and the vertical direction are respectively:
The boundary image obtained after edge detection of the mosaic image by utilizing sobe l operators is:
S22: grouping pixel values in the boundary image according to rows, and respectively carrying out Fourier transform on each group of pixel values to obtain a spectrogram corresponding to each group of pixel values;
in this step, the pixel value grouping method is as follows: the grouping is performed by the height value of the image, and the width value of the image is used as the pixel value data of each group.
S23: respectively carrying out frequency spectrum analysis on each spectrogram, and calculating to obtain the frequency ratio of the low frequency to the high frequency of each spectrogram according to the set high-low frequency critical point value to obtain a high-low frequency ratio chart of each spectrogram;
In this step, the spectrum of the area without mosaic is relatively smooth, and the spectrum of the mosaic area is relatively oscillating, so the application performs spectrum analysis based on the rule. The high and low frequency threshold values may be set based on empirical statistics.
S24: respectively carrying out peak point detection on the high-low frequency ratio graphs of each spectrogram by using a peak detection method to obtain a peak point diagram, screening all peak points with longitudinal axes larger than a first preset value from the peak point diagram, and recording the abscissa of the screened peak points;
In this step, the first preset value may be set according to the position and the size of the mosaic area in the mosaic image.
S25: sorting the abscissa of the screened peak points, screening the peak points with the abscissa interval value larger than a second preset value, and recording the ordinate of the screened peak points to obtain the ordinate position information of the mosaic area in the mosaic image;
In this step, the peak point screening method is as follows: counting the abscissa interval value of the peak points obtained by screening in each peak point diagram, and if the abscissa interval value of one peak point and the other peak point in one peak point diagram is larger than a second preset value, considering the peak point as the initial position of the mosaic region in the mosaic image, wherein the ordinate corresponding to the peak point is the ordinate position information of the initial position of the mosaic region; and the like, until screening of all peak point diagrams is completed, the ordinate position information of the whole mosaic area in the mosaic image can be obtained. The second preset value can be set according to the length or the size of the mosaic area.
S26: rotating the mosaic image by 90 degrees, and re-executing S21 to S25, and positioning the ordinate of the mosaic area on the rotated mosaic image again;
In this step, it can be understood that the ordinate positioning result of the mosaic area obtained after rotation is the abscissa of the mosaic area in the mosaic image.
S27: cutting the mosaic area according to the two ordinate positioning results to obtain an image with the mosaic area removed;
based on the above, the second embodiment of the present application performs edge detection on the mosaic image, and then performs spectrum analysis on the edge detection image by using fourier transform to detect spectrum differences between the mosaic area and other areas, thereby performing automatic, rapid and accurate positioning on the mosaic area in the image, and removing the mosaic area obtained by positioning in the image, thereby avoiding noise introduction, being beneficial to improving accuracy of image analysis, and being well embodied in terms of efficiency and practicality.
In order to more clearly illustrate the implementation of the present application, the following embodiments specifically describe the application of the present application to the removal of an eye mosaic in a face image. As shown in fig. 3, in order to extract a frame of face image from a video, it can be clearly seen that eyes in the face image are covered by a mosaic, and color transformation of the mosaic area is consistent with color variation of key parts of the face, so that the color difference of the mosaic in the image cannot be directly utilized to remove the color variation. As can be seen intuitively from the figure, the mosaic area is a large rectangular block formed by splicing a plurality of small rectangular color blocks, so that the mosaic area can be positioned only by finding the difference between the mosaic area and other areas in the face image, and the mosaic area is removed.
Specifically, please refer to fig. 4, which is a flowchart illustrating a mosaic removal method based on spectrum analysis according to a third embodiment of the present invention. The mosaic removing method based on spectrum analysis according to the third embodiment of the present invention comprises the following steps:
S30: reading a face image containing a mosaic area;
s31: performing edge pixel point detection on the face image by utilizing sobe l operators to obtain a face boundary image;
Taking the face head portrait shown in fig. 2 as an example, the detected face boundary image is shown in fig. 5.
S32: grouping pixel values in the face boundary image according to rows, and performing Fourier transform on each group of pixel values respectively to obtain a spectrogram corresponding to each group of pixel values;
The face boundary image shown in fig. 5 is assumed to be an image of high 473 and wide 373, and is divided into 473 sets of pixel data, each set of pixel data includes 373 pixel values, then fourier transform is performed on each set of pixel data, and the obtained spectrograms are shown in fig. 6, (a), (b), (c), and (d) are spectrograms of 180 th, 184 th, 200 th, and 294 th lines in fig. 5, respectively, the horizontal axis represents frequency, and the vertical axis represents amplitude.
S33: carrying out spectrum analysis on the spectrograms, and calculating to obtain the frequency ratio of the low frequency to the high frequency of each spectrogram according to the set high-low frequency critical point value to obtain a high-low frequency ratio chart of each spectrogram;
As can be seen from fig. 6, most of the information is concentrated in the low frequency part in the spectrogram after fourier transform, as shown in lines 180 and 200 in (a) and (c) of fig. 6; the mosaic is located in the section line with a wider frequency distribution, such as lines 184 and 200 shown in fig. 6 (b) and (d). In the embodiment of the application, the high-low frequency critical point value is set to be 15 according to the experience statistical value of the face image, and the high-low frequency ratio chart calculated according to the high-low frequency critical point value is shown in fig. 7.
S34: respectively carrying out peak point detection on the high-low frequency ratio graphs of each spectrogram by using a peak detection method to obtain a peak point diagram, screening all peak points with longitudinal axes larger than a first preset value from the peak point diagram, and recording the abscissa of the screened peak points;
The peak point diagram obtained by peak point detection is shown in fig. 8. According to the priori value counted by all face image data sets, mosaics do not appear in the front 80 rows in the face images, so that the peak points with the horizontal axis smaller than 80 in the high-low frequency ratio chart are removed, and then screening of the peak points is carried out. Preferably, the first preset value is set to 0.5.
S35: sorting the abscissa of the screened peak points, screening out peak points with the abscissa interval value larger than a second preset value, and recording the ordinate of the screened peak points to obtain the ordinate position information of the eye mosaics in the mosaic image;
wherein the second preset value is preferably set to 25.
S36: rotating the face image by 90 degrees, and re-executing S31 to S35, and performing ordinate positioning of the eye mosaic area on the rotated face image again;
It can be understood that the ordinate positioning result of the eye mosaic area obtained after rotation is the abscissa of the mosaic area in the original face image.
S37: cutting the eye mosaic area according to the two ordinate positioning results to obtain a face image with the eye mosaic area removed;
the face image after the mosaic area is removed is shown in fig. 9.
It can be appreciated that the embodiment of the application can be applied to various types of mosaic images, and can position and remove a plurality of mosaic areas in one image. The following embodiment specifically describes an example of removing an eye mosaic and a mouth mosaic in a face image shown in fig. 2.
Fig. 10 is a flowchart of a mosaic removal method based on spectrum analysis according to a fourth embodiment of the present invention. The mosaic removing method based on spectrum analysis according to the fourth embodiment of the present invention comprises the following steps:
S40: reading a face image containing a mosaic area;
S41: performing edge pixel point detection on the face image by utilizing sobe l operators to obtain a face boundary image;
wherein the detected face boundary image is shown in fig. 5.
S42: grouping pixel values in the face boundary image according to rows, and performing Fourier transform on each group of pixel values respectively to obtain a spectrogram corresponding to each group of pixel values;
The face boundary image shown in fig. 5 is assumed to be an image of high 473 and wide 373, and is divided into 473 sets of pixel data, each set of pixel data includes 373 pixel values, then fourier transform is performed on each set of pixel data, and the obtained spectrograms are shown in fig. 6, (a), (b), (c), and (d) are spectrograms of 180 th, 184 th, 200 th, and 294 th lines in fig. 5, respectively, the horizontal axis represents frequency, and the vertical axis represents amplitude.
S33: carrying out spectrum analysis on the spectrograms, and calculating to obtain the frequency ratio of the low frequency to the high frequency of each spectrogram according to the set high-low frequency critical point value to obtain a high-low frequency ratio chart of each spectrogram;
As can be seen from fig. 6, most of the information is concentrated in the low frequency part in the spectrogram after fourier transform, as shown in lines 180 and 200 in (a) and (c) of fig. 6; the mosaic is located in the section line with a wider frequency distribution, such as lines 184 and 200 shown in fig. 6 (b) and (d). In the embodiment of the application, the high-low frequency critical point value is set to be 15 according to the experience statistical value of the face image, and the high-low frequency ratio chart calculated according to the high-low frequency critical point value is shown in fig. 7.
S44: respectively carrying out peak point detection on the high-low frequency ratio graphs of each spectrogram by using a peak detection method to obtain a peak point diagram, screening all peak points with longitudinal axes larger than a first preset value from the peak point diagram, and recording the abscissa of the screened peak points;
The peak point diagram obtained by peak point detection is shown in fig. 8. According to the priori value counted by all face image data sets, mosaics do not appear in the front 80 rows in the face images, so that the peak points with the horizontal axis smaller than 80 in the high-low frequency ratio chart are removed, and then screening of the peak points is carried out. Preferably, the first preset value is set to 0.5.
S45: sorting the abscissa of the screened peak points, screening out peak points with the abscissa interval value larger than a second preset value, and recording the ordinate of the screened peak points to obtain the ordinate position information of two mosaic areas of the eye and the mouth in the mosaic image respectively;
wherein the second preset value is preferably set to 25.
S46: dividing two mosaic areas of an eye and a mouth in a face image into two images, rotating the two images by 90 degrees respectively, and re-executing S41 to S45 to respectively position the vertical coordinates of the mosaic areas of the eye and the mosaic areas of the mouth on the rotated two images;
It can be understood that the ordinate positioning result of the eye mosaic region and the mouth mosaic region obtained after rotation is the abscissa of the two mosaic regions in the original face image.
S47: cutting the mosaic areas of the eyes and the mouth respectively according to the two ordinate positioning results to obtain a face image with the mosaic areas removed;
the face image after the mosaic area is removed is shown in fig. 11.
In an alternative embodiment, it is also possible to: and uploading the result of the mosaic removing method based on the spectrum analysis to a blockchain.
Specifically, corresponding summary information is obtained based on the result of the mosaic removal method based on spectrum analysis, specifically, the summary information is obtained by performing hash processing on the result of the mosaic removal method based on spectrum analysis, for example, the summary information is obtained by processing by using a sha256s algorithm. Uploading summary information to the blockchain can ensure its security and fair transparency to the user. The user can download the summary information from the blockchain to verify whether the result of the spectral analysis-based mosaic removal method is tampered. The blockchain referred to in this example is a novel mode of application for computer technology such as distributed data storage, point-to-point transmission, consensus mechanisms, encryption algorithms, and the like. The blockchain (B l ockcha i n), essentially a de-centralized database, is a string of data blocks that are generated in association using cryptographic methods, each of which contains information from a batch of network transactions for verifying the validity (anti-counterfeit) of its information and generating the next block. The blockchain may include a blockchain underlying platform, a platform product services layer, an application services layer, and the like.
Fig. 12 is a schematic structural diagram of a mosaic removal system based on spectrum analysis according to an embodiment of the present invention. The mosaic removal system 40 based on spectrum analysis according to the embodiment of the present invention includes:
fourier transform module 41: the method comprises the steps of carrying out Fourier transform on a mosaic image to obtain a spectrogram of the mosaic image;
mosaic positioning module 42: the method comprises the steps of respectively carrying out spectrum analysis on the spectrograms, carrying out peak point detection on the spectrum analysis result by using a peak detection method, screening out peak points meeting preset conditions, and positioning a mosaic area in the mosaic image according to coordinates of the screened peak points;
Wherein, the preset conditions are as follows:
The vertical axis of the peak point is larger than a first preset value, and the abscissa interval value of the peak point is larger than a second preset value;
mosaic clipping module 43: and cutting the mosaic area according to the positioning to obtain an image with the mosaic area removed.
Fig. 13 is a schematic diagram of a terminal structure according to an embodiment of the invention. The terminal 50 includes a processor 51, a memory 52 coupled to the processor 51.
The memory 52 stores program instructions for implementing the above-described spectral analysis-based mosaic removal method.
The processor 51 is configured to execute program instructions stored by the memory 52 to perform a spectral analysis based mosaic removal operation.
The processor 51 may also be referred to as a CPU (Centra l Process i ng Un it ). The processor 51 may be an integrated circuit chip with signal processing capabilities. Processor 51 may also be a general purpose processor, a Digital Signal Processor (DSP), an application specific integrated circuit (asic), an off-the-shelf programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
Referring to fig. 14, fig. 14 is a schematic structural diagram of a storage medium according to an embodiment of the invention. The storage medium of the embodiment of the present invention stores a program file 61 capable of implementing all the methods described above, where the program file 61 may be stored in the storage medium in the form of a software product, and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor (processor) to execute all or part of the steps of the methods of the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, an optical disk, or other various media capable of storing program codes, or a terminal device such as a computer, a server, a mobile phone, a tablet, or the like.
In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices, and methods may be implemented in other manners. For example, the system embodiments described above are merely illustrative, e.g., the partitioning of elements is merely a logical functional partitioning, and there may be additional partitioning in actual implementation, e.g., multiple elements or components may be combined or integrated into another system, or some features may be omitted, or not implemented. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form. In addition, the face data related to the application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the related data need to comply with related laws and regulations and standards of related countries and regions.
In addition, each functional unit in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units. The foregoing is only the embodiments of the present invention, and therefore, the patent scope of the invention is not limited thereto, and all equivalent structures or equivalent processes using the descriptions of the present invention and the accompanying drawings, or direct or indirect application in other related technical fields, are included in the scope of the invention.
Claims (10)
1. A mosaic removal method based on spectral analysis, comprising:
performing Fourier transform on the mosaic image to obtain a spectrogram of the mosaic image;
Performing spectrum analysis on the spectrograms respectively, and performing peak point detection on the spectrum analysis result by using a peak detection method, and screening out peak points meeting preset conditions, wherein the preset conditions are as follows: the vertical axis of the peak point is larger than a first preset value, and the abscissa interval value of the peak point is larger than a second preset value;
positioning a mosaic area in the mosaic image according to the coordinates of the screened peak points;
and cutting the mosaic area according to the positioning to obtain an image with the mosaic area removed.
2. The method for removing mosaic based on spectrum analysis according to claim 1, wherein before fourier transforming the mosaic image, further comprising:
and detecting edge pixel points of the mosaic image by using a sobel operator to obtain a boundary image of the mosaic image.
3. The method for removing mosaic based on spectrum analysis according to claim 2, wherein said fourier transforming the mosaic image comprises:
And grouping the pixel values in the boundary image according to the rows, and respectively carrying out Fourier transform on each group of pixel values to obtain a spectrogram corresponding to each group of pixel values.
4. A method of spectral analysis based mosaic removal according to claim 3, wherein said locating a mosaic area in said mosaic image according to coordinates of said screened peak points comprises:
calculating according to the set high-low frequency critical point value to obtain the frequency ratio of the low frequency to the high frequency of each spectrogram, and obtaining a high-low frequency ratio chart of each spectrogram;
Performing peak point detection on the high-low frequency ratio graph by using a peak detection method to obtain a peak point graph;
Screening all peak points with longitudinal axes larger than a first preset value from the peak point diagram, and recording the abscissa where the peak points are located;
the first preset value is set according to the position and the size of the mosaic area;
Sorting the abscissa of the peak points, counting the interval value of the abscissa of the peak points, screening out the peak points with the interval value larger than a second preset value, and recording the ordinate of the peak points to obtain the ordinate position information of the mosaic region in the mosaic image;
The second preset value is set according to the length of the mosaic area.
5. The spectral analysis-based mosaic removal method according to claim 4, wherein said locating a mosaic area in said mosaic image according to coordinates of said screened peak points further comprises:
and rotating the mosaic image by 90 degrees, and carrying out ordinate positioning on the mosaic region on the rotated mosaic image again, wherein the ordinate positioning result obtained after rotation is the abscissa of the mosaic region in the mosaic image.
6. The method for removing mosaic based on spectral analysis according to claim 5, wherein the number of said mosaic areas is at least one.
7. The method of spectral analysis based mosaic removal according to claim 6, wherein when the number of mosaic areas is two or more, said rotating the mosaic image by 90 ° and repositioning the rotated mosaic image on the ordinate of the mosaic area further comprises:
Dividing the mosaic image into two or more images according to the position of the mosaic region, and respectively rotating the two or more images by 90 degrees and performing ordinate positioning operation.
8. A spectral analysis-based mosaic removal system, comprising:
and a Fourier transform module: the method comprises the steps of carrying out Fourier transform on a mosaic image to obtain a spectrogram of the mosaic image;
mosaic positioning module: the method comprises the steps of respectively carrying out spectrum analysis on the spectrograms, carrying out peak point detection on the spectrum analysis result by using a peak detection method, screening out peak points meeting preset conditions, and positioning a mosaic area in the mosaic image according to coordinates of the screened peak points;
Wherein, the preset conditions are as follows:
The vertical axis of the peak point is larger than a first preset value, and the abscissa interval value of the peak point is larger than a second preset value;
mosaic clipping module: and cutting the mosaic area according to the positioning to obtain an image with the mosaic area removed.
9. A terminal for implementing a mosaic removal method based on spectral analysis, characterized in that the terminal comprises a processor, a memory coupled to the processor, wherein,
The memory stores program instructions for implementing the spectral analysis-based mosaic removal method according to any one of claims 1 to 7;
The processor is configured to execute the program instructions stored by the memory to perform the spectral analysis-based mosaic removal method.
10. A computer readable storage medium, characterized in that a program instruction executable by a processor is stored, said program instruction being for performing the spectral analysis based mosaic removal method according to any one of claims 1 to 7.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010921922.1A CN112070671B (en) | 2020-09-04 | 2020-09-04 | Mosaic removing method, system, terminal and storage medium based on spectrum analysis |
| PCT/CN2020/124814 WO2021143272A1 (en) | 2020-09-04 | 2020-10-29 | Spectral analysis-based de-mosaicing method, system, terminal, and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010921922.1A CN112070671B (en) | 2020-09-04 | 2020-09-04 | Mosaic removing method, system, terminal and storage medium based on spectrum analysis |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112070671A CN112070671A (en) | 2020-12-11 |
| CN112070671B true CN112070671B (en) | 2024-07-19 |
Family
ID=73665064
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010921922.1A Active CN112070671B (en) | 2020-09-04 | 2020-09-04 | Mosaic removing method, system, terminal and storage medium based on spectrum analysis |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN112070671B (en) |
| WO (1) | WO2021143272A1 (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116453173B (en) * | 2022-12-16 | 2023-09-08 | 南京奥看信息科技有限公司 | Picture processing method based on picture region segmentation technology |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH02295354A (en) * | 1989-05-10 | 1990-12-06 | Canon Inc | Picture processor |
| CN106372584A (en) * | 2016-08-26 | 2017-02-01 | 浙江银江研究院有限公司 | Video image mosaic detection method |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP3136071B2 (en) * | 1995-04-14 | 2001-02-19 | シャープ株式会社 | Image processing device and imaging device |
| JPH09128529A (en) * | 1995-10-30 | 1997-05-16 | Sony Corp | Removing method for noise in digital image based on projection |
| EP1534016A3 (en) * | 2003-11-21 | 2007-03-07 | Matsushita Electric Industrial Co., Ltd. | Using motion vectors and the occurrence of coefficient quantization levels to drive a noise removal image postprocessing |
| CN104517269A (en) * | 2014-12-25 | 2015-04-15 | 深圳市一体太赫兹科技有限公司 | Terahertz image strip processing method and system |
| CN109410123B (en) * | 2018-10-15 | 2023-08-18 | 深圳市能信安科技股份有限公司 | Deep learning-based mosaic removing method and device and electronic equipment |
| CN110415175B (en) * | 2019-08-07 | 2023-06-02 | 杭州微帧信息科技有限公司 | A Fast Method for Removing Coding Mosaics in Flat Regions |
| CN110517206B (en) * | 2019-09-05 | 2021-10-15 | 锐芯微电子股份有限公司 | Method and device for eliminating color moire |
-
2020
- 2020-09-04 CN CN202010921922.1A patent/CN112070671B/en active Active
- 2020-10-29 WO PCT/CN2020/124814 patent/WO2021143272A1/en active Application Filing
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH02295354A (en) * | 1989-05-10 | 1990-12-06 | Canon Inc | Picture processor |
| CN106372584A (en) * | 2016-08-26 | 2017-02-01 | 浙江银江研究院有限公司 | Video image mosaic detection method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112070671A (en) | 2020-12-11 |
| WO2021143272A1 (en) | 2021-07-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10210415B2 (en) | Method and system for recognizing information on a card | |
| Kakar et al. | Exposing digital image forgeries by detecting discrepancies in motion blur | |
| US10410087B2 (en) | Automated methods and systems for locating document subimages in images to facilitate extraction of information from the located document subimages | |
| CN102279926B (en) | The method and system of the character of identification original character collection | |
| US10621740B2 (en) | 3D modeling method based on point cloud data | |
| US11551388B2 (en) | Image modification using detected symmetry | |
| US10679094B2 (en) | Automatic ruler detection | |
| US10657421B2 (en) | Shape detection | |
| CN117313364A (en) | Digital twin three-dimensional scene construction method and device | |
| CN112070671B (en) | Mosaic removing method, system, terminal and storage medium based on spectrum analysis | |
| CN116884065A (en) | Face recognition method and computer-readable storage medium | |
| CN111402177A (en) | Definition detection method, system, device and medium | |
| CN113393506B (en) | Image registration method and related device and equipment | |
| CN110880023A (en) | Method and device for detecting certificate picture | |
| CN114092925B (en) | Video subtitle detection method, device, terminal equipment and storage medium | |
| CN112487943A (en) | Method and device for removing duplicate of key frame and electronic equipment | |
| US8358803B2 (en) | Navigation using fourier phase technique | |
| CN111599080A (en) | Spliced paper money detection method and device, financial machine tool equipment and storage medium | |
| US11605245B2 (en) | Systems and methods for masking biometric information in images | |
| CN113448470B (en) | Webpage long screenshot method, device, equipment and storage medium | |
| US20230233943A1 (en) | Method and system for processing textual depictions in a computer game screenshot | |
| WO2015162027A2 (en) | Method, device, user equipment and computer program for object extraction from multimedia content | |
| Wang et al. | Spotting the Fakes: A Deep Dive into GAN-Generated Face Detection | |
| CN116483214A (en) | Mouse position detection method, device, terminal and medium | |
| CN114092559A (en) | A method and device for training a panoramic image feature point descriptor generation network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |