[go: up one dir, main page]

CN116248897B - Image processing method, system, electronic device and computer readable storage medium - Google Patents

Image processing method, system, electronic device and computer readable storage medium

Info

Publication number
CN116248897B
CN116248897B CN202310144875.8A CN202310144875A CN116248897B CN 116248897 B CN116248897 B CN 116248897B CN 202310144875 A CN202310144875 A CN 202310144875A CN 116248897 B CN116248897 B CN 116248897B
Authority
CN
China
Prior art keywords
image
jnd
residual
point
sub
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202310144875.8A
Other languages
Chinese (zh)
Other versions
CN116248897A (en
Inventor
高伟
郑慧明
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Peking University Shenzhen Graduate School
Original Assignee
Peking University Shenzhen Graduate School
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Peking University Shenzhen Graduate School filed Critical Peking University Shenzhen Graduate School
Priority to CN202310144875.8A priority Critical patent/CN116248897B/en
Publication of CN116248897A publication Critical patent/CN116248897A/en
Application granted granted Critical
Publication of CN116248897B publication Critical patent/CN116248897B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/124—Quantisation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/90—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using coding techniques not provided for in groups H04N19/10-H04N19/85, e.g. fractals
    • H04N19/91—Entropy coding, e.g. variable length coding [VLC] or arithmetic coding
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

The application discloses an image processing method, an image processing system, electronic equipment and a computer readable storage medium, wherein the method comprises the steps of dividing an original image into a JND point image and a residual image; inputting the JND point image and the residual image into a preset multi-level residual compensation network model, and outputting to obtain potential representation parameters; and sequentially quantizing and entropy coding the potential representation parameters to obtain a target compressed image corresponding to the original image. The application improves the compression effect of image compression.

Description

Image processing method, system, electronic device and computer readable storage medium
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to an image processing method, an image processing system, an electronic device, and a computer readable storage medium.
Background
When image compression is performed, the traditional HEVC-SCC (HIGH EFFICIENCY Video Coding-Screen Content Coding) based mode relies on the characteristics of the screen contents of manual production and the traditional optimization interest rate to minimize the rate distortion loss, but the compression efficiency is severely limited due to the adoption of the manual production operation, so that the compression effect on the image compression is poor.
Disclosure of Invention
The invention mainly aims to provide an image processing method, an image processing system, electronic equipment and a computer readable storage medium, and aims to solve the technical problem of how to improve the compression effect of image compression.
To achieve the above object, the present application provides an image processing method comprising:
dividing an original image into a JND point image and a residual image;
inputting the JND point image and the residual image into a preset multi-level residual compensation network model, and outputting to obtain potential representation parameters;
and sequentially quantizing and entropy coding the potential representation parameters to obtain a target compressed image corresponding to the original image.
Optionally, the multi-level residual compensation network model includes two sub-encoder models sharing the same network structure, and the network model formulas corresponding to the two sub-encoder moduli respectively include:
Wherein x m represents a residual image, x p represents a JND point image, i represents the number of processing times, and Representing a previously processed residual image of a processing input by a multi-level residual compensation network model, saidThe JND point image of the previous processing which represents the processing input of the multistage residual compensation network model, and the ≡indicates the multiplication by element.
Optionally, the step of inputting the JND point image and the residual image into a preset multi-level residual compensation network model and outputting to obtain the potential representation parameters includes:
Inputting the residual image into a sub-encoder model for forward transformation processing to obtain a first transformation processing result;
Forward transforming the JND point image and the first transforming result through another sub-encoder to obtain a second transforming result;
and determining potential representation parameters according to the preset processing times and the second transformation processing result.
Optionally, the step of dividing the original image into the JND point image and the residual image includes:
determining a distorted image corresponding to the original image in the JND data set;
Determining a JND point image based on the original image and the distorted image;
And determining a residual image according to the JND point image and the original image.
Optionally, before the step of determining a distorted image in the JND dataset corresponding to the original image, the method further includes:
Constructing a data table with a corresponding relation between an original image and a distorted image, and taking the data table as a JND data set, wherein the data table comprises a corresponding relation between at least one original image and at least one distorted image, and the distorted image is obtained by compressing the original image.
Optionally, the step of determining a JND point image based on the original image and the distorted image includes:
sequentially comparing a plurality of distorted images with the original image;
And if a distorted image with visual difference with the original image exists in the plurality of distorted images, taking the distorted image with visual difference as a JND point image, wherein the visual difference comprises image difference pixels which can be obviously recognized by human eyes of a user.
Optionally, determining a residual image according to the JND point image and the original image includes:
and carrying out pixel value subtraction on the same position points between the original image and the JND point image, and taking the original image with the pixel values of all the position points subjected to pixel value subtraction as a residual image.
In addition, in order to achieve the above object, the present application also provides an image processing system, which includes a preprocessing module, an analysis transformation module, and a quantization encoding module;
the preprocessing module is used for dividing an original image into a JND point image and a residual image;
the analysis transformation module is used for inputting the JND point image and the residual image into a preset multi-level residual compensation network model, and outputting to obtain potential representation parameters;
and the quantization coding module is used for sequentially quantizing and entropy coding the potential representation parameters to obtain a target compressed image corresponding to the original image.
In addition, in order to achieve the above object, the present application also provides an electronic device including a memory, a processor, and an image processing program stored in the memory and executable on the processor, the image processing program implementing the steps of the image processing method as described above when executed by the processor.
In addition, in order to achieve the above object, the present application also provides a computer-readable storage medium having stored thereon an image processing program which, when executed by a processor, implements the steps of the image processing method as described above.
According to the embodiment of the application, the original image is divided into the JND point image and the residual image through just observing distortion guidance, the mode of manual feature extraction is not relied on, so that the divided JND point image is as close as possible to the original image, in the process of image compression, the JND point image and the residual image are input into a multi-level residual compensation network model together, potential representation parameters are output to obtain, and then the potential representation parameters are quantized and entropy coded to obtain a target compressed image, namely, the JND point image and the residual image are combined to perform transformation processing, so that bits can be allocated in a self-adaptive mode when the target compressed image is generated by image compression according to the residual image, and the compression effect of image compression is improved.
Drawings
FIG. 1 is a flowchart of a first embodiment of an image processing method according to the present application;
FIG. 2 is a flowchart of a second embodiment of an image processing method according to the present application;
FIG. 3 is a schematic block diagram of an image processing system according to the present application;
FIG. 4 is a schematic overall flow chart of the image processing method of the present application;
FIG. 5 is a schematic diagram of a preprocessing module in the image processing method of the present application;
FIG. 6 is a schematic diagram corresponding to a multi-level residual error compensation network model in the image processing method of the present application;
fig. 7 is a schematic device structure diagram of a hardware operating environment related to an image processing method in an embodiment of the present application.
The achievement of the objects, functional features and advantages of the present invention will be further described with reference to the accompanying drawings, in conjunction with the embodiments.
Detailed Description
It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
Because the HEVC-SCC (HIGH EFFICIENCY Video Coding-Screen Content Coding) based mode has low compression efficiency and poor compression effect on image compression, in the embodiment, the end-to-end screen content image compression is guided by adopting just-noticeable distortion, bits can be adaptively allocated by using human eye perception priori information, and the phenomenon of poor compression performance on the screen content image is avoided. And the reconstructed image can more accord with human eye perception rules when the right-to-visible distortion is used for guiding end-to-end screen content image compression, bits can be distributed in a self-adaptive mode by utilizing human eye perception priori information, and compression performance is remarkably improved.
Embodiments of the present application will be further described with reference to the accompanying drawings.
Referring to fig. 1, the present application provides an image processing method, in a first embodiment of the image processing method, applied to an image processing system, including:
Step S10, dividing an original image into a JND point image and a residual image;
step S20, inputting the JND point image and the residual image into a preset multi-level residual compensation network model, and outputting to obtain potential representation parameters;
And step S30, sequentially quantizing and entropy coding the potential representation parameters to obtain a target compressed image corresponding to the original image.
In this embodiment, image compression may be implemented by encoding an image, and when one frame of image is compressed, since the one frame of image is first divided into a plurality of image blocks, each image block or one frame of image is subtracted from a prediction block obtained by prediction in a prediction mode to obtain a residual block, and then transformed and quantized, and encoded by an entropy encoder, to form an encoded bitstream, and transmitted to a decoding end, where the decoding end reconstructs the image using a synthetic transform after decoding. Alternatively, the image compression or image encoding and decoding process may include analyzing a context model of transformation, synthesis transformation, quantization and entropy encoding, on the basis of which the present embodiment adds a just-in-sight distortion-based preprocessing module (JND-ESN) and a multi-level residual compensation structure. It may thus be that the input raw image x is fed to a pre-processing module to eliminate human visual redundancy, and then the output of the pre-processing module is converted into a potential representation y by an analysis transformation module in which a multi-stage residual compensation structure is provided. And then, using additive noise approximation to replace quantization on the potential representation y to obtain compressed data, coding the compressed data through entropy coding to obtain a bit stream, and transmitting the bit stream to a decoding end. After decoding the bit stream at the decoding end, the decoded data is reconstructed into a reconstructed image using a synthetic transform. For example, as shown in fig. 4, an original image is input to a JND-ESN, a residual image and a JND point image are output, the JND point image and the residual image are input to an analysis transformation module, the JND point image and the residual image are processed by a multi-level residual compensation network model in the analysis transformation module, potential representation parameters are output, and then the potential representation parameters are processed by two coding branches, wherein the first branch is processed by a quantization module and an entropy coding module to obtain a bar code stream, and the bar code stream is decoded by an entropy decoder at a decoding end, and entropy estimation is performed to obtain an entropy estimation result. The other branch is processed by the quantization module and the entropy coding module, the processing result and the entropy estimation result are coded together to obtain another bar code stream, and the bar code stream is decoded according to the entropy decoder in the decoding end to obtain a reconstructed image.
Further, for step S10, the original image is divided into a JND point image and a residual image;
In this embodiment, when image compression is required, an original image to be subjected to image compression is acquired first, and then the original image is subjected to segmentation processing to obtain a JND point image and a residual image. The original image may be a frame image, or may be a certain image block after dividing the image, which is not limited herein. The JND point image may be an image generated by inputting an original image to a JND model set in advance for processing. The residual image may be a JND image generated by the original processing module after processing the original image, where the JND image includes energy removed by the preprocessing module.
The minimum perceptible error (JND, just Noticeable Distortion) is used to represent the maximum image distortion that cannot be perceived by human eyes, and represents the tolerance of human eyes to image changes.
In this embodiment, the original image may be subjected to a segmentation process by the preprocessing module to segment the original image into the JND point image and the residual image. Alternatively, the preprocessing module may be set to JND-ESN, and may generate a low-energy image by reducing perceptual redundancy, that is, an image after removing the signal components that are difficult to perceive, as shown in fig. 5, which is a JND-ESN framework of the preprocessing module, including the GT dataset, the original image, the preprocessing module, the JND dot image, and the residual image. And the preprocessing module can enable the image to be as close to the original image as possible, so that the JND-ESN can realize the perception lossless image preprocessing.
And it should be noted that the residual image includes energy removed by the preprocessing module. The higher the pixel value at a position in the residual image, the higher the distortion level at that position. Therefore, in the subsequent process, the compression distortion of the areas with the same code rate has larger influence, and the areas need to be compensated. At this time, pixel value compensation can be performed according to the multi-level residual compensation network model.
For example, if there is an original image x ori that needs to be compressed, the original image x ori may be divided into a predicted JND point image x p and a residual image x m by a preprocessing module. The JND point image x p and the residual image x m are respectively expressed as:
xp＝E(xori);
xm＝xori-xp;
Where E () represents the ESN-JND network process. E (x ori) represents that the original image is network processed by ESN-JND.
Further, for step S20, the JND point image and the residual image are input into a preset multi-level residual compensation network model, and potential representation parameters are obtained through output;
In this embodiment, after the JND point image and the residual image are obtained by the preprocessing module, the JND point image and the residual image may be synthesized to perform analysis transformation processing, so as to obtain the potential representation parameters. Wherein the potential representation parameter may be a compressed hidden vector that the encoder converts the image. The JND point image and the residual image are required to be subjected to network processing through a multi-level residual compensation network model in the process of analysis transformation processing.
Further, the multi-level residual compensation network model includes two sub-encoder models sharing the same network structure, and the network model formulas corresponding to the two sub-encoder modules include:
Wherein x m represents a residual image, x p represents a JND point image, i represents the number of processing times, and Representing a previously processed residual image of a processing input by a multi-level residual compensation network model, saidThe JND point image of the previous processing which represents the processing input of the multistage residual compensation network model, and the ≡indicates the multiplication by element.
For example, as shown in fig. 6, the multi-level residual compensation network model includes two sub-encoder models sharing the same network structure, and a multi-level network architecture may be set in the sub-encoder models, and the number of layers of the network architecture is related to i in the network model formula, that is, i in the network model formula may be set equal to the number of layers of the network architecture. As shown in fig. 6, with a four-tier network architecture, i may be 4. Wherein the first layer network architecture is RBS, the second layer network architecture is RB and RBS, the third layer network architecture is CBAM and RBS, and the fourth layer network architecture is RB and RBS. Where RB includes Conv (convolutional layer), GDN (normalized layer), LRELU (activate function layer), conv, GDN, and LRELU. RBS included Conv WITH STRIDE, GDN, LRELU, conv WITH STRIDE, GDN and LRELU. And RB denotes a residual block, RBs denotes one downsampling on the basis of RB.
That is, in this embodiment, when multiple transformations are required for the residual image and the JND point image, a transformation process may be performed on the pixel points in the residual image to obtain a transformation process result, and when the transformation process is performed on the same pixel point of the JND point image, the transformation process is performed on the same pixel point of the JND point image at this time in combination with the transformation process result obtained when the transformation process is performed on the same pixel point of the JND point image at the previous time, so as to compensate the region with a higher distortion level in the JND point image, and achieve the purpose of adaptively allocating bits according to the residual image.
Further, step S20, inputting the JND point image and the residual image into a preset multi-level residual compensation network model, and outputting to obtain potential representation parameters, includes:
step a, inputting the residual image into a sub-encoder model for forward transformation processing to obtain a first transformation processing result;
b, performing forward transformation processing on the JND point image and the first transformation processing result through another sub-encoder to obtain a second transformation processing result;
And c, determining potential representation parameters according to the preset processing times and the second transformation processing result.
In this embodiment, when performing the analysis conversion processing on the JND point image and the residual image, the number of processing times that the analysis conversion processing needs to be performed may be determined first, and then the analysis conversion processing of the number of processing times may be performed on the JND point image and the residual image. And when analysis transformation processing is performed, the residual image can be input into a sub-encoder to perform first forward transformation processing, so as to obtain a first residual transformation result. And (3) inputting the JND point image to another sub-encoder for performing first forward conversion processing to obtain a first JND conversion result. And when the second transformation is performed, the first residual transformation result is directly subjected to the second forward transformation by a sub-encoder until the number of times of the processing is reached. Wherein, the network model formula corresponding to one sub-encoder can be
When the first JND conversion result is subjected to the second conversion process, the first residual conversion result and the first JND conversion result need to be subjected to element-by-element multiplication, and then the processing result of the element-by-element multiplication and the first JND conversion result are combined by another sub-encoder to perform the second forward conversion process until the processing times are reached. Wherein, the network model formula corresponding to another sub-encoder may be:
And the result of the last transform process of the two sub-encoders may be used as a potential representation parameter.
Further, for step S30, the potential representation parameters are quantized and entropy encoded sequentially, so as to obtain a target compressed image corresponding to the original image.
In this embodiment, after the latent representation parameter is obtained, quantization processing may be performed on the latent representation parameter, for example, using additive noise approximation instead of quantization, and entropy encoding processing may be performed after quantization is completed, so as to achieve a target compressed image corresponding to the original image. I.e. the compression of the original image has been completed at this time.
In this embodiment, the original image is split into the JND point image and the residual image through just noticeable distortion guidance, and the split JND point image is made to be as close as possible to the original image without relying on a manual feature extraction mode, and when the image is compressed, the JND point image and the residual image are input into a multi-level residual compensation network model together, the latent representation parameters are output to obtain, and then quantization and entropy coding are performed on the latent representation parameters to obtain a target compressed image, that is, transformation processing is performed by combining the JND point image and the residual image, so that bits can be allocated adaptively when the target compressed image is generated by performing image compression according to the residual image, and the compression effect of image compression is improved.
Further, on the basis of the first embodiment described above, a second embodiment of the image processing method of the present application is proposed, and referring to fig. 2, in the second embodiment, step S10, a step of dividing an original image into a JND point image and a residual image, includes:
Step d, determining a distorted image corresponding to the original image in the JND data set;
step e, determining a JND point image based on the original image and the distorted image;
and f, determining a residual image according to the JND point image and the original image.
In this embodiment, when the original image is divided into the JND point image and the residual image, all the distorted images in the JND dataset corresponding to the original image in a matching manner may be determined first, then the original image and all the distorted images are input to the preprocessing module, and each distorted image is compared with the original image by the preprocessing module, so as to select a proper distorted image as the JND point image. The residual image is then determined by comparing the original image with the JND dot image. Wherein the original image can be obtained by combining the residual image and the JND dot image.
In this embodiment, the original image is split into the JND point image and the residual image by determining the distorted image corresponding to the original image in the JND dataset, determining the JND point image according to the original image and the distorted image, and determining the residual image according to the JND point image and the original image.
Further, step d, before the step of determining a distorted image in the JND dataset corresponding to the original image, includes:
And step x, constructing a data table with the corresponding relation between an original image and a distorted image, and taking the data table as a JND data set, wherein the data table comprises the corresponding relation between at least one original image and at least one distorted image, and the distorted image is obtained by compressing the original image.
In this embodiment, before the original image is divided into the JND point image and the residual image by the preprocessing module, a JND data set corresponding to the original image needs to be constructed. The method can acquire a plurality of original images in advance, acquire a compressed image corresponding to each original image, and take the compressed image as a distorted image. Wherein, the distortion degree of each distorted image corresponding to the original image is different. At this time, a blank data table can be constructed, all original images are filled into the blank data table, the blank data table is sequentially filled according to the corresponding relation between the original images and the distorted images, the data table with the corresponding relation between the original images and the distorted images is obtained, and the data table is used as a JND data set.
In the present embodiment, by constructing a data table including correspondence relation between original image and distorted image, and taking the data table as the JND dataset, it is possible to facilitate subsequent determination of JND dot images from the JND dataset.
Further, step e, determining a JND point image based on the original image and the distorted image, includes:
Step e1, comparing a plurality of distorted images with the original image in sequence;
And e2, if a distorted image with visual difference with the original image exists in the plurality of distorted images, taking the distorted image with visual difference as a JND point image, wherein the visual difference comprises image difference pixels which can be obviously recognized by human eyes of a user.
In this embodiment, a plurality of distorted images may be compared with an original image in sequence, whether there is a large visual difference between the distorted image and the original image is determined, and when it is determined that there is a distorted image having a large visual difference with the original image, the distorted image may be directly used as a JND point image. The method for judging whether the distorted image with the visual difference from the original image exists in the plurality of distorted images may include at least one of the following:
Firstly, each distorted image and the original image are simultaneously placed in a screen for comparison, and after a comparison consistent instruction or a comparison inconsistent instruction input by a user is detected, the distorted images in the screen are replaced until all the distorted images are compared. And counting all the distorted images corresponding to the inconsistent comparison instruction, and selecting one distorted image closest to the original image from all the distorted images corresponding to the inconsistent comparison instruction as a JND point image.
And secondly, determining the distortion degree of each distorted image relative to the original image, taking the distorted image corresponding to the distortion degree within a preset threshold value range as a target distorted image, and selecting one of the target distorted images as a JND point image.
And thirdly, sequentially placing the distorted images in the screen according to the distortion degree of each distorted image relative to the original image so as to compare with the original image in the screen, and taking the distorted images displayed on the screen as JND point images when the user inputs the inconsistent comparison instruction after detecting the inconsistent comparison instruction input by the user.
In this embodiment, by comparing the plurality of distorted images with the original image in sequence and using the distorted image having a visual difference with the original image as the JND point image, it is ensured that the acquired JND point image is attached to the original image as much as possible, and bits can be adaptively allocated during subsequent image compression according to human eye perception learning.
Further, step f, determining a residual image according to the JND point image and the original image, includes:
And f1, subtracting the pixel values of the same position points between the original image and the JND point image, and taking the original image with the pixel values of all the position points subtracted as a residual image.
In this embodiment, after the JND point image is determined, the original image and the JND point image may be input to the preprocessing module for processing, for example, for each pixel point in the original image, subtraction processing may be performed on the pixel points in the JND point image, and the original image after the pixel values of all the position points are subtracted is used as the residual image. In the residual image, the higher the pixel value is, the higher the distortion level at the position where the pixel value is located is.
In this embodiment, by subtracting all the pixel points in the original image from all the pixel points in the JND point image, a residual image is obtained, so that the validity of the obtained residual image is ensured.
In addition, referring to fig. 3, the present application also provides an image processing system, which includes a preprocessing module a10, an analysis transformation module a20, and a quantization coding module a30;
the preprocessing module a10 is configured to divide an original image into a JND point image and a residual image;
The analysis transformation module a20 is configured to input the JND point image and the residual image into a preset multi-level residual compensation network model, and output the JND point image and the residual image to obtain potential representation parameters;
and the quantization coding module A30 is used for sequentially quantizing and entropy coding the potential representation parameters to obtain a target compressed image corresponding to the original image.
Optionally, the multi-level residual compensation network model includes two sub-encoder models sharing the same network structure, and the network model formulas corresponding to the two sub-encoder moduli respectively include:
Wherein x m represents a residual image, x p represents a JND point image, i represents the number of processing times, and Representing a previously processed residual image of a processing input by a multi-level residual compensation network model, saidThe JND point image of the previous processing which represents the processing input of the multistage residual compensation network model, and the ≡indicates the multiplication by element.
Optionally, the analysis transformation module a20 is configured to:
Inputting the residual image into a sub-encoder model for forward transformation processing to obtain a first transformation processing result;
Forward transforming the JND point image and the first transforming result through another sub-encoder to obtain a second transforming result;
and determining potential representation parameters according to the preset processing times and the second transformation processing result.
Optionally, the preprocessing module a10 is configured to:
determining a distorted image corresponding to the original image in the JND data set;
Determining a JND point image based on the original image and the distorted image;
And determining a residual image according to the JND point image and the original image.
Optionally, the preprocessing module a10 is configured to:
Constructing a data table with a corresponding relation between an original image and a distorted image, and taking the data table as a JND data set, wherein the data table comprises a corresponding relation between at least one original image and at least one distorted image, and the distorted image is obtained by compressing the original image.
Optionally, the preprocessing module a10 is configured to:
sequentially comparing a plurality of distorted images with the original image;
And if a distorted image with visual difference with the original image exists in the plurality of distorted images, taking the distorted image with visual difference as a JND point image, wherein the visual difference comprises image difference pixels which can be obviously recognized by human eyes of a user.
Optionally, the preprocessing module a10 is configured to:
and carrying out pixel value subtraction on the same position points between the original image and the JND point image, and taking the original image with the pixel values of all the position points subjected to pixel value subtraction as a residual image.
In addition, the application also provides an electronic device, which comprises a memory, a processor and an image processing program stored on the memory and capable of running on the processor, wherein the image processing program realizes the steps of the image processing method when being executed by the processor.
In addition, in an embodiment, fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present invention, where, as shown in fig. 7, the electronic device includes a processor, and optionally further includes an internal bus, a network interface, and a memory at a hardware level. The Memory may include a Memory, such as a Random-Access Memory (RAM), and may further include a non-volatile Memory (non-volatile Memory), such as at least 1 disk Memory. Of course, the electronic device may also include hardware required for other services. The processor, network interface, and memory may be interconnected by an internal bus, which may be an ISA (Ind ustry Standa rd Architecture ) bus, a PCI (PERIPHERAL COMPONENT INTERCONNECT, peripheral component interconnect standard) bus, or EISA (Extended Industry Standard Architecture ) bus, among others. The buses may be classified as address buses, data buses, control buses, etc. For ease of illustration, only one bi-directional arrow is shown in FIG. 7, but not only one bus or type of bus. And the memory is used for storing programs. In particular, the program may include program code including computer-operating instructions. The processor reads the corresponding computer program from the nonvolatile memory into the memory and then runs, forming the shared resource access control device on a logic level. And a processor for executing the program stored in the memory, and specifically for executing the steps of the image processing method.
The specific implementation manner of the electronic device of the present application is substantially the same as that of each embodiment of the image processing method, and will not be repeated here.
In addition, in order to achieve the above object, the present application also provides a computer-readable storage medium having stored thereon an image processing program which, when executed by a processor, implements the steps of the image processing method as described above.
The specific implementation of the computer readable storage medium of the present application is substantially the same as the above embodiments of the image processing method, and will not be described herein.
Those of ordinary skill in the art will appreciate that all or some of the steps, systems, functional modules/units in the apparatus, and methods disclosed above may be implemented as software, firmware, hardware, and suitable combinations thereof. In a hardware implementation, the division between functional modules/units mentioned in the above description does not necessarily correspond to the division of physical components, for example, one physical component may have a plurality of functions, or one function or step may be cooperatively performed by several physical components. Some or all of the physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on computer readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). The term computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data, as known to those skilled in the art. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital Versatile Disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by a computer. Furthermore, as is well known to those of ordinary skill in the art, communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media.
The foregoing description is only of the preferred embodiments of the present invention, and is not intended to limit the scope of the invention, but rather is intended to cover any equivalents of the structures or equivalent processes disclosed herein or in the alternative, which may be employed directly or indirectly in other related arts.

Claims (8)

1.一种图像处理方法，其特征在于，所述图像处理方法包括：1. An image processing method, characterized in that the image processing method comprises: 将原始图像分割为恰可察失真JND点图像和残差图像；The original image is segmented into a Just-Observable Distortion (JND) point image and a residual image; 将所述JND点图像和所述残差图像输入到预设的多级残差补偿网络模型中，输出得到潜在表示参数；The JND point image and the residual image are input into a preset multi-level residual compensation network model, and the latent representation parameters are output. 对所述潜在表示参数依次进行量化和熵编码，得到所述原始图像对应的目标压缩图像；The latent representation parameters are sequentially quantized and entropy encoded to obtain the target compressed image corresponding to the original image; 其中，所述多级残差补偿网络模型包括两个共享相同网络结构的子编码器模型，两个所述子编码器模型对应的网络模型公式分别包括：The multi-level residual compensation network model includes two sub-encoder models sharing the same network structure. The network model formulas corresponding to the two sub-encoder models are as follows: 其中，所述xm表示残差图像，所述xp表示JND点图像，所述i表示处理次数，所述表示多级残差补偿网络模型进行处理输入的前一次处理的残差图像，所述表示多级残差补偿网络模型进行处理输入的前一次处理的JND点图像，所述⊙表示逐元素相乘；Wherein, x<sub> m </sub> represents the residual image, x<sub> p </sub> represents the JND point image, and i represents the number of processing steps. This represents the residual image from the previous processing stage that is processed by the multi-level residual compensation network model. The image represents the JND point image processed in the previous step of the multi-level residual compensation network model, where ⊙ represents element-wise multiplication. 其中，所述将所述JND点图像和所述残差图像输入到预设的多级残差补偿网络模型中，输出得到潜在表示参数的步骤，包括：The step of inputting the JND point image and the residual image into a preset multi-level residual compensation network model and outputting the latent representation parameters includes: 将所述残差图像输入至一个子编码器模型进行前向变换处理，得到第一变换处理结果；The residual image is input into a sub-encoder model for forward transformation processing to obtain the first transformation processing result; 通过另一个子编码器对JND点图像和所述第一变换处理结果进行前向变换处理，得到第二变换处理结果；The JND point image and the first transformation result are subjected to a forward transformation process by another sub-encoder to obtain the second transformation result. 依据预设的处理次数和所述第二变换处理结果确定潜在表示参数。The potential representation parameters are determined based on the preset number of processing steps and the result of the second transformation process. 2.如权利要求1所述的图像处理方法，其特征在于，所述将原始图像分割为JND点图像和残差图像的步骤，包括：2. The image processing method as described in claim 1, characterized in that the step of segmenting the original image into a JND point image and a residual image includes: 确定JND数据集中与所述原始图像对应的失真图像；Identify the distorted image in the JND dataset that corresponds to the original image; 基于所述原始图像和所述失真图像确定JND点图像；Determine the JND point image based on the original image and the distorted image; 根据所述JND点图像和所述原始图像确定残差图像。The residual image is determined based on the JND point image and the original image. 3.如权利要求2所述的图像处理方法，其特征在于，所述确定JND数据集中与所述原始图像对应的失真图像的步骤之前，包括：3. The image processing method as described in claim 2, characterized in that, before the step of determining the distorted image in the JND dataset corresponding to the original image, it includes: 构建包含具有原始图像和失真图像对应关系的数据表，并将所述数据表作为JND数据集，其中，所述数据表包括有至少一原始图像与至少一失真图像之间的对应关系，所述失真图像为所述原始图像进行压缩得到的。Construct a data table containing the correspondence between original images and distorted images, and use the data table as the JND dataset. The data table includes the correspondence between at least one original image and at least one distorted image, wherein the distorted image is obtained by compressing the original image. 4.如权利要求2所述的图像处理方法，其特征在于，所述基于所述原始图像和所述失真图像确定JND点图像的步骤，包括：4. The image processing method as described in claim 2, characterized in that the step of determining the JND point image based on the original image and the distorted image includes: 将多个所述失真图像依次与所述原始图像进行对比；The distorted images are compared sequentially with the original image; 若在多个所述失真图像中存在有和所述原始图像具有视觉差异的失真图像，则将具有视觉差异的失真图像作为JND点图像，其中，所述视觉差异包括用户人眼能明显识别到的图像差异像素点。If among the multiple distorted images there exists a distorted image that has a visual difference from the original image, then the distorted image with the visual difference is taken as the JND point image, wherein the visual difference includes image difference pixels that can be clearly identified by the human eye. 5.如权利要求2所述的图像处理方法，其特征在于，所述根据所述JND点图像和所述原始图像确定残差图像，包括：5. The image processing method as described in claim 2, characterized in that, determining the residual image based on the JND point image and the original image includes: 进行所述原始图像与所述JND点图像之间相同位置点的像素值相减，并将所有位置点的像素值均进行像素值相减后的原始图像作为残差图像。The pixel values at the same locations between the original image and the JND point image are subtracted, and the original image after subtracting the pixel values at all locations is used as the residual image. 6.一种图像处理系统，其特征在于，所述图像处理系统包括前处理模块、分析变换模块和量化编码模块；6. An image processing system, characterized in that the image processing system comprises a preprocessing module, an analysis and transformation module, and a quantization and encoding module; 所述前处理模块，用于将原始图像分割为JND点图像和残差图像；The preprocessing module is used to segment the original image into a JND point image and a residual image; 所述分析变换模块，用于对将所述JND点图像和所述残差图像输入到预设的多级残差补偿网络模型中，输出得到潜在表示参数；其中，所述多级残差补偿网络模型包括两个共享相同网络结构的子编码器模型，两个所述子编码器模型对应的网络模型公式分别包括：The analysis and transformation module is used to input the JND point image and the residual image into a preset multi-level residual compensation network model and output latent representation parameters; wherein, the multi-level residual compensation network model includes two sub-encoder models sharing the same network structure, and the network model formulas corresponding to the two sub-encoder models respectively include: 其中，所述xm表示残差图像，所述xp表示JND点图像，所述i表示处理次数，所述表示多级残差补偿网络模型进行处理输入的前一次处理的残差图像，所述表示多级残差补偿网络模型进行处理输入的前一次处理的JND点图像，所述⊙表示逐元素相乘；Wherein, x<sub> m </sub> represents the residual image, x<sub> p </sub> represents the JND point image, and i represents the number of processing steps. This represents the residual image from the previous processing stage that is processed by the multi-level residual compensation network model. The image represents the JND point image processed in the previous step of the multi-level residual compensation network model, where ⊙ represents element-wise multiplication. 其中，所述将所述JND点图像和所述残差图像输入到预设的多级残差补偿网络模型中，输出得到潜在表示参数，包括：将所述残差图像输入至一个子编码器模型进行前向变换处理，得到第一变换处理结果；通过另一个子编码器对JND点图像和所述第一变换处理结果进行前向变换处理，得到第二变换处理结果；依据预设的处理次数和所述第二变换处理结果确定潜在表示参数；The step of inputting the JND point image and the residual image into a preset multi-level residual compensation network model and outputting latent representation parameters includes: inputting the residual image into a sub-encoder model for forward transformation processing to obtain a first transformation processing result; performing forward transformation processing on the JND point image and the first transformation processing result through another sub-encoder to obtain a second transformation processing result; and determining the latent representation parameters based on a preset number of processing steps and the second transformation processing result. 所述量化编码模块，用于对所述潜在表示参数依次进行量化和熵编码，得到所述原始图像对应的目标压缩图像。The quantization and encoding module is used to sequentially quantize and entropy encode the latent representation parameters to obtain the target compressed image corresponding to the original image. 7.一种电子设备，其特征在于，所述电子设备包括：存储器、处理器及存储在所述存储器上并可在所述处理器上运行的图像处理程序，所述图像处理程序被所述处理器执行时实现如权利要求1至5中任一项所述的图像处理方法的步骤。7. An electronic device, characterized in that the electronic device comprises: a memory, a processor, and an image processing program stored in the memory and executable on the processor, wherein the image processing program, when executed by the processor, implements the steps of the image processing method as described in any one of claims 1 to 5. 8.一种计算机可读存储介质，其特征在于，所述计算机可读存储介质上存储有图像处理程序，所述图像处理程序被处理器执行时实现如权利要求1至5中任一项所述的图像处理方法的步骤。8. A computer-readable storage medium, characterized in that an image processing program is stored on the computer-readable storage medium, the image processing program, when executed by a processor, implements the steps of the image processing method as described in any one of claims 1 to 5.
CN202310144875.8A 2023-02-16 2023-02-16 Image processing method, system, electronic device and computer readable storage medium Active CN116248897B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310144875.8A CN116248897B (en) 2023-02-16 2023-02-16 Image processing method, system, electronic device and computer readable storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310144875.8A CN116248897B (en) 2023-02-16 2023-02-16 Image processing method, system, electronic device and computer readable storage medium

Publications (2)

Publication Number Publication Date
CN116248897A CN116248897A (en) 2023-06-09
CN116248897B true CN116248897B (en) 2025-11-21

Family

ID=86629221

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310144875.8A Active CN116248897B (en) 2023-02-16 2023-02-16 Image processing method, system, electronic device and computer readable storage medium

Country Status (1)

Country Link
CN (1) CN116248897B (en)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112437302A (en) * 2020-11-12 2021-03-02 深圳大学 JND prediction method and device for screen content image, computer device and storage medium
CN114501010A (en) * 2020-10-28 2022-05-13 Oppo广东移动通信有限公司 Image encoding method, image decoding method and related device

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020085278A1 (en) * 2018-10-22 2020-04-30 日本放送協会 Encoding device, decoding device, and program

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114501010A (en) * 2020-10-28 2022-05-13 Oppo广东移动通信有限公司 Image encoding method, image decoding method and related device
CN112437302A (en) * 2020-11-12 2021-03-02 深圳大学 JND prediction method and device for screen content image, computer device and storage medium

Also Published As

Publication number Publication date
CN116248897A (en) 2023-06-09

Similar Documents

Publication Publication Date Title
CN111641832B (en) Encoding method, decoding method, device, electronic device and storage medium
US10834425B2 (en) Image compression/decompression method and device, and image processing system
US20220215595A1 (en) Systems and methods for image compression at multiple, different bitrates
US11955994B2 (en) Data processing apparatuses, methods, computer programs and computer-readable media
CN112715029B (en) AI encoding apparatus and operation method thereof, and AI decoding apparatus and operation method thereof
CN111641826A (en) Method, device and system for encoding and decoding data
CN117315189A (en) Point cloud reconstruction method, system, terminal equipment and computer storage medium
US12388999B2 (en) Method, an apparatus and a computer program product for video encoding and video decoding
US10129565B2 (en) Method for processing high dynamic range video in order to improve perceived visual quality of encoded content
JP7665776B2 (en) Multidistribution Entropy Modeling of Latency Features in Image and Video Coding Using Neural Networks
US12439065B2 (en) Hybrid inter-frame coding using an autoregressive model
CN116248897B (en) Image processing method, system, electronic device and computer readable storage medium
CN111050167B (en) Method and apparatus for recovering degraded frames resulting from reconstruction of source frames
CN119180874B (en) Image coding and decoding method and compression method for machine and human vision
KR20070053098A (en) Decoding apparatus, dequantization method and program thereof
CN119052478A (en) Image coding method, image reconstruction method and device
CN115190313A (en) Video compression method and device, electronic equipment and storage medium
WO2025077704A1 (en) Systems and methods for encoding and decoding parameters of a machine learning model
WO2025087288A1 (en) Method, apparatus, and medium for visual data processing
CN114359418B (en) Image processing method, device, electronic device and storage medium
WO2025212980A1 (en) Method, apparatus, and medium for visual data processing
CN119094798A (en) Method for determining video encoder parameters, electronic device and storage medium
CN120017831A (en) Lossy Image Compression Using Diffusion Model
CN120036001A (en) AI-based image encoding device and image decoding device and image encoding and decoding methods thereof
CN119946264A (en) Video encoding bit rate control method, device, electronic device and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant