[go: up one dir, main page]

WO2022262599A1 - 图像处理方法、装置、计算机设备和存储介质 - Google Patents

图像处理方法、装置、计算机设备和存储介质 Download PDF

Info

Publication number
WO2022262599A1
WO2022262599A1 PCT/CN2022/097077 CN2022097077W WO2022262599A1 WO 2022262599 A1 WO2022262599 A1 WO 2022262599A1 CN 2022097077 W CN2022097077 W CN 2022097077W WO 2022262599 A1 WO2022262599 A1 WO 2022262599A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
layer
processed
displacement
adjacent
Prior art date
Application number
PCT/CN2022/097077
Other languages
English (en)
French (fr)
Inventor
谢朝毅
苏坦
Original Assignee
影石创新科技股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 影石创新科技股份有限公司 filed Critical 影石创新科技股份有限公司
Publication of WO2022262599A1 publication Critical patent/WO2022262599A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/70—Denoising; Smoothing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20016—Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20021—Dividing image into blocks, subimages or windows
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20024—Filtering details
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G06T2207/20221—Image fusion; Image merging

Definitions

  • the present application relates to the technical field of image processing, in particular to an image processing method, device, computer equipment and storage medium.
  • An image processing method comprising:
  • the image pyramid corresponding to the image to be processed is a first image pyramid
  • the image pyramid corresponding to the adjacent image is a second image pyramid
  • the adjacent image and the image to be processed are fused to obtain a processed image.
  • the pixel values of the image to be processed after the primary filtering process and the adjacent image determine the motion area in the image to be processed and the adjacent image after the primary filtering process
  • the calculating the image layer displacement between each corresponding layer of the first image pyramid and the second image pyramid includes:
  • the initial image block displacement being the displacement between the first image block in the first image layer and the corresponding second image block in the second image layer
  • the The first image layer is an image layer in the first image pyramid
  • the second image layer is an image layer in the second image pyramid
  • the first image block is preset in the first image layer an image block of a size
  • the second image block is an image block of the preset size in the second image layer
  • an image layer displacement between each corresponding layer of the first image pyramid and the second image pyramid is determined.
  • the calculating the initial image block displacement of the first image layer includes:
  • the previous image layer is in the first image pyramid, and is similar to the first image layer an adjacent image layer with a lower resolution than the first image layer;
  • a first image block displacement of the first image layer is determined according to the interpolation calculation result, and the initial image block displacement includes the first image block displacement.
  • the calculating the initial image block displacement of the first image layer includes:
  • the previous image layer is in the first image pyramid, and is the same as the first image layer a first image layer adjacent to and having a lower resolution than the first image layer;
  • a second image block displacement of the first image layer is determined according to the statistical calculation result, and the initial image block displacement includes the second image block displacement.
  • the calculating the initial image block displacement of the first image layer includes:
  • a third image block displacement of the first image layer is determined according to the image block whose image block displacement has been determined, and the initial image block displacement includes the third image block displacement.
  • the calculating the initial metric value corresponding to the initial image block displacement includes:
  • an initial metric value corresponding to the displacement of the initial image block is calculated.
  • the aligning the adjacent image with the image to be processed according to the image layer displacement and the index value corresponding to the image layer displacement includes:
  • the image layer displacement between each corresponding layer of the first image pyramid and the second image pyramid is determined based on the image block displacement of the first image block in the first image layer After that, also include:
  • the merging the adjacent image and the image to be processed to obtain the processed image includes:
  • the adjacent image and the image to be processed are fused to obtain a processed image.
  • the processed image is corrected according to the preset fluctuation range of the pixel value.
  • An image processing device comprising:
  • An image acquisition module configured to acquire an image to be processed and adjacent images corresponding to the image to be processed
  • An image pyramid construction module configured to respectively construct image pyramids of the image to be processed and the adjacent image, the image pyramid corresponding to the image to be processed is the first image pyramid, and the image pyramid corresponding to the adjacent image is the first image pyramid Two image pyramids;
  • a calculation module configured to calculate the image layer displacement between each corresponding layer of the first image pyramid and the second image pyramid, and the index value corresponding to the image layer displacement
  • An image alignment module configured to align the adjacent image with the image to be processed according to the displacement of each image layer and the index value corresponding to the displacement of the image layer;
  • An image fusion module configured to fuse the adjacent image and the image to be processed to obtain a processed image.
  • a computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the steps of the above image processing method when executing the computer program.
  • a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above-mentioned image processing method are realized.
  • the above image processing method, device, computer equipment, and storage medium by obtaining the image to be processed and the adjacent image corresponding to the image to be processed; respectively constructing the image pyramid of the image to be processed and the adjacent image, the image pyramid corresponding to the image to be processed is the first An image pyramid, the image pyramid corresponding to the adjacent image is the second image pyramid; calculate the image layer displacement between the first image pyramid and the second image pyramid corresponding to each layer, and the corresponding measurement index value of the image layer displacement; according to each image Align the adjacent image with the image to be processed; merge the adjacent image with the image to be processed to obtain the processed image.
  • the accuracy of displacement calculation can be improved; Aligning with the image to be processed can improve the degree of alignment between the adjacent image and the image to be processed; merging the aligned adjacent image with the image to be processed to obtain the processed image can effectively reduce image noise and improve image quality. Thereby improving image quality.
  • Fig. 1 is an application environment diagram of an image processing method in an embodiment
  • Fig. 2 is a schematic flow chart of an image processing method in an embodiment
  • Fig. 3 is a schematic diagram of an image processing method in a specific embodiment
  • Fig. 4 is a schematic diagram of an image to be processed in a specific embodiment
  • FIG. 5 is a schematic diagram of image quality optimization processing in a specific embodiment
  • Fig. 6 is a schematic diagram of calculating image layer displacement in a specific embodiment
  • Fig. 7 is a structural block diagram of an image processing device in an embodiment
  • Figure 8 is an internal structural diagram of a computer device in one embodiment
  • Fig. 9 is an internal structure diagram of a computer device in another embodiment.
  • the image processing method provided in this application can be applied to the application environment shown in FIG. 1 .
  • the application environment involves both the terminal 102 and the server 104 .
  • the terminal 102 may communicate with the server 104 through other communication methods such as a network or a protocol.
  • the server 104 acquires the image to be processed and the adjacent image corresponding to the image to be processed through the terminal 102, and constructs image pyramids of the image to be processed and the adjacent image respectively.
  • the image pyramid corresponding to the image to be processed is the first image pyramid, and the adjacent The image pyramid corresponding to the image is the second image pyramid; calculate the image layer displacement between each layer corresponding to the first image pyramid and the second image pyramid, and the measurement index value corresponding to the image layer displacement; according to each image layer displacement and image layer displacement Align the adjacent image with the image to be processed according to the corresponding metric value; merge the adjacent image and the image to be processed to obtain the processed image.
  • the application environment of the image processing method provided in this application may only involve the terminal 102 .
  • the terminal 102 may directly acquire the image to be processed and the adjacent images corresponding to the image to be processed, perform image processing on the image to be processed and the adjacent images in the terminal 102, and finally acquire the processed image.
  • the application environment of the image processing method provided in this application may only involve the server 104 .
  • the server 104 may directly acquire the image to be processed and the adjacent images corresponding to the image to be processed, perform image processing on the image to be processed and the adjacent images in the server 104, and finally acquire the processed image.
  • the terminal 102 may be, but not limited to, various cameras, personal computers, notebook computers, smart phones, tablet computers, portable wearable devices, and the like.
  • the server 104 can be implemented by an independent server or a server cluster composed of multiple servers.
  • an image processing method is provided, and the method is applied to the terminal 102 and/or server 104 in FIG. 1 as an example for illustration, including the following steps:
  • Step S202 acquiring the image to be processed and adjacent images corresponding to the image to be processed.
  • an image refers to an image captured by a photographing device.
  • the photographing device may be a camera, or a video camera, or the like.
  • An image to be processed is referred to as an image to be processed, and an image adjacent to the image to be processed is referred to as an adjacent image, and the adjacent image may be one image or multiple images.
  • the shooting device when the shooting device is a camera, the camera can directly capture an image, and the captured image that needs to be processed is called an image to be processed, and the previous or Multiple images are referred to as adjacent images corresponding to the image to be processed, or one or more images obtained after the image to be processed are referred to as adjacent images corresponding to the image to be processed.
  • the shooting device when the shooting device is a camera, the camera can shoot a video, and it is necessary to extract a video frame image from the video.
  • the extracted video frame image that needs to be processed is called the image to be processed, and the extracted frame or multiple frames of the image to be processed are called the adjacent image corresponding to the image to be processed, or the extracted, to be processed image
  • the next frame or frames of images to be processed are referred to as adjacent images corresponding to the image to be processed.
  • the video may be historical video or real-time video
  • the image may be historical video frame image or real-time video frame image.
  • Step S204 building image pyramids of the image to be processed and the adjacent image respectively, the image pyramid corresponding to the image to be processed is the first image pyramid, and the image pyramid corresponding to the adjacent image is the second image pyramid.
  • down-sampling or up-sampling may be performed on the image to be processed and adjacent images respectively to obtain an image pyramid, such as a Gaussian pyramid or a Laplacian pyramid.
  • the image pyramid corresponding to the image to be processed is the first image pyramid
  • the image pyramid corresponding to the adjacent image is the second image pyramid.
  • downsampling Down sampling refers to downsampling the image.
  • the image pyramid corresponding to the image can be obtained.
  • An image pyramid is a structure that describes an image in multiple resolutions. It is a collection of a series of pyramid-shaped resolutions that are gradually reduced and originate from the same image. The higher the level of the image pyramid, the smaller the image and the lower the resolution.
  • the OpenCV software library may be used for downsampling, or the image may be downsampled directly, or an improved weighted downsampling method may be used for downsampling.
  • the OpenCV software library is a computer vision and machine learning software library that can run on operating systems such as Linux, Windows, Android, and Mac OS, and can implement general algorithms for computer vision and image processing.
  • the image to be processed and the adjacent image are down-sampled in the same manner.
  • the image to be processed is down-sampled to obtain a first image pyramid, where the first image pyramid includes each first image layer.
  • the adjacent images are down-sampled to obtain a second image pyramid, where the second image pyramid includes each second image layer. Since the downsampling method is the same, and the image to be processed and the adjacent image are images obtained by the same shooting device, after the image to be processed and the adjacent image are downsampled in the same way, the obtained first image layers are respectively Corresponding to each second image layer.
  • Step S206 calculating the image layer displacement between each corresponding layer of the first image pyramid and the second image pyramid, and the corresponding index value of the image layer displacement.
  • the offset between the image to be processed and the adjacent image can be decomposed into the image layer displacement between each layer corresponding to the first image pyramid and the second image pyramid, and the image layer displacement is used to determine the Handles the displacement between an image and its neighbors.
  • the adjacent image of the image to be processed is an image
  • the adjacent image when the adjacent image is aligned with the image to be processed, the displacement between the adjacent image and the image to be processed only needs to be calculated once.
  • the adjacent images of the image to be processed are multiple images
  • the displacement between the adjacent image and the image to be processed needs to be calculated multiple times.
  • the adjacent images of the image to be processed are two images, which are the previous image and the next image of the image to be processed. When aligning these three images, it is necessary to calculate the difference between the previous image and the image to be processed.
  • Displacement so that the previous image is aligned with the image to be processed, and the displacement between the image to be processed and the next image needs to be calculated, so that the next image is aligned with the image to be processed, that is, the displacement needs to be calculated twice.
  • the measurement index value can measure the quality of the image layer displacement
  • the image layer displacement corresponds to the measurement index value. The smaller the value of the metric, the better the corresponding image layer displacement.
  • the optimal image layer displacement corresponding to each layer can be determined according to the measurement index value.
  • Step S208 according to the displacement of each image layer and the index value corresponding to the displacement of the image layer, align the adjacent image with the image to be processed.
  • the adjacent image may be aligned with the image to be processed according to each image layer displacement and the index value corresponding to the image layer displacement.
  • the adjacent images can be aligned with the image to be processed layer by layer, so as to improve the alignment degree of the images, so that the aligned adjacent images are basically consistent with the image to be processed.
  • Step S210 fusing the adjacent image and the image to be processed to obtain a processed image.
  • image fusion refers to merging several processed images into one image, which can improve the accuracy of the image, increase the resolution of the original image, and the like. After aligning the adjacent image with the image to be processed, the adjacent image and the image to be processed are fused. Specifically, fusion may be performed based on the weights of the image to be processed and adjacent images. After fusion, a fused image is obtained, which is called a processed image.
  • the image pyramid corresponding to the image to be processed is the first image pyramid, and the adjacent image
  • the corresponding image pyramid is the second image pyramid; calculate the image layer displacement between each layer corresponding to the first image pyramid and the second image pyramid, and the corresponding measurement index value of the image layer displacement; according to each image layer displacement and the corresponding image layer displacement Align the adjacent image with the image to be processed; merge the adjacent image and the image to be processed to obtain the processed image.
  • the accuracy of displacement calculation can be improved; Aligning with the image to be processed can improve the degree of alignment between the adjacent image and the image to be processed; merging the aligned adjacent image with the image to be processed to obtain the processed image can effectively reduce image noise and improve image quality. Thereby improving image quality.
  • step S202 after obtaining the image to be processed and the adjacent image corresponding to the image to be processed in step S202, and before constructing the image pyramids of the image to be processed and the adjacent image respectively in step S204, it also includes:
  • Step S302 performing image quality optimization processing on the image to be processed and adjacent images, to obtain the image to be processed and adjacent images after image quality optimization processing.
  • image quality optimization processing can be performed on the image to be processed and the adjacent image respectively by means of image contrast processing, image sharpening, and the like.
  • image contrast processing may be used.
  • the image contrast refers to the size of the gray contrast of the image.
  • the image is an image captured at night, the image is overall dark due to low visibility at night. Therefore, the contrast of the image can be reduced to improve the details of the dark area of the image.
  • due to the complex lighting conditions at night and the influence of lights such as street lights and neon lights it is inevitable that there will be areas in the image whose brightness is higher than that of the surrounding areas, called highlight areas. After reducing the contrast of the image and improving the details of the dark area of the image, the highlight area will be lost, so the contrast of the image needs to be increased to restore the highlight area of the image.
  • the manner of performing image quality optimization processing on the image to be processed and the adjacent image is the same.
  • the following embodiments take the processing of the image to be processed as an example, and the contrast processing of the image to be processed may be performed by means of curve mapping.
  • the look-up table (Look-up table) can be used to improve the image quality, improve the details of the dark area, reduce the contrast of the image, and restore the high light area of the image to improve the contrast of the image.
  • image processing software can be used to perform lookup table operations, such as exposure adjustment, tone scale mapping, etc. in the image processing software PS.
  • the dark channel defogging method can also be used to restore the high light area of the image and improve the contrast of the image.
  • the contrast-processed image to be processed may be weighted with the original image to be processed to obtain an image to be processed with more detail in highlight areas.
  • the image to be processed and the image to be processed after contrast processing may be weighted to obtain a weight map corresponding to the image to be processed.
  • the pixel value of the original image to be processed can be multiplied by a preset multiple and blurred to obtain a weight map corresponding to the image to be processed.
  • the weight map can be expressed as W, and the preset multiple can be 1.5 times, 2 times or 3 times, etc.
  • the blurring can be a mean blurring and its radius can be 21.
  • Image processing software can also be used to obtain the weight map corresponding to the image to be processed. Specifically, highlight extraction in image processing software PS and the like can be used.
  • the preset formula can be used to restore the highlight area details of the weight map corresponding to the image to be processed, so as to obtain the image to be processed after image quality optimization processing.
  • the preset formula is as follows:
  • W(y,x) represents the value at the pixel point (y,x) in the weight map W corresponding to the image to be processed
  • r, g, and b represent the pixel values of the red, green, and blue channels of the original image to be processed
  • r new , g new , b new represent the pixel values of the red, green and blue channels of the image to be processed after contrast processing.
  • step S202 after obtaining the image to be processed and the adjacent image corresponding to the image to be processed in step S202, and before constructing the image pyramids of the image to be processed and the adjacent image respectively in step S204, it also includes:
  • Step S402 performing a filter process on the image to be processed and the adjacent image to obtain the image to be processed and the adjacent image after the filter process once.
  • impulse noise is a common noise in the image, manifested as white or black spots randomly appearing in the image, which may be black pixels in the highlight area, or white pixels in the dark area, or both.
  • Impulse noise can be caused by strong disturbances in the image signal, errors in the analog-to-digital converter or bit transmission, etc.
  • filter processing is used to remove impulse noise, for example, median filter processing, or other filter processing for removing isolated points.
  • the impulse noise will be expanded to several pixels, which requires multiple filtering processes. However, multiple filtering processes will blur the image. Therefore, it is necessary to perform targeted filtering processing on the image, for example, filter processing for flat areas of the image, or filter processing for moving areas of the image, etc., to effectively reduce the image quality. noise.
  • the image to be processed and the adjacent image are filtered in the same manner.
  • the global filtering process is performed on the entire image, and the global filtering process is called primary filtering process.
  • a filter process is performed on the image to be processed and the adjacent image to obtain the image to be processed and the adjacent image after the filter process once.
  • a median filtering process or an improved median filtering process may be used.
  • Step S404 according to the pixel values of the image to be processed after the first filtering process and the adjacent image, determine the motion area in the image to be processed after the primary filtering process and the adjacent image.
  • the noise in the non-moving area of the image can be removed through subsequent image alignment and fusion processes, filtering can be performed on the moving area of the image, and the moving area of the image needs to be determined.
  • the motion area in the image to be processed after the primary filtering process and the adjacent image is determined.
  • the motion area threshold is set as the first preset motion threshold. set value.
  • the difference between the pixel values of the current area of the image to be processed and the corresponding area of the adjacent image is less than the preset motion threshold, the current area is determined as a non-motion area, and the non-motion area threshold is set as the second preset value.
  • the value range of the preset motion threshold is (0, 255), which can be specifically set to 35.
  • the motion area threshold, that is, the first preset value may be set to 255
  • the non-motion area threshold that is, the second preset value may be set to 0.
  • Step S406 calculating boundary values between the image to be processed and the adjacent image after the filtering process.
  • the filtering process should not be performed at the boundary of the image, and it is necessary to calculate the boundary value of the image to be processed and the adjacent image after filtering once.
  • the Sobel operator Sobel operator
  • an edge detection algorithm can be used to calculate the boundary value of the image to be processed and the adjacent image after a filtering process.
  • the current area is determined as the boundary area, and the boundary area threshold is set as the first preset value.
  • the current area is determined as a non-boundary area, and the non-boundary area threshold is set as a second preset value.
  • the value range of the preset boundary threshold is (0, 255), which can be specifically set to 35.
  • the boundary area threshold, that is, the first preset value may be set to 255, and the non-boundary area threshold, that is, the second preset value may be set to 0.
  • Step S408 based on the boundary value, determine the non-boundary motion area in the image to be processed after the primary filtering process and the adjacent image.
  • the image to be processed and the non-boundary motion area in the adjacent image after the primary filtering process are determined.
  • Step S410 performing secondary filtering on the non-boundary motion area to obtain the noise-processed image to be processed and adjacent images.
  • secondary filtering processing After determining the non-boundary motion area of the image to be processed and the adjacent image, performing filtering processing on the non-boundary motion area is called secondary filtering processing.
  • the manner of the secondary filtering process and the primary filtering process may be the same. Specifically, median filtering or improved median filtering may be used to obtain the image to be processed and adjacent images after noise processing, and apply them to subsequent calculations.
  • step S206 calculates the image layer displacement between each layer corresponding to the first image pyramid and the second image pyramid, including:
  • Step S502 calculate the initial image block displacement of the first image layer, the initial image block displacement is the displacement between the first image block in the first image layer and the corresponding second image block in the second image layer, the first image The layer is the image layer in the first image pyramid, the second image layer is the image layer in the second image pyramid, the first image block is an image block with a preset size in the first image layer, and the second image block is the second image Image blocks of preset size in the layer.
  • the image layer displacement may be determined by calculating the image block displacement of each image block of each image layer corresponding to the image pyramid, so as to improve calculation accuracy.
  • each first image layer is obtained after down-sampling is performed on the image to be processed.
  • each second image layer is obtained.
  • An image block with a preset size in the first image layer is referred to as a first image block, and each first image block with a preset size is obtained.
  • image blocks with a preset size in the second image layer are referred to as second image blocks, and each second image block with a preset size is obtained.
  • the obtained first image block with a preset size corresponds to the second image block.
  • the calculated displacement between the first image block and the corresponding second image block is called the initial image block displacement.
  • the calculation is performed according to each image block of each image layer.
  • the displacement of the minimum resolution image layer needs to be initialized so that the image block displacement is 0.
  • Step S504 calculating an initial metric value corresponding to the initial image block displacement.
  • multiple calculation methods may be used to calculate various initial image block displacements of the first image layer, and then an optimal image block displacement is determined according to a corresponding measurement index value.
  • the initial metric value corresponding to the displacement of the initial image block is calculated.
  • the initial image block displacement corresponds to the initial metric value.
  • the initial metric value can measure the quality of the calculated initial image block displacement, and an optimal image block displacement can be determined from multiple initial image block displacements.
  • Step S506 according to the initial image block displacement and the initial metric value, determine the image layer displacement between each corresponding layer of the first image pyramid and the second image pyramid.
  • the smaller the value of the initial metric the better the corresponding image block displacement.
  • the initial image block displacement corresponding to the initial metric value with the smallest value may be determined as the optimal image block displacement of the first image layer.
  • the image layer displacement between each corresponding layer of the first image pyramid and the second image pyramid is determined through the image block displacement of the first image layer.
  • step S502 calculates the initial image block displacement of the first image layer, including:
  • Step S602 perform interpolation calculation on the image layer displacement of the previous image layer of the first image layer, and obtain the interpolation calculation result, the previous image layer is in the first image pyramid, adjacent to the first image layer, and has a resolution of The image layer below the first image layer.
  • the first image pyramid is obtained after the image to be processed is down-sampled.
  • an image layer adjacent to the first image layer and having a lower resolution than the first image layer, that is, a higher level image layer is used as the previous image layer.
  • the image layer displacement of the previous image layer is essentially obtained based on the image layer displacement of the image layer with the minimum resolution.
  • the interpolation calculation is performed on the image layer displacement of the previous image layer of the first image layer to obtain an interpolation calculation result.
  • the interpolation calculation may adopt methods such as Lagrangian interpolation, Newton interpolation, segmental interpolation, bilinear interpolation, and the like.
  • Step S604 Determine the first image block displacement of the first image layer according to the interpolation calculation result, where the initial image block displacement includes the first image block displacement.
  • the interpolation calculation result of the image layer displacement of the previous image layer needs to be enlarged by the corresponding resolution After that, it can be used as the image block displacement of the first image block of the first image layer, and the image block displacement determined by using the steps of this embodiment is called the first image block displacement.
  • the multiple of the resolution amplification is the ratio of the resolution of the first image layer to the previous image layer.
  • the resolution of the first image layer is twice the resolution of the previous image layer, that is, the corresponding resolution magnification factor is 2, and the interpolation calculation result needs to be multiplied by 2 before it can be used as the first image layer of the first image layer.
  • the image block displacement of the image block is the image block displacement of the image block.
  • step S502 calculates the initial image block displacement of the first image layer, including:
  • Step S702 perform statistical calculation on the image layer displacement of the previous image layer of the first image layer, and obtain the statistical calculation result, the previous image layer is in the first image pyramid, adjacent to the first image layer, and has a resolution of The first image layer below the first image layer.
  • the previous image layer is an image layer adjacent to the first image layer and having a lower resolution than the first image layer among the first image layers.
  • Step S704 according to the statistical calculation result, determine the second image block displacement of the first image layer, where the initial image block displacement includes the second image block displacement.
  • the statistical calculation result of the image layer displacement of the previous image layer needs to be enlarged by the corresponding resolution before it can be used as the image block displacement of the first image block of the first image layer.
  • the image block displacement determined in the example step is called the second image block displacement.
  • step S502 calculates the initial image block displacement of the first image layer, including:
  • Step S802 determining an image block whose image block displacement has been determined in the first image layer.
  • the initial image block displacement of the first image layer may also be determined based on the image blocks whose image block displacement has been determined in the first image layer. Wherein, the image block whose image block displacement has been determined also needs to be an image block adjacent to the first image block.
  • the calculation when calculating the image block displacement of each first image block of the first image layer, the calculation may be performed on each first image block in a preset manner.
  • the preset method may be to perform calculation according to a serpentine shape, specifically, the image blocks in the first row of the first image layer are calculated from left to right, and the image blocks in the second row are calculated from right to left, and the cycle repeats in turn.
  • its image block displacement may be determined according to step S602-step S604, step S702-step S704, or may be determined according to the steps of this embodiment.
  • the image blocks adjacent to the first image block in the first image layer and whose image block displacement has been determined are a preset number of image blocks. Wherein, the preset number may be multiple, such as 2, 3 or 4, and so on.
  • Step S804 Determine a third image block displacement of the first image layer according to the image block whose image block displacement has been determined, and the initial image block displacement includes the third image block displacement.
  • the image block displacement of each first image block of the first image layer can be determined according to the image block whose image block displacement has been determined, and the image block displacement determined by the steps of this embodiment is called the third image block displacement.
  • the image block whose image block displacement needs to be determined is the first calculated image block in the first image layer
  • the image block, that is, the first image block does not have any adjacent image blocks whose image block displacement has been determined.
  • the image block displacement of the first image block can only be determined according to step S602-step S604 or step S702-step S704.
  • step S504 calculates the initial metric value corresponding to the initial image block displacement, including:
  • Step S902 determining a second image block in the second image layer whose displacement is the same as that of the initial image block in the first image layer.
  • the second image layer refers to a second image layer corresponding to the first image layer.
  • the corresponding displaced second image block in the second image layer may be determined according to the initial image block displacement of the first image block in the first image layer.
  • a second image layer corresponding to the first image layer may be determined, and in the second image layer, a second image block having the same displacement as the initial image block of the first image block of the first image layer is determined.
  • Step S904 according to the pixel value of the first image block in the first image layer and the pixel value of the second image block, calculate the initial index value corresponding to the displacement of the initial image block.
  • the sum of absolute differences of pixel values between two image blocks may be used as the initial measurement index, or the correlation between the two image blocks may be used as the initial measurement index.
  • the pixel value of the first image block in the first image layer and the second image with the same displacement of the above-mentioned initial image block can be The pixel value of the block, and calculate the initial metric value corresponding to the displacement of the initial image block.
  • step S208 aligns the adjacent image with the image to be processed according to the displacement of each image layer and the index value corresponding to the displacement of the image layer, including:
  • Step S1002 based on the initial metric value, determine the image block displacement of the first image block in the first image layer from the initial image block displacement.
  • the optimal first image can be determined from the first image block displacement, the second image block displacement, and the third image block displacement corresponding to the initial metric value with the smallest value The tile displacement of the first tile in the layer.
  • Step S1004 based on the image block displacement of the first image block in the first image layer, determine the image layer displacement between each corresponding layer of the first image pyramid and the second image pyramid, and the index value corresponding to the image layer displacement.
  • the image layer displacement between each corresponding layer of the first image pyramid and the second image pyramid is determined through the image block displacement of the first image block in the first image layer. After determining the corresponding image layer displacement between each layer, determine the metric value corresponding to the image layer displacement.
  • Step S1006 according to the displacement of each image layer and the index value corresponding to the displacement of the image layer, align the adjacent image with the image to be processed.
  • the adjacent image may be aligned with the image to be processed according to the determined displacement of the image layer.
  • step S1004 determines the image layer displacement between each corresponding layer of the first image pyramid and the second image pyramid based on the image block displacement of the first image block in the first image layer, it can also be based on the first
  • the image block displacement around an image block updates the image layer displacement, including:
  • Step S1102 determining a preset neighborhood image block adjacent to the first image block in the first image layer.
  • the image block displacement of each first image block of the first image layer can be determined only according to one of the methods in step S602-step S604, step S702-step S704 or step S802-step S804, or At the same time, it is determined according to the above three methods.
  • a preset neighborhood image block adjacent to the first image block in the first image layer is determined.
  • the preset neighboring image blocks are image blocks around the first image block and for which the displacement of the image block has been determined.
  • the number of image blocks in the preset neighborhood can be multiple, for example, 4 around the first image block, 8 in a circle around it, or 24 in two circles around it, etc., the determined image blocks
  • the displacement may be determined according to one of the methods of step S602-step S604, step S702-step S704 or step S802-step S804.
  • Step S1104 update the image block displacement of the first image block in the first image layer according to the image block displacement of each preset neighborhood image block and the corresponding measurement index value, and determine the updated image layer displacement.
  • the image block displacement of the first image block in the first image layer may be updated according to the determined image block displacement of each preset neighborhood image block and the corresponding metric value. Specifically, when there is a smaller and better metric value in the preset neighborhood image block of the first image block, the image block displacement of the first image block is updated to the image block displacement corresponding to the better metric value , otherwise no update.
  • step S210 fuses the adjacent image and the image to be processed, and acquires the processed image, including:
  • Step S1202 Determine the initial weight of the adjacent image according to the updated index value corresponding to the displacement of the image layer.
  • the adjacent image and the image to be processed need to be fused into a noise-reduced image.
  • image fusion may be performed based on image weights. Since the initial image block displacement corresponds to the initial metric value, after the image block displacement of the first image block of the first image layer is determined, there is only one corresponding initial metric value, called the metric value.
  • the second image of the adjacent image corresponding to the image to be processed is determined
  • the initial weight of the block which is to determine the initial weight of the adjacent image.
  • the method of determining the initial weight can also be, when the threshold value of the second image block is greater than or equal to a preset larger threshold value, make the second image block
  • the initial weight of the second image block is 0, when the threshold of the second image block is less than the preset smaller threshold, the initial weight of the second image block is 1, when the threshold of the second image block is at the preset smaller threshold When it is between and the preset larger threshold, its initial weight decreases linearly in turn.
  • step S1204 the preset weight of the image to be processed is fused with the initial weight of the adjacent image to obtain the fusion weight of the adjacent image and the image to be processed.
  • the weighted sum of the preset weight of the image to be processed and the initial weight of the corresponding adjacent image is obtained to obtain the fusion weight of the adjacent image and the image to be processed.
  • the preset weight of the image to be processed may be set to 1.
  • Step S1206 based on the fusion weight, the adjacent image and the image to be processed are fused to obtain a processed image.
  • the pixels of the processed image The value should be within the range of pixel values of the original image.
  • the pixel value fluctuation range of the processed image is set in advance.
  • the pixel value fluctuation range may be set to ⁇ 10.
  • the pixel value in the original image is 50, and the adjacent image is fused with the image to be processed according to the fusion weight.
  • the pixel value of the image is 62, it needs to be corrected to 60. If the A pixel value of 56 requires no correction.
  • the schematic diagram of the image processing method is shown in Figure 3, wherein, after preprocessing the image, the preprocessed image is obtained, and then by aligning and fusing multiple frames of images, one frame after processing is obtained Image.
  • the schematic diagram of the image to be processed is shown in Fig. 4, where there is obvious impulse noise in the image.
  • the specific steps of the image processing method are as follows:
  • image preprocessing Before performing multi-frame noise reduction processing on the image, image preprocessing may also be included, wherein the image preprocessing mainly includes: at least one of image quality optimization processing and noise processing, taking the preprocessing process of the image to be processed as an example, Specific steps include:
  • the pixel value of the original image to be processed is multiplied by 2 times, and the mean value blur processing with a radius of 21 is performed to obtain the weight map W corresponding to the image to be processed.
  • the formula is as follows:
  • W(y,x) represents the value at the pixel point (y,x) in the weight map W corresponding to the image to be processed
  • r, g, and b represent the pixel values of the red, green, and blue channels of the original image to be processed
  • r new , g new , b new represent the pixel values of the red, green and blue channels of the image to be processed after contrast processing.
  • a non-boundary motion region in the image to be processed after global filtering is determined and filtered to obtain an image to be processed after noise processing.
  • Multi-frame image noise reduction processing mainly includes: the alignment process of the image to be processed and the adjacent image, and the fusion process of the aligned image.
  • the specific steps include:
  • the image to be processed and the adjacent image are respectively down-sampled to obtain the first image pyramid corresponding to the image to be processed, including each first image layer; the second image pyramid corresponding to the adjacent image is obtained, and each second image layer included , segmenting the first image layer and the second image layer according to a preset size to obtain first image blocks and second image blocks respectively.
  • the manner of calculating the displacement of the initial image block includes:
  • Method 1 Perform interpolation calculation on the image layer displacement of the previous image layer of the first image layer to obtain the interpolation calculation result.
  • the previous image layer is in the first image pyramid, adjacent to the first image layer, and the resolution is For an image layer lower than the first image layer, a first image block displacement of a first image block of the first image layer is determined according to an interpolation calculation result, and the initial image block displacement includes the first image block displacement.
  • Mode 2 Statistically calculate the image layer displacement of the previous image layer of the first image layer, obtain the statistical calculation result, and determine the second image block displacement of the first image block of the first image layer according to the statistical calculation result, initially
  • the image block displacement includes a second image block displacement.
  • Method 3 Determine the image block adjacent to the first image block in the first image layer and whose displacement has been determined, specifically as shown in Figure 6(a), the arrows indicate the calculation order, and ⁇ indicates the first image block in the first image layer An image block, ⁇ represents the three image blocks adjacent to the first image block and whose image block displacement has been determined, according to the image blocks whose image block displacement has been determined, determine the third image of the first image block of the first image layer
  • the block displacement, the initial image block displacement includes the third image block displacement.
  • the initial metric value corresponding to the initial image block displacement, the initial image block displacement corresponding to the initial metric value with the smallest value is determined as the image block displacement of the first image block, and based on this, it is determined that the first image pyramid and the second image pyramid correspond to each Image layer displacement between layers.
  • the first image block, ⁇ represents the 8 preset neighborhood image blocks adjacent to the first image block, and the image of the first image block is updated according to the image block displacement and the corresponding measurement index value of each preset neighborhood image block The block displacement, based on which the updated image layer displacement is determined.
  • the preset weight of the first image block of the image to be processed is 1, which is fused with the initial weight of the second image block of the adjacent image to obtain the fusion weight of the adjacent image and the image to be processed.
  • the adjacent image and the image to be processed are fused to obtain the processed image, and the processed image is corrected according to the preset fluctuation range of the pixel value.
  • the pixel value fluctuation range is set to ⁇ 10
  • the pixel value of the original image is 50. If the pixel value of the processed image is 62, it needs to be corrected to 60. If the pixel value of the processed image is 56, no correction is required. .
  • steps in the flow chart of FIG. 2 are displayed sequentially as indicated by the arrows, these steps are not necessarily executed sequentially in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least some of the steps in FIG. 2 may include multiple steps or stages. These steps or stages are not necessarily executed at the same time, but may be executed at different times. The execution order of these steps or stages is also It is not necessarily performed sequentially, but may be performed alternately or alternately with other steps or at least a part of steps or stages in other steps.
  • an image processing device including: an image acquisition module 710, an image pyramid construction module 720, a calculation module 730, an image alignment module 740 and an image fusion module 750, wherein:
  • the image acquiring module 710 is configured to acquire an image to be processed and adjacent images corresponding to the image to be processed.
  • An image pyramid construction module 720 configured to respectively construct image pyramids of the image to be processed and the adjacent image, the image pyramid corresponding to the image to be processed is a first image pyramid, and the image pyramid corresponding to the adjacent image is The second image pyramid.
  • the calculation module 730 is configured to calculate the image layer displacement between each corresponding layer of the first image pyramid and the second image pyramid, and a metric value corresponding to the image layer displacement.
  • the image alignment module 740 is configured to align the adjacent image with the image to be processed according to each image layer displacement and the index value corresponding to the image layer displacement.
  • the image fusion module 750 is configured to fuse the adjacent image and the image to be processed, and obtain a processed image.
  • the image processing device also includes:
  • An image quality optimization processing module configured to, after the acquisition of the image to be processed and the adjacent image corresponding to the image to be processed, and before the construction of the image pyramids of the image to be processed and the adjacent image respectively, The image to be processed and the adjacent image are subjected to image quality optimization processing to obtain the image to be processed and the adjacent image after the image quality optimization processing.
  • a noise reduction processing module configured to, after the acquisition of the image to be processed and the adjacent image corresponding to the image to be processed, and before the construction of the image pyramids of the image to be processed and the adjacent image, Perform noise reduction processing on the image to be processed and the adjacent image to obtain the image to be processed and the adjacent image after noise processing.
  • the noise reduction processing module includes the following units:
  • the primary filtering processing unit is configured to perform primary filtering processing on the image to be processed and the adjacent image to obtain the image to be processed and the adjacent image after the primary filtering processing.
  • a motion area determining unit configured to determine the motion area in the image to be processed and the adjacent image after the primary filtering process according to the pixel values of the image to be processed and the adjacent image after the primary filtering process.
  • a boundary value calculation unit configured to calculate boundary values between the image to be processed and the adjacent image after the primary filtering process.
  • the non-boundary motion area determination unit is configured to determine a non-boundary motion area in the image to be processed and adjacent images after the primary filtering process based on the boundary value.
  • the secondary filtering processing unit is configured to perform secondary filtering processing on the non-boundary moving area to obtain the noise-processed image to be processed and adjacent images.
  • the calculation module 730 includes the following units:
  • the initial image block displacement calculation unit is used to calculate the initial image block displacement of the first image layer, the initial image block displacement is the first image block in the first image layer and the second image block in the corresponding second image layer Displacement between image blocks, the first image layer is an image layer in the first image pyramid, the second image layer is an image layer in the second image pyramid, and the first image block is An image block of a preset size in the first image layer, and the second image block is an image block of the preset size in the second image layer.
  • the initial metric value calculation unit is configured to calculate the initial metric value corresponding to the initial image block displacement.
  • An image layer displacement determining unit configured to determine an image layer displacement between each corresponding layer of the first image pyramid and the second image pyramid according to the initial image block displacement and the initial metric value.
  • the initial image block displacement calculation unit includes a first image block displacement calculation unit
  • the first image block displacement calculation unit includes the following units:
  • An interpolation calculation unit configured to perform interpolation calculation on the image layer displacement of the previous image layer of the first image layer to obtain an interpolation calculation result, the previous image layer is in the first image pyramid, and An image layer that is adjacent to the first image layer and has a lower resolution than the first image layer.
  • the first image block displacement determining unit is configured to determine a first image block displacement of the first image layer according to the interpolation calculation result, and the initial image block displacement includes the first image block displacement.
  • the initial image block displacement calculation unit includes a second image block displacement calculation unit
  • the second image block displacement calculation unit includes the following units:
  • a statistical calculation unit configured to perform statistical calculations on the image layer displacement of the previous image layer of the first image layer to obtain a statistical calculation result, the previous image layer is in the first image pyramid, and The first image layer is adjacent to the first image layer and has a lower resolution than the first image layer.
  • the second image block displacement determining unit is configured to determine a second image block displacement of the first image layer according to the statistical calculation result, and the initial image block displacement includes the second image block displacement.
  • the initial image block displacement calculation unit includes a third image block displacement calculation unit
  • the third image block displacement calculation unit includes the following units:
  • An image block determining unit configured to determine an image block whose image block displacement has been determined in the first image layer.
  • a third image block displacement determining unit configured to determine a third image block displacement of the first image layer according to the image block whose image block displacement has been determined, and the initial image block displacement includes the third image block displacement .
  • the initial metric value calculation unit includes the following units:
  • the second image block determination unit is configured to determine a second image block in the second image layer that is displaced the same as the initial image block of the first image layer.
  • the initial metric value determination unit is configured to calculate an initial metric value corresponding to the displacement of the initial image block according to the pixel value of the first image block in the first image layer and the pixel value of the second image block.
  • the image alignment module 740 includes the following units:
  • An image block displacement determining unit configured to determine an image block displacement of a first image block in the first image layer from the initial image block displacement based on the initial metric value.
  • An image layer displacement determination unit configured to determine an image layer displacement between each corresponding layer of the first image pyramid and the second image pyramid based on the image block displacement of the first image block in the first image layer, and The metric value corresponding to the displacement of the image layer.
  • An image alignment unit configured to align the adjacent image with the image to be processed according to each of the image layer displacements and the index value corresponding to the image layer displacement.
  • the calculation unit also includes an image block displacement update unit, and the image block displacement update unit includes the following units:
  • the second image block determination unit is configured to determine an image between each layer of the first image pyramid and the second image pyramid based on the image block displacement of the first image block in the first image layer After layer shifting, a preset neighborhood image block adjacent to the first image block in the first image layer is determined.
  • An image block displacement updating unit configured to update the image block displacement of the first image block in the first image layer according to the image block displacement of each preset neighborhood image block and the corresponding measurement index value, and determine the updated Image layer displacement.
  • the image fusion module 760 includes the following units:
  • the initial weight determining unit is configured to determine the initial weight of the adjacent image according to the metric value corresponding to the updated image layer displacement.
  • a fusion weight calculation unit configured to fuse the preset weight of the image to be processed with the initial weight of the adjacent image to obtain the fusion weight of the adjacent image and the image to be processed.
  • An image fusion unit configured to fuse the adjacent image and the image to be processed based on the fusion weight to obtain a processed image.
  • the image processing device also includes:
  • the image correction unit is configured to correct the processed image according to a preset fluctuation range of pixel values after the acquired processed image.
  • Each module in the above-mentioned image processing device may be fully or partially realized by software, hardware or a combination thereof.
  • the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, and can also be stored in the memory of the computer device in the form of software, so that the processor can call and execute the corresponding operations of the above modules.
  • a computer device in one embodiment, the computer device may be a server, and its internal structure may be as shown in FIG. 8 .
  • the computer device includes a processor, memory and a network interface connected by a system bus. Wherein, the processor of the computer device is used to provide calculation and control capabilities.
  • the memory of the computer device includes a non-volatile storage medium and an internal memory.
  • the non-volatile storage medium stores an operating system, computer programs and databases.
  • the internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium.
  • the database of the computer device is used to store image processing data.
  • the network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by a processor, an image processing method is realized.
  • a computer device is provided.
  • the computer device may be a terminal, and its internal structure may be as shown in FIG. 9 .
  • the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Wherein, the processor of the computer device is used to provide calculation and control capabilities.
  • the memory of the computer device includes a non-volatile storage medium and an internal memory.
  • the non-volatile storage medium stores an operating system and computer programs.
  • the internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium.
  • the communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be realized through WIFI, an operator network, NFC (Near Field Communication) or other technologies.
  • WIFI Wireless Fidelity
  • NFC Near Field Communication
  • an image processing method is realized.
  • the display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen
  • the input device of the computer device may be a touch layer covered on the display screen, or a button, a trackball or a touch pad provided on the casing of the computer device , and can also be an external keyboard, touchpad or mouse.
  • FIG. 8-9 is only a block diagram of a part of the structure related to the solution of this application, and does not constitute a limitation on the computer equipment on which the solution of this application is applied.
  • the specific computer Devices may include more or fewer components than shown in the figures, or combine certain components, or have a different arrangement of components.
  • a computer device including a memory and a processor, where a computer program is stored in the memory, and the processor implements the steps of the above image processing method when executing the computer program.
  • a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above-mentioned image processing method are realized.
  • Non-volatile memory may include read-only memory (Read-Only Memory, ROM), magnetic tape, floppy disk, flash memory or optical memory, etc.
  • Volatile memory can include Random Access Memory (RAM) or external cache memory.
  • RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Image Processing (AREA)

Abstract

一种图像处理方法、装置、计算机设备和存储介质。所述方法包括：获取待处理图像及待处理图像对应的相邻图像（S202）；分别构建待处理图像和相邻图像的图像金字塔，待处理图像对应的图像金字塔为第一图像金字塔，相邻图像对应的图像金字塔为第二图像金字塔（S204）；计算第一图像金字塔与第二图像金字塔对应每层之间的图像层位移，以及图像层位移对应的衡量指标值（S206）；根据各图像层位移以及图像层位移对应的衡量指标值，将相邻图像与待处理图像对齐（S208）；融合相邻图像与待处理图像，获取处理后的图像（S210）。上述方法能够有效降低图像噪声，提高图像质量。

Description

图像处理方法、装置、计算机设备和存储介质 技术领域
本申请涉及图像处理技术领域，特别是涉及一种图像处理方法、装置、计算机设备和存储介质。
背景技术
随着短视频软件的兴起，人们越来越喜欢使用视频拍摄和记录生活，对视频拍摄的质量要求越来越高。特别是夜晚的能见度比较低，而且光照条件复杂，容易导致视频图像质量不高。因此，出现了针对于夜景的视频拍摄技术，使夜景视频图像质量有所提升。
技术问题
目前，在针对于夜景的视频拍摄技术中，多采用视频提亮和噪声滤除的方法进行图像降噪，从而提高图像质量。然而，使用这种方法降噪会带来时域上的抖动，例如，车辆灯光存在拖影和地面抖动的情况，导致图像质量低。
技术解决方案
基于此，有必要针对上述技术问题，提供一种能够提高图像质量的图像处理方法、装置、计算机设备和存储介质。
一种图像处理方法，所述方法包括：
获取待处理图像及所述待处理图像对应的相邻图像；
分别构建所述待处理图像和所述相邻图像的图像金字塔，所述待处理图像对应的图像金字塔为第一图像金字塔，所述相邻图像对应的图像金字塔为第二图像金字塔；
计算所述第一图像金字塔与所述第二图像金字塔对应每层之间的图像层位移，以及所述图像层位移对应的衡量指标值；
根据各所述图像层位移以及所述图像层位移对应的衡量指标值，将所述相邻图像与所述待处理图像对齐；
融合所述相邻图像与所述待处理图像，获取处理后的图像。
在其中一个实施例中，在所述获取待处理图像及所述待处理图像对应的相邻图像之后，在所述分别构建所述待处理图像和所述相邻图像的图像金字塔之前，还包括：
对所述待处理图像和所述相邻图像进行画质优化处理，得到画质优化处理处理后的待处理图像和相邻图像。
在其中一个实施例中，在所述获取待处理图像及所述待处理图像对应的相邻图像之后，在所述分别构建所述待处理图像和所述相邻图像的图像金字塔之前，还包括：
对所述待处理图像和所述相邻图像进行一次滤波处理，得到一次滤波处理后的待处理图像和相邻图像；
根据所述一次滤波处理后的待处理图像和相邻图像的像素值，确定所述一次滤波处理后的待处理图像和相邻图像中的运动区域；
计算所述一次滤波处理后的待处理图像和相邻图像的边界值；
基于所述边界值，确定所述一次滤波处理后的待处理图像和相邻图像中的非边界的运动区域；
对所述非边界的运动区域进行二次滤波处理，得到噪声处理后的待处理图像和相邻图像。
在其中一个实施例中，所述计算所述第一图像金字塔与所述第二图像金字塔对应每层之间的图像层位移，包括：
计算第一图像层的初始图像块位移，所述初始图像块位移为所述第一图像层中的第一图像块与对应的第二图像层中的第二图像块之间的位移，所述第一图像层为所述第一图像金字塔中的图像层，所述第二图像层为所述第二图像金字塔中的图像层，所述第一图像块为所述第一图像层中预设大小的图像块，所述第二图像块为所述第二图像层中所述预设大小的图像块；
计算所述初始图像块位移对应的初始衡量指标值；
根据所述初始图像块位移和所述初始衡量指标值，确定所述第一图像金字塔与所述第二图像金字塔对应每层之间的图像层位移。
在其中一个实施例中，所述计算第一图像层的初始图像块位移，包括：
对所述第一图像层的前一层图像层的图像层位移进行插值计算，获得插值计算结果，所述前一层图像层为所述第一图像金字塔中，与所述第一图像层相邻、且分辨率低于所述第一图像层的图像层；
根据所述插值计算结果，确定所述第一图像层的第一图像块位移，所述初始图像块位移包括所述第一图像块位移。
在其中一个实施例中，所述计算第一图像层的初始图像块位移，包括：
对所述第一图像层的前一层图像层的图像层位移进行统计计算，获得统计计算结果，所述前一层图像层为所述第一图像金字塔中，与所述第一图像层相邻、且分辨率低于所述第一图像层的第一图像层；
根据所述统计计算结果，确定所述第一图像层的第二图像块位移，所述初始图像块位移包括所述第二图像块位移。
在其中一个实施例中，所述计算第一图像层的初始图像块位移，包括：
确定所述第一图像层中已确定图像块位移的图像块；
根据所述已确定图像块位移的图像块，确定所述第一图像层的第三图像块位移，所述初始图像块位移包括所述第三图像块位移。
在其中一个实施例中，所述计算所述初始图像块位移对应的初始衡量指标值，包括：
确定所述第二图像层中，与所述第一图像层的初始图像块位移相同的第二图像块；
根据所述第一图像层中第一图像块的像素值，以及所述第二图像块的像素值，计算所述初始图像块位移对应的初始衡量指标值。
在其中一个实施例中，所述根据各所述图像层位移以及所述图像层位移对应的衡量指标值，将所述相邻图像与所述待处理图像对齐，包括：
基于所述初始衡量指标值，从所述初始图像块位移中确定所述第一图像层中第一图像块的图像块位移；
基于所述第一图像层中第一图像块的图像块位移，确定所述第一图像金字塔与所述第二图像金字塔对应每层之间的图像层位移，以及所述图像层位移对应的衡量指标值；
根据各所述图像层位移及所述图像层位移对应的衡量指标值，将所述相邻图像与所述待处理图像对齐。
在其中一个实施例中，在所述基于所述第一图像层中第一图像块的图像块位移，确定所述第一图像金字塔与所述第二图像金字塔对应每层之间的图像层位移之后，还包括：
确定所述第一图像层中与所述第一图像块相邻的预设邻域图像块；
根据各所述预设邻域图像块的图像块位移及对应的衡量指标值，更新所述第一图像层中第一图像块的图像块位移，确定更新后的图像层位移。
在其中一个实施例中，所述融合所述相邻图像与所述待处理图像，获取处理后的图像，包括：
根据所述更新后的图像层位移对应的衡量指标值，确定所述相邻图像的初始权重；
将所述待处理图像的预设权重与所述相邻图像的初始权重进行融合，得到所述相邻图像与所述待处理图像的融合权重；
基于所述融合权重，融合所述相邻图像与所述待处理图像，获取处理后的图像。
在其中一个实施例中，在所述获取处理后的图像之后，还包括：
根据预设的像素值波动范围，对所述处理后的图像进行校正。
一种图像处理装置，所述装置包括：
图像获取模块，用于获取待处理图像及所述待处理图像对应的相邻图像；
图像金字塔构建模块，用于分别构建所述待处理图像和所述相邻图像的图像金字塔，所述待处理 图像对应的图像金字塔为第一图像金字塔，所述相邻图像对应的图像金字塔为第二图像金字塔；
计算模块，用于计算所述第一图像金字塔与所述第二图像金字塔对应每层之间的图像层位移，以及所述图像层位移对应的衡量指标值；
图像对齐模块，用于根据各所述图像层位移以及所述图像层位移对应的衡量指标值，将所述相邻图像与所述待处理图像对齐；
图像融合模块，用于融合所述相邻图像与所述待处理图像，获取处理后的图像。
一种计算机设备，包括存储器和处理器，所述存储器存储有计算机程序，所述处理器执行所述计算机程序时实现上述的图像处理方法的步骤。
一种计算机可读存储介质，其上存储有计算机程序，所述计算机程序被处理器执行时实现上述的图像处理方法的步骤。
有益效果
上述图像处理方法、装置、计算机设备和存储介质，通过获取待处理图像及待处理图像对应的相邻图像；分别构建待处理图像和相邻图像的图像金字塔，待处理图像对应的图像金字塔为第一图像金字塔，相邻图像对应的图像金字塔为第二图像金字塔；计算第一图像金字塔与第二图像金字塔对应每层之间的图像层位移，以及图像层位移对应的衡量指标值；根据各图像层位移以及图像层位移对应的衡量指标值，将相邻图像与待处理图像对齐；融合相邻图像与待处理图像，获取处理后的图像。采用上述实施例方法，通过构建待处理图像和相邻图像的图像金字塔之后，计算图像金字塔对应每层之间的图像层位移，能够提高位移计算的准确性；根据各图像层位移将相邻图像与待处理图像对齐，能够提高相邻图像与待处理图像的对齐程度；将对齐后的相邻图像与待处理图像进行融合，得到处理后的图像，能够有效减少图像噪声，提升图像画质，从而提高图像质量。
附图说明
图1为一个实施例中图像处理方法的应用环境图；
图2为一个实施例中图像处理方法的流程示意图；
图3为一个具体实施例中图像处理方法的示意图；
图4为一个具体实施例中待处理图像的示意图；
图5为一个具体实施例中画质优化处理的示意图；
图6为一个具体实施例中计算图像层位移的示意图；
图7为一个实施例中图像处理装置的结构框图；
图8为一个实施例中计算机设备的内部结构图；
图9为另一个实施例中计算机设备的内部结构图。
本发明的实施方式
为了使本申请的目的、技术方案及优点更加清楚明白，以下结合附图及实施例，对本申请进行进一步详细说明。应当理解，此处描述的具体实施例仅仅用以解释本申请，并不用于限定本申请。
在其中一个实施例中，本申请提供的图像处理方法，可以应用于如图1所示的应用环境中。该应用环境同时涉及终端102和服务器104。终端102可以通过网络或协议等其他通信方式与服务器104进行通信。具体地，服务器104通过终端102获取待处理图像及待处理图像对应的相邻图像，分别构建待处理图像和相邻图像的图像金字塔，待处理图像对应的图像金字塔为第一图像金字塔，相邻图像对应的图像金字塔为第二图像金字塔；计算第一图像金字塔与第二图像金字塔对应每层之间的图像层位移，以及图像层位移对应的衡量指标值；根据各图像层位移以及图像层位移对应的衡量指标值，将相邻图像与待处理图像对齐；融合相邻图像与待处理图像，获取处理后的图像。
在其中一个实施例中，本申请提供的图像处理方法，其应用环境可以只涉及终端102。具体地，终端102可以直接获取待处理图像及待处理图像对应的相邻图像，在终端102中对待处理图像和相邻图像进行图像处理，最终获取处理后的图像。
在其中一个实施例中，本申请提供的图像处理方法，其应用环境可以只涉及服务器104。具体地，服务器104可以直接获取待处理图像及待处理图像对应的相邻图像，在服务器104中对待处理图像和相邻图像进行图像处理，最终获取处理后的图像。
其中，终端102可以但不限于是各种相机、个人计算机、笔记本电脑、智能手机、平板电脑和便携式可穿戴设备等。服务器104可以用独立的服务器或者是多个服务器组成的服务器集群来实现。
在其中一个实施例中，如图2所示，提供了一种图像处理方法，以该方法应用于图1中的终端102和/或服务器104为例进行说明，包括以下步骤：
步骤S202，获取待处理图像及待处理图像对应的相邻图像。
在其中一个实施例中，图像是指由拍摄装置拍摄得到的图像。其中，拍摄装置可以是相机，也可以是摄像头等。将需要进行处理的图像称为待处理图像，将与待处理图像相邻的图像称为相邻图像，相邻图像可以为一张图像，也可以为多张图像。
在其中一个实施例中，当拍摄装置为相机时，相机可以直接拍摄得到图像，将拍摄得到的、需要进行处理的图像称为待处理图像，将拍摄得到的、待处理图像的前一张或者多张图像称为待处理图像对应的相邻图像，或者，将拍摄得到的、待处理图像的后一张或者多张图像称为待处理图像对应的相邻图像。
在其中一个实施例中，当拍摄装置为摄像机时，摄像机可以拍摄得到视频，需要从视频中提取获得视频帧图像。将提取的、需要进行处理的视频帧图像称为待处理图像，将提取的、待处理图像的前一帧或者多帧图像称为待处理图像对应的相邻图像，或者，将提取的、待处理图像的后一帧或者多帧图像称为待处理图像对应的相邻图像。其中，视频可以是历史视频，也可以是实时视频，图像可以是历史视频帧图像，也可以是实时视频帧图像。
步骤S204，分别构建待处理图像和相邻图像的图像金字塔，待处理图像对应的图像金字塔为第一图像金字塔，相邻图像对应的图像金字塔为第二图像金字塔。
在其中一个实施例中，可以对待处理图像和相邻图像分别进行降采样或上采样，得到图像金字塔，例如高斯金字塔或者拉普拉斯金字塔等。待处理图像对应的图像金字塔为第一图像金字塔，相邻图像对应的图像金字塔为第二图像金字塔。其中，降采样(Down sampling)是指对图像进行下采样。在降采样或者上采样之后，可以得到图像对应的图像金字塔。图像金字塔是一种以多分辨率来描述图像的结构，是一系列以金字塔形状排列的分辨率逐步降低、且来源于同一张图像的集合。图像金字塔的层级越高，则图像越小、分辨率越低。
在其中一个实施例中，降采样的方式有多种，可以任意选取其中的至少一种。具体地，可以使用OpenCV软件库进行降采样，或者，可以直接对图像进行降采样，或者，可以使用改进的加权降采样方式进行降采样。其中，OpenCV软件库是一种计算机视觉和机器学习软件库，可以运行在Linux、Windows、Android和Mac OS等操作系统上，可以实现计算机视觉和图像处理等方面的通用算法。
在其中一个实施例中，在对待处理图像和相邻图像进行降采样的方式相同。具体地，对待处理图像进行降采样，得到第一图像金字塔，第一图像金字塔包括各第一图像层。对相邻图像进行降采样，得到第二图像金字塔，第二图像金字塔包括各第二图像层。由于降采样的方式相同，且待处理图像与相邻图像为同一个拍摄装置得到的图像，因此在对待处理图像和相邻图像进行相同方式的降采样后，得到的各第一图像层分别与各第二图像层相对应。
步骤S206，计算第一图像金字塔与第二图像金字塔对应每层之间的图像层位移，以及图像层位移对应的衡量指标值。
在其中一个实施例中，由于待处理图像与相邻图像之间存在偏移，因此需要计算待处理图像与相邻图像之间的位移，根据位移将待处理图像与相邻图像对齐。具体地，在降采样之后，可以将待处理图像与相邻图像之间的偏移，分解成第一图像金字塔与第二图像金字塔对应每层之间的图像层位移，通过图像层位移确定待处理图像与相邻图像之间的位移。
在其中一个实施例中，当待处理图像的相邻图像为一张图像，则将相邻图像与待处理图像对齐 时，相邻图像与待处理图像之间的位移只需计算一次。在待处理图像的相邻图像为多张图像，则将相邻图像与待处理图像对齐时，相邻图像与待处理图像之间的位移需计算多次。例如，待处理图像的相邻图像为两张图像，分别是待处理图像的前一张图像和后一张图像，将此三张图像对齐时，则需计算前一张图像与待处理图像的位移，从而将前一张图像与待处理图像对齐，还需计算待处理图像与后一张图像的位移，从而将后一张图像与待处理图像对齐，即，需要计算两次位移。
在其中一个实施例中，在计算第一图像金字塔与第二图像金字塔对应每层之间的图像层位移时，还需计算对应每层之间的图像层位移对应的衡量指标值。其中，衡量指标值可以衡量图像层位移的好坏程度，图像层位移与衡量指标值相对应。衡量指标的数值越小，其对应的图像层位移就越优。当采用不同的计算方式，得到对应每层的不同的图像层位移时，可以根据衡量指标值确定对应每层的最优的图像层位移。
步骤S208，根据各图像层位移以及图像层位移对应的衡量指标值，将相邻图像与待处理图像对齐。
在其中一个实施例中，在获得各图像层位移以及图像层位移对应的衡量指标值之后，可以根据各图像层位移以及图像层位移对应的衡量指标值，将相邻图像与待处理图像对齐。具体地，可以一层一层的将相邻图像与待处理图像对齐，以提高图像的对齐程度，使得对齐后的相邻图像与待处理图像基本一致。
步骤S210，融合相邻图像与待处理图像，获取处理后的图像。
在其中一个实施例中，图像融合是指将处理后的几张图像融合成一张图像，能够提高图像的精度，提升原始图像的分辨率等。在将相邻图像与待处理图像对齐之后，融合相邻图像与待处理图像。具体地，可以基于待处理图像和相邻图像的权重进行融合。融合之后得到一张融合后的图像，称为处理后的图像。
上述图像处理方法中，通过获取待处理图像及待处理图像对应的相邻图像；分别构建待处理图像和相邻图像的图像金字塔，待处理图像对应的图像金字塔为第一图像金字塔，相邻图像对应的图像金字塔为第二图像金字塔；计算第一图像金字塔与第二图像金字塔对应每层之间的图像层位移，以及图像层位移对应的衡量指标值；根据各图像层位移以及图像层位移对应的衡量指标值，将相邻图像与待处理图像对齐；融合相邻图像与待处理图像，获取处理后的图像。采用上述实施例方法，通过构建待处理图像和相邻图像的图像金字塔之后，计算图像金字塔对应每层之间的图像层位移，能够提高位移计算的准确性；根据各图像层位移将相邻图像与待处理图像对齐，能够提高相邻图像与待处理图像的对齐程度；将对齐后的相邻图像与待处理图像进行融合，得到处理后的图像，能够有效减少图像噪声，提升图像画质，从而提高图像质量。
在其中一个实施例中，在步骤S202获取待处理图像及待处理图像对应的相邻图像之后，在步骤S204分别构建待处理图像和相邻图像的图像金字塔之前，还包括：
步骤S302，对待处理图像和相邻图像进行画质优化处理，得到画质优化处理处理后的待处理图像和相邻图像。
在其中一个实施例中，可以采用图像对比度处理、图像锐化等方式，分别对待处理图像和相邻图像进行画质优化处理。具体地，可以采用图像对比度处理的方式。其中，图像对比度是指图像灰度反差的大小。当图像为在夜晚拍摄的图像时，由于夜晚的能见度低，导致图像整体较暗。因此可以降低图像的对比度，以提升图像的暗部区域细节。另外，由于夜晚的光照条件复杂，多存在路灯、霓虹灯等灯光的影响，因此图像中还不可避免的出现亮度高于周边亮度的区域，称为高光区域。在降低图像的对比度，提升图像的暗部区域细节之后，会导致高光区域丢失，因此又需要提升图像的对比度，以对图像高光区域进行还原。
在其中一个实施例中，对待处理图像和相邻图像进行画质优化处理的方式相同。下述实施例以待处理图像的处理过程为例，可以通过曲线映射的方式对待处理图像进行对比度处理。其中，可以使用查找表(Look-up table)改善图像质量，提升暗部区域细节，降低图像的对比度，并对图像高光区域进行还原，提升图像的对比度。具体地，可以使用图像处理软件进行查找表的操作，例如图像处理软件PS中的曝光调节、色阶映射等。其中，还可以通过暗通道去雾方式，对图像高光区域进行还原，提升图像的对比度。
在其中一个实施例中，可以将对比度处理后的待处理图像与原始的待处理图像加权，得到具有更多高光区域细节的待处理图像。具体地，可以加权待处理图像与对比度处理后的待处理图像，获得待处理图像对应的权重图。其中，由于图像的高光区域至普通区域的亮度是渐变的，因此，可以将原始的待处理图像的像素值与预设倍数相乘，并进行模糊处理，得到待处理图像对应的权重图。其中，可以将权重图表示为W，预设倍数可以是1.5倍、2倍或3倍等。模糊处理可以是均值模糊处理，其半径可以是21。还可以使用图像处理软件得到待处理图像对应的权重图。具体地，可以使用图像处理软件PS中的高光提取等。
在其中一个实施例中，可以使用预设公式还原待处理图像对应的权重图的高光区域细节，得到画质优化处理后的待处理图像。具体地，预设公式如下：
Figure PCTCN2022097077-appb-000001
式中，W(y,x)表示待处理图像对应的权重图W中像素点(y,x)处的值，r、g、b表示原始的待处理 图像的红绿蓝三通道的像素值，r new、g new、b new表示对比度处理后的待处理图像的红绿蓝三通道的像素值。
在其中一个实施例中，在步骤S202获取待处理图像及待处理图像对应的相邻图像之后，在步骤S204分别构建待处理图像和相邻图像的图像金字塔之前，还包括：
步骤S402，对待处理图像和相邻图像进行一次滤波处理，得到一次滤波处理后的待处理图像和相邻图像。
其中，脉冲噪声是图像中常见的噪声，表现为图像中随机出现的白点或者黑点，可能是高光区域存在黑色的像素点，或者是暗部区域存在白色的像素点，或者是两者皆存在。脉冲噪声的成因可能是影像信号受到强烈干扰、模数转换器或位元传输错误等。一般情况下使用滤波处理的方式去除脉冲噪声，例如，中值滤波处理，或者其他去除孤立点的滤波处理等。在拍摄装置将原始数据转成为图像时，会将脉冲噪声扩大成几个像素，需要进行多次滤波处理。然而，多次滤波处理会使图像模糊，因此，需要对图像进行有针对性的滤波处理，例如，针对图像的平坦区域进行滤波处理，或者针对图像的运动区域进行滤波处理等，以有效减少图像的噪声。
在其中一个实施例中，对待处理图像和相邻图像进行滤波处理的方式相同。对整张图像进行全局滤波处理，将全局滤波处理称为一次滤波处理。具体地，对待处理图像和相邻图像进行一次滤波处理，得到一次滤波处理后的待处理图像和相邻图像。其中，可以使用中值滤波处理，或者改进的中值滤波处理等方式。
步骤S404，根据一次滤波处理后的待处理图像和相邻图像的像素值，确定一次滤波处理后的待处理图像和相邻图像中的运动区域。
在其中一个实施例中，由于图像的非运动区域的噪声可以通过后续的图像的对齐和融合过程去除，因此，可以针对图像的运动区域进行滤波处理，需要确定图像的运动区域。其中，根据一次滤波处理后的待处理图像和相邻图像的像素值，确定一次滤波处理后的待处理图像和相邻图像中的运动区域。
具体地，在待处理图像的当前区域与相邻图像的对应区域的像素值之差大于预设运动阈值时，则将该当前区域确定为运动区域，并将该运动区域阈值设置为第一预设值。在待处理图像的当前区域与相邻图像的对应区域的像素值之差小于预设运动阈值时，则将该当前区域确定为非运动区域，并将该非运动区域阈值设置为第二预设值。其中，预设运动阈值的取值范围是(0，255)，可以具体设置为35。运动区域阈值，即第一预设值可以设置为255，非运动区域阈值，即第二预设值可以设置为0。
步骤S406，计算一次滤波处理后的待处理图像和相邻图像的边界值。
在其中一个实施例中，由于对图像的边界进行滤波处理时会超出图像区域，因此，滤波处理不应 在图像的边界处，需要计算一次滤波处理后的待处理图像和相邻图像的边界值。其中，可以通过索贝尔算子(Sobel operator)，或者边缘检测算法等，计算一次滤波处理后的待处理图像和相邻图像的边界值。
具体地，在待处理图像的当前区域计算得到的边界值大于预设边界阈值时，则将该当前区域确定为边界区域，并将该边界区域阈值设置为第一预设值。在待处理图像的当前区域计算得到的边界值小于预设边界阈值时，则将该当前区域确定为非边界区域，并将该非边界区域阈值设置为第二预设值。其中，预设边界阈值的取值范围是(0，255)，可以具体设置为35。边界区域阈值，即第一预设值可以设置为255，非边界区域阈值，即第二预设值可以设置为0。
步骤S408，基于边界值，确定一次滤波处理后的待处理图像和相邻图像中的非边界的运动区域。
具体地，根据上述计算得到的非边界区域阈值以及运动区域阈值，确定一次滤波处理后的待处理图像和相邻图像中的非边界的运动区域。
步骤S410，对非边界的运动区域进行二次滤波处理，得到噪声处理后的待处理图像和相邻图像。
其中，在确定待处理图像和相邻图像的非边界的运动区域之后，对非边界的运动区域进行滤波处理，称为二次滤波处理。二次滤波处理与一次滤波处理的方式可以相同。具体地，可以采用中值滤波处理，或者改进的中值滤波处理等方式，得到噪声处理后的待处理图像和相邻图像，并应用于后续计算。
在其中一个实施例中，步骤S206计算第一图像金字塔与第二图像金字塔对应每层之间的图像层位移，包括：
步骤S502，计算第一图像层的初始图像块位移，初始图像块位移为第一图像层中的第一图像块与对应的第二图像层中的第二图像块之间的位移，第一图像层为第一图像金字塔中的图像层，第二图像层为第二图像金字塔中的图像层，第一图像块为第一图像层中预设大小的图像块，第二图像块为第二图像层中预设大小的图像块。
在其中一个实施例中，可以通过计算图像金字塔对应的每一层图像层的每一个图像块的图像块位移来确定图像层位移，以提高计算的准确性。其中，在对待处理图像进行降采样之后，得到各第一图像层。同理，在对相邻图像进行相同方式的降采样之后，得到各第二图像层。将第一图像层中预设大小的图像块称为第一图像块，获得预设大小的各第一图像块。同理，将第二图像层中的预设大小图像块称为第二图像块，获得预设大小的各第二图像块。由于第一图像层与第二图像层相对应，因此得到的预设大小的第一图像块与第二图像块相对应。将计算得到的第一图像块与对应的第二图像块之间的位移，称为初始图像块位移。其中，在计算对齐位移时，按照每一层图像层的每一个图像块进行计算。 在计算第一图像层的初始图像块位移之前，需要将最小分辨率图像层的位移进行初始化，使其图像块位移为0。
步骤S504，计算初始图像块位移对应的初始衡量指标值。
在其中一个实施例中，可以采用多种计算方式，计算得到第一图像层的多种初始图像块位移，进而根据对应的衡量指标值确定最优的一个图像块位移。具体地，计算初始图像块位移对应的初始衡量指标值。其中，初始图像块位移与初始衡量指标值对应。初始衡量指标值可以衡量计算所得的初始图像块位移的好坏程度，可以从多个初始图像块位移中，确定最优的一个图像块位移。
步骤S506，根据初始图像块位移和初始衡量指标值，确定第一图像金字塔与第二图像金字塔对应每层之间的图像层位移。
在其中一个实施例中，初始衡量指标的数值越小，其对应的图像块位移就越优。具体地，可以根据初始图像块位移和初始衡量指标值，将数值最小的初始衡量指标值对应的初始图像块位移确定为最优的第一图像层的图像块位移。进而，通过第一图像层的图像块位移，确定第一图像金字塔与第二图像金字塔对应每层之间的图像层位移。
在其中一个实施例中，步骤S502计算第一图像层的初始图像块位移，包括：
步骤S602，对第一图像层的前一层图像层的图像层位移进行插值计算，获得插值计算结果，前一层图像层为第一图像金字塔中，与第一图像层相邻、且分辨率低于第一图像层的图像层。
在其中一个实施例中，在对待处理图像进行降采样后，得到第一图像金字塔。在第一图像金字塔中，将与第一图像层相邻、且分辨率低于第一图像层，即层级高的一层图像层作为前一层图像层。前一层图像层的图像层位移，本质上是基于最小分辨率的图像层的图像层位移得到。
在其中一个实施例中，对第一图像层的前一层图像层的图像层位移进行插值计算，获得插值计算结果。其中，插值计算可以采用拉格朗日插值、牛顿插值、分段插值、双线性插值等方式。
步骤S604，根据插值计算结果，确定第一图像层的第一图像块位移，初始图像块位移包括第一图像块位移。
在其中一个实施例中，由于第一图像金字塔中的各第一图像层的分辨率是成比例的，因此，前一层图像层的图像层位移的插值计算结果需经过相对应的分辨率放大后，才可以作为第一图像层的第一图像块的图像块位移，使用本实施例步骤确定的图像块位移称为第一图像块位移。其中，分辨率放大的倍数为第一图像层与前一层图像层的分辨率比值。例如，第一图像层的分辨率是前一层图像层的分辨率的2倍，即对应的分辨率放大倍数为2，则插值计算结果需乘以2才可以作为第一图像层的第一图像块的图像块位移。
在其中一个实施例中，步骤S502计算第一图像层的初始图像块位移，包括：
步骤S702，对第一图像层的前一层图像层的图像层位移进行统计计算，获得统计计算结果，前一层图像层为第一图像金字塔中，与第一图像层相邻、且分辨率低于第一图像层的第一图像层。
在其中一个实施例中，可以统计前一层图像层的图像层位移中各图像块位移的众数、均值或者中位数的至少一种，作为统计计算结果。其中，前一层图像层为各第一图像层中，与第一图像层相邻、且分辨率低于第一图像层的图像层。
步骤S704，根据统计计算结果，确定第一图像层的第二图像块位移，初始图像块位移包括第二图像块位移。
在其中一个实施例中，前一层图像层的图像层位移的统计计算结果需经过相对应的分辨率放大后，才可以作为第一图像层的第一图像块的图像块位移，使用本实施例步骤确定的图像块位移称为第二图像块位移。
在其中一个实施例中，步骤S502计算第一图像层的初始图像块位移，包括：
步骤S802，确定第一图像层中已确定图像块位移的图像块。
在其中一个实施例中，第一图像层的初始图像块位移，还可以基于第一图像层中已确定图像块位移的图像块进行确定。其中，已确定图像块位移的图像块还需是与第一图像块相邻的图像块。
在其中一个实施例中，在计算第一图像层的各第一图像块的图像块位移时，可以按照预设方式分别对各第一图像块进行计算。预设方式可以为按照蛇形形状进行计算，具体为第一图像层的第一行的图像块从左往右计算，第二行的图像块从右往左计算，依次循环往复。已确定图像块位移的图像块，其图像块位移可以根据步骤S602-步骤S604、步骤S702-步骤S704确定，也可以根据本实施例步骤确定。具体地，在第一图像层中与第一图像块相邻、且已确定图像块位移的图像块为预设个数图像块。其中，预设个数可以为多个，例如2个、3个或者4个等。
步骤S804，根据已确定图像块位移的图像块，确定第一图像层的第三图像块位移，初始图像块位移包括第三图像块位移。
在其中一个实施例中，可以根据已确定图像块位移的图像块，确定第一图像层的各第一图像块的图像块位移，采用本实施例步骤确定的图像块位移称为第三图像块位移。
在其中一个实施例中，在第一图像层中只有一个第一图像块，或者，在第一图像层中，需要确定图像块位移的图像块为该第一图像层中的第一个计算的图像块，即该第一图像块不存在任何与其相邻、且已确定图像块位移的图像块。则在此种情况下，第一图像块的图像块位移仅可以根据步骤S602-步骤S604或者步骤S702-步骤S704确定。
在其中一个实施例中，步骤S504计算初始图像块位移对应的初始衡量指标值，包括：
步骤S902，确定第二图像层中，与第一图像层的初始图像块位移相同的第二图像块。
在其中一个实施例中，第二图像层是指与第一图像层对应的第二图像层。其中，可以按照第一图像层的第一图像块的初始图像块位移，去确定第二图像层中对应的位移的第二图像块。具体地，可以确定与第一图像层对应的第二图像层，在第二图像层中，确定与第一图像层的第一图像块的初始图像块位移相同的第二图像块。
步骤S904，根据第一图像层中第一图像块的像素值，以及第二图像块的像素值，计算初始图像块位移对应的初始衡量指标值。
在其中一个实施例中，可以将两个图像块之间像素值的绝对差之和作为初始衡量指标，或者，可以将两个图像块之间的相关性作为初始衡量指标。具体地，在将两个图像块之间像素值的绝对差之和作为初始衡量指标时，可以根据第一图像层中第一图像块的像素值，以及上述初始图像块位移相同的第二图像块的像素值，计算初始图像块位移对应的初始衡量指标值。
在其中一个实施例中，步骤S208根据各图像层位移以及图像层位移对应的衡量指标值，将相邻图像与待处理图像对齐，包括：
步骤S1002，基于初始衡量指标值，从初始图像块位移中确定第一图像层中第一图像块的图像块位移。
在其中一个实施例中，可以根据初始衡量指标值，将数值最小的初始衡量指标值对应的第一图像块位移、第二图像块位移和第三图像块位移中，确定最优的第一图像层中第一图像块的图像块位移。
步骤S1004，基于第一图像层中第一图像块的图像块位移，确定第一图像金字塔与第二图像金字塔对应每层之间的图像层位移，以及图像层位移对应的衡量指标值。
在其中一个实施例中，通过第一图像层中第一图像块的图像块位移，确定第一图像金字塔与第二图像金字塔对应每层之间的图像层位移。在确定了对应每层之间的图像层位移之后，确定图像层位移对应的衡量指标值。
步骤S1006，根据各图像层位移及图像层位移对应的衡量指标值，将相邻图像与待处理图像对齐。
在其中一个实施例中，在确定了各图像层位移及图像层位移对应的衡量指标值之后，可以根据确定的图像层位移，将相邻图像与待处理图像对齐。
在其中一个实施例中，在步骤S1004基于第一图像层中第一图像块的图像块位移，确定第一图像金字塔与第二图像金字塔对应每层之间的图像层位移之后，还可以根据第一图像块四周的图像块位移对图像层位移进行更新，包括：
步骤S1102，确定第一图像层中与第一图像块相邻的预设邻域图像块。
在其中一个实施例中，第一图像层的各第一图像块的图像块位移可以只根据步骤S602-步骤S604、步骤S702-步骤S704或者步骤S802-步骤S804中的一种方式确定，还可以同时根据上述三种方式确定。其中，确定第一图像层中与当第一图像块相邻的预设邻域图像块。具体地，预设邻域图像块是第一图像块四周的、且已确定图像块位移的图像块。其中，预设邻域图像块的数量可以为多个，例如第一图像块的四周上下左右4个、四周一圈的8个或者四周向外两圈的24个等，其已经确定的图像块位移，可以是根据步骤S602-步骤S604、步骤S702-步骤S704或者步骤S802-步骤S804中的一种方式确定的。
步骤S1104，根据各预设邻域图像块的图像块位移及对应的衡量指标值，更新第一图像层中第一图像块的图像块位移，确定更新后的图像层位移。
在其中一个实施例中，可以根据各预设邻域图像块已确定的图像块位移及对应的衡量指标值，更新第一图像层中第一图像块的图像块位移。具体地，在第一图像块的预设邻域图像块存在更小、更优的衡量指标值时，则将第一图像块的图像块位移更新为更优的衡量指标值对应的图像块位移，否则不更新。
在其中一个实施例中，步骤S210融合相邻图像与待处理图像，获取处理后的图像，包括：
步骤S1202，根据更新后的图像层位移对应的衡量指标值，确定相邻图像的初始权重。
在其中一个实施例中，在将相邻图像与待处理图像对齐之后，需要将相邻图像和待处理图像融合为一张降噪处理后的图像。其中，可以基于图像的权重进行图像融合。由于初始图像块位移与初始衡量指标值对应，因此，在确定第一图像层的第一图像块的图像块位移后，仅存在一个对应的初始衡量指标值，称为衡量指标值。
在其中一个实施例中，根据预设的衡量指标值与阈值之间的关系，以及第一图像层的第一图像块的衡量指标值，确定与待处理图像对应的相邻图像的第二图像块的初始权重，也就是确定相邻图像的初始权重。具体地，确定初始权重的方式为，当第二图像块的阈值大于衡量指标值时，使该第二图像块的初始权重为0，此时待处理图像与相邻图像对齐不佳；当第二图像块的阈值小于衡量指标值时，使该第二图像块的初始权重＝1-衡量指标值/阈值。
在其中一个实施例中，为了提高图像融合的质量，得到更高的初始权重，确定初始权重的方式还可以为，当第二图像块的阈值大于或等于预设较大阈值时，使该第二图像块的初始权重为0，当第二图像块的阈值小于预设较小阈值时，使该第二图像块的初始权重为1，当第二图像块的阈值在在预设较小阈值和预设较大阈值之间时，使其初始权重依次线性减少。
步骤S1204，将待处理图像的预设权重与相邻图像的初始权重进行融合，得到相邻图像与待处理图像的融合权重。
在其中一个实施例中，将待处理图像的预设权重与对应的相邻图像的初始权重加权求和，得到相邻图像与待处理图像的融合权重。其中，待处理图像的预设权重可以设置为1。
步骤S1206，基于融合权重，融合相邻图像与待处理图像，获取处理后的图像。
在其中一个实施例中，在基于融合权重，融合相邻图像与待处理图像，获取处理后的图像之后，为了提高处理后的图像与原始的图像之间的相关度，处理后的图像的像素值应当在原始的图像的像素值的取值范围内。
在其中一个实施例中，在获取处理后的图像之后，还包括：根据预设的像素值波动范围，对处理后的图像进行校正。其中，预先设置处理后的图像的像素值波动范围。具体地，可以将像素值波动范围设置为±10。例如，原始的图像中的像素值为50，根据融合权重将相邻图像与待处理图像进行融合，得到处理后的图像后，若图像的像素值为62，则需要校正为60，若图像的像素值为56，则无需校正。
为了使本申请的目的、技术方案及优点更加清楚明白，以下结合附图及其中一个具体实施例，对本申请进行进一步详细说明。应当理解，此处描述的具体实施例仅仅用以解释本申请，并不用于限定本申请。
在其中一个具体实施例中，图像处理方法的示意图如图3所示，其中，在对图像进行预处理后，得到预处理后的图像，再通过对齐和融合多帧图像，获取一帧处理后的图像。待处理图像的示意图如图4所示，其中，图像中存在明显的脉冲噪声。图像处理方法的具体步骤如下：
一、图像预处理
确定待处理图像，以及待处理图像对应的相邻图像。在对图像进行多帧降噪处理之前，还可以包括图像预处理，其中，图像预处理主要包括：画质优化处理和噪声处理中的至少一种，以待处理图像的预处理过程为例，具体步骤包括：
画质优化预处理：
使用如图5(a)所示的曲线映射方式降低图像的对比度，提升图像的暗部区域细节，通过如图5(b)所示的曲线映射方式提升图像的对比度，以对图像的高光区域进行还原。
将原始的待处理图像的像素值乘以2倍，并进行半径为21的均值模糊处理，得到待处理图像对应的权重图W。使用公式还原权重图的高光细节，得到画质优化处理后的待处理图像，公式如下：
Figure PCTCN2022097077-appb-000002
式中，W(y,x)表示待处理图像对应的权重图W中像素点(y,x)处的值，r、g、b表示原始的待处理图像的红绿蓝三通道的像素值，r new、g new、b new表示对比度处理后的待处理图像的红绿蓝三通道的像素值。
噪声预处理：
对待处理图像进行全局滤波处理，得到全局滤波处理后的待处理图像。
确定全局滤波处理后的待处理图像中的运动区域，将运动区域阈值设置为255，非运动区域阈值设置为0。
通过Sobel算子或者边缘检测算法，计算全局滤波处理后的待处理图像的边界值，将边界区域阈值设置为255，非边界区域阈值设置为0。
确定全局滤波处理后的待处理图像中的非边界的运动区域并进行滤波处理，得到噪声处理后的待处理图像。
二、多帧图像降噪处理
多帧图像降噪处理主要包括：待处理图像与相邻图像的对齐过程，以及对齐后的图像的融合过程，具体步骤包括：
图像的对齐过程：
对待处理图像与相邻图像分别进行降采样，得到待处理图像对应的第一图像金字塔，其中包括各第一图像层；得到相邻图像对应的第二图像金字塔，其中包括的各第二图像层，按照预设大小对第一图像层和第二图像层进行分割，分别得到各第一图像块和第二图像块。
通过第一图像块与对应的第二图像块之间的初始图像块位移，以及初始图像块位移对应的初始衡量指标值，确定第一图像金字塔与第二图像金字塔对应每层之间的图像层位移。
其中，计算初始图像块位移的方式包括：
方式①：对第一图像层的前一层图像层的图像层位移进行插值计算，获得插值计算结果，前一层图像层为第一图像金字塔中，与第一图像层相邻、且分辨率低于第一图像层的图像层，根据插值计算结果，确定第一图像层的第一图像块的第一图像块位移，初始图像块位移包括第一图像块位移。
方式②：对第一图像层的前一层图像层的图像层位移进行统计计算，获得统计计算结果，根据统计计算结果，确定第一图像层的第一图像块的第二图像块位移，初始图像块位移包括第二图像块位移。
方式③：确定第一图像层中与第一图像块相邻、且已确定图像块位移的图像块，具体如图6(a)所示，箭头表示计算顺序，●表示第一图像层的第一图像块，×表示与第一图像块相邻、且已确定图像块位移的3个图像块，根据已确定图像块位移的图像块，确定第一图像层的第一图像块的第三图像 块位移，初始图像块位移包括第三图像块位移。
确定第一图像层对应的第二图像层中，与第一图像层的初始图像块位移相同的第二图像块，并根据第一图像块的像素值，以及第二图像块的像素值，计算初始图像块位移对应的初始衡量指标值，将数值最小的初始衡量指标值对应的初始图像块位移确定为第一图像块的图像块位移，基于此确定第一图像金字塔与第二图像金字塔对应每层之间的图像层位移。
在确定第一图像块的图像块位移之后，确定第一图像层中与第一图像块相邻的预设邻域图像块，具体如图6(b)所示，●表示第一图像层的第一图像块，×表示与第一图像块相邻的8个预设邻域图像块，根据各预设邻域图像块的图像块位移及对应的衡量指标值，更新第一图像块的图像块位移，基于此确定更新后的图像层位移。
图像的融合过程：
根据更新后的图像层位移对应的衡量指标值，确定相邻图像的第二图像块的初始权重，当第二图像块的阈值大于衡量指标值时，使该第二图像块的初始权重为0，当第二图像块的阈值小于衡量指标值时，使该第二图像块的初始权重＝1-衡量指标值/阈值。
待处理图像的第一图像块的预设权重为1，将其与相邻图像的第二图像块的初始权重进行融合，得到相邻图像与待处理图像的融合权重。
基于融合权重，融合相邻图像与待处理图像，获取处理后的图像，并根据预设的像素值波动范围，对处理后的图像进行校正。像素值波动范围设置为±10时，原始的图像的像素值为50，若处理后的图像的像素值为62，则需要校正为60，若处理后的图像的像素值为56，则无需校正。
至此，得到一帧降噪处理后的图像。
应该理解的是，虽然图2的流程图中的各个步骤按照箭头的指示依次显示，但是这些步骤并不是必然按照箭头指示的顺序依次执行。除非本文中有明确的说明，这些步骤的执行并没有严格的顺序限制，这些步骤可以以其它的顺序执行。而且，图2中的至少一部分步骤可以包括多个步骤或者多个阶段，这些步骤或者阶段并不必然是在同一时刻执行完成，而是可以在不同的时刻执行，这些步骤或者阶段的执行顺序也不必然是依次进行，而是可以与其它步骤或者其它步骤中的步骤或者阶段的至少一部分轮流或者交替地执行。
在其中一个实施例中，如图7所示，提供了一种图像处理装置，包括：图像获取模块710、图像金字塔构建模块720、计算模块730、图像对齐模块740和图像融合模块750，其中：
图像获取模块710，用于获取待处理图像及所述待处理图像对应的相邻图像。
图像金字塔构建模块720，用于分别构建所述待处理图像和所述相邻图像的图像金字塔，所述待 处理图像对应的图像金字塔为第一图像金字塔，所述相邻图像对应的图像金字塔为第二图像金字塔。
计算模块730，用于计算所述第一图像金字塔与所述第二图像金字塔对应每层之间的图像层位移，以及所述图像层位移对应的衡量指标值。
图像对齐模块740，用于根据各所述图像层位移以及所述图像层位移对应的衡量指标值，将所述相邻图像与所述待处理图像对齐。
图像融合模块750，用于融合所述相邻图像与所述待处理图像，获取处理后的图像。
在其中一个实施例中，图像处理装置还包括：
画质优化处理模块，用于在所述获取待处理图像及所述待处理图像对应的相邻图像之后，在所述分别构建所述待处理图像和所述相邻图像的图像金字塔之前，对所述待处理图像和所述相邻图像进行画质优化处理，得到画质优化处理处理后的待处理图像和相邻图像。
降噪处理模块，用于在所述获取待处理图像及所述待处理图像对应的相邻图像之后，在所述分别构建所述待处理图像和所述相邻图像的图像金字塔之前，对所述待处理图像和所述相邻图像进行降噪处理，得到噪声处理后的待处理图像和相邻图像。
在其中一个实施例中，降噪处理模块包括以下单元：
一次滤波处理单元，用于对所述待处理图像和所述相邻图像进行一次滤波处理，得到一次滤波处理后的待处理图像和相邻图像。
运动区域确定单元，用于根据所述一次滤波处理后的待处理图像和相邻图像的像素值，确定所述一次滤波处理后的待处理图像和相邻图像中的运动区域。
边界值计算单元，用于计算所述一次滤波处理后的待处理图像和相邻图像的边界值。
非边界运动区域确定单元，用于基于所述边界值，确定所述一次滤波处理后的待处理图像和相邻图像中的非边界的运动区域。
二次滤波处理单元，用于对所述非边界的运动区域进行二次滤波处理，得到噪声处理后的待处理图像和相邻图像。
在其中一个实施例中，计算模块730包括以下单元：
初始图像块位移计算单元，用于计算第一图像层的初始图像块位移，所述初始图像块位移为所述第一图像层中的第一图像块与对应的第二图像层中的第二图像块之间的位移，所述第一图像层为所述第一图像金字塔中的图像层，所述第二图像层为所述第二图像金字塔中的图像层，所述第一图像块为所述第一图像层中预设大小的图像块，所述第二图像块为所述第二图像层中所述预设大小的图像块。
初始衡量指标值计算单元，用于计算所述初始图像块位移对应的初始衡量指标值。
图像层位移确定单元，用于根据所述初始图像块位移和所述初始衡量指标值，确定所述第一图像金字塔与所述第二图像金字塔对应每层之间的图像层位移。
在其中一个实施例中，初始图像块位移计算单元包括第一图像块位移计算单元，第一图像块位移计算单元包括以下单元：
插值计算单元，用于对所述第一图像层的前一层图像层的图像层位移进行插值计算，获得插值计算结果，所述前一层图像层为所述第一图像金字塔中，与所述第一图像层相邻、且分辨率低于所述第一图像层的图像层。
第一图像块位移确定单元，用于根据所述插值计算结果，确定所述第一图像层的第一图像块位移，所述初始图像块位移包括所述第一图像块位移。
在其中一个实施例中，初始图像块位移计算单元包括第二图像块位移计算单元，第二图像块位移计算单元包括以下单元：
统计计算单元，用于对所述第一图像层的前一层图像层的图像层位移进行统计计算，获得统计计算结果，所述前一层图像层为所述第一图像金字塔中，与所述第一图像层相邻、且分辨率低于所述第一图像层的第一图像层。
第二图像块位移确定单元，用于根据所述统计计算结果，确定所述第一图像层的第二图像块位移，所述初始图像块位移包括所述第二图像块位移。
在其中一个实施例中，初始图像块位移计算单元包括第三图像块位移计算单元，第三图像块位移计算单元包括以下单元：
图像块确定单元，用于确定所述第一图像层中已确定图像块位移的图像块。
第三图像块位移确定单元，用于根据所述已确定图像块位移的图像块，确定所述第一图像层的第三图像块位移，所述初始图像块位移包括所述第三图像块位移。
在其中一个实施例中，初始衡量指标值计算单元包括以下单元：
第二图像块确定单元，用于确定所述第二图像层中，与所述第一图像层的初始图像块位移相同的第二图像块。
初始衡量指标值确定单元，用于根据所述第一图像层中第一图像块的像素值，以及所述第二图像块的像素值，计算所述初始图像块位移对应的初始衡量指标值。
在其中一个实施例中，图像对齐模块740包括以下单元：
图像块位移确定单元，用于基于所述初始衡量指标值，从所述初始图像块位移中确定所述第一图像层中第一图像块的图像块位移。
图像层位移确定单元，用于基于所述第一图像层中第一图像块的图像块位移，确定所述第一图像金字塔与所述第二图像金字塔对应每层之间的图像层位移，以及所述图像层位移对应的衡量指标值。
图像对齐单元，用于根据各所述图像层位移及所述图像层位移对应的衡量指标值，将所述相邻图像与所述待处理图像对齐。
在其中一个实施例中，计算单元还包括图像块位移更新单元，图像块位移更新单元包括以下单元：
图像块第二确定单元，用于在所述基于所述第一图像层中第一图像块的图像块位移，确定所述第一图像金字塔与所述第二图像金字塔对应每层之间的图像层位移之后，确定所述第一图像层中与所述第一图像块相邻的预设邻域图像块。
图像块位移更新单元，用于根据各所述预设邻域图像块的图像块位移及对应的衡量指标值，更新所述第一图像层中第一图像块的图像块位移，确定更新后的图像层位移。
在其中一个实施例中，图像融合模块760包括以下单元：
初始权重确定单元，用于根据所述更新后的图像层位移对应的衡量指标值，确定所述相邻图像的初始权重。
融合权重计算单元，用于将所述待处理图像的预设权重与所述相邻图像的初始权重进行融合，得到所述相邻图像与所述待处理图像的融合权重。
图像融合单元，用于基于所述融合权重，融合所述相邻图像与所述待处理图像，获取处理后的图像。
在其中一个实施例中，图像处理装置还包括：
图像校正单元，用于在所述获取处理后的图像之后，根据预设的像素值波动范围，对所述处理后的图像进行校正。
关于图像处理装置的具体限定可以参见上文中对于图像处理方法的限定，在此不再赘述。上述图像处理装置中的各个模块可全部或部分通过软件、硬件及其组合来实现。上述各模块可以硬件形式内嵌于或独立于计算机设备中的处理器中，也可以以软件形式存储于计算机设备中的存储器中，以便于处理器调用执行以上各个模块对应的操作。
在其中一个实施例中，提供了一种计算机设备，该计算机设备可以是服务器，其内部结构图可以如图8所示。该计算机设备包括通过系统总线连接的处理器、存储器和网络接口。其中，该计算机设备的处理器用于提供计算和控制能力。该计算机设备的存储器包括非易失性存储介质、内存储器。该非易失性存储介质存储有操作系统、计算机程序和数据库。该内存储器为非易失性存储介质中的操作系统和计算机程序的运行提供环境。该计算机设备的数据库用于存储图像处理数据。该计算机设备的 网络接口用于与外部的终端通过网络连接通信。该计算机程序被处理器执行时以实现一种图像处理方法。
在其中一个实施例中，提供了一种计算机设备，该计算机设备可以是终端，其内部结构图可以如图9所示。该计算机设备包括通过系统总线连接的处理器、存储器、通信接口、显示屏和输入装置。其中，该计算机设备的处理器用于提供计算和控制能力。该计算机设备的存储器包括非易失性存储介质、内存储器。该非易失性存储介质存储有操作系统和计算机程序。该内存储器为非易失性存储介质中的操作系统和计算机程序的运行提供环境。该计算机设备的通信接口用于与外部的终端进行有线或无线方式的通信，无线方式可通过WIFI、运营商网络、NFC(近场通信)或其他技术实现。该计算机程序被处理器执行时以实现一种图像处理方法。该计算机设备的显示屏可以是液晶显示屏或者电子墨水显示屏，该计算机设备的输入装置可以是显示屏上覆盖的触摸层，也可以是计算机设备外壳上设置的按键、轨迹球或触控板，还可以是外接的键盘、触控板或鼠标等。
本领域技术人员可以理解，图8-9中示出的结构，仅仅是与本申请方案相关的部分结构的框图，并不构成对本申请方案所应用于其上的计算机设备的限定，具体的计算机设备可以包括比图中所示更多或更少的部件，或者组合某些部件，或者具有不同的部件布置。
在其中一个实施例中，提供了一种计算机设备，包括存储器和处理器，存储器中存储有计算机程序，该处理器执行计算机程序时实现上述的图像处理方法的步骤。
在其中一个实施例中，提供了一种计算机可读存储介质，其上存储有计算机程序，计算机程序被处理器执行时实现上述的图像处理方法的步骤。
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程，是可以通过计算机程序来指令相关的硬件来完成，所述的计算机程序可存储于一非易失性计算机可读取存储介质中，该计算机程序在执行时，可包括如上述各方法的实施例的流程。其中，本申请所提供的各实施例中所使用的对存储器、存储、数据库或其它介质的任何引用，均可包括非易失性和易失性存储器中的至少一种。非易失性存储器可包括只读存储器(Read-Only Memory，ROM)、磁带、软盘、闪存或光存储器等。易失性存储器可包括随机存取存储器(Random Access Memory，RAM)或外部高速缓冲存储器。作为说明而非局限，RAM可以是多种形式，比如静态随机存取存储器(Static Random Access Memory，SRAM)或动态随机存取存储器(Dynamic Random Access Memory，DRAM)等。
以上实施例的各技术特征可以进行任意的组合，为使描述简洁，未对上述实施例中的各技术特征所有可能的组合都进行描述，然而，只要这些技术特征的组合不存在矛盾，都应当认为是本说明书记载的范围。
以上所述实施例仅表达了本申请的几种实施方式，其描述较为具体和详细，但并不能因此而理解为对发明专利范围的限制。应当指出的是，对于本领域的普通技术人员来说，在不脱离本申请构思的前提下，还可以做出若干变形和改进，这些都属于本申请的保护范围。因此，本申请专利的保护范围应以所附权利要求为准。

Claims (12)

  1. 一种图像处理方法，所述方法包括：
    获取待处理图像及所述待处理图像对应的相邻图像；
    分别构建所述待处理图像和所述相邻图像的图像金字塔，所述待处理图像对应的图像金字塔为第一图像金字塔，所述相邻图像对应的图像金字塔为第二图像金字塔；
    计算所述第一图像金字塔与所述第二图像金字塔对应每层之间的图像层位移，以及所述图像层位移对应的衡量指标值；
    根据各所述图像层位移以及所述图像层位移对应的衡量指标值，将所述相邻图像与所述待处理图像对齐；
    融合所述相邻图像与所述待处理图像，获取处理后的图像。
  2. 根据权利要求1所述的图像处理方法，其特征在于，在所述获取待处理图像及所述待处理图像对应的相邻图像之后，在所述分别构建所述待处理图像和所述相邻图像的图像金字塔之前，还包括以下至少一项：
    第一项：
    对所述待处理图像和所述相邻图像进行画质优化处理，得到画质优化处理处理后的待处理图像和相邻图像；
    第二项：
    对所述待处理图像和所述相邻图像进行一次滤波处理，得到一次滤波处理后的待处理图像和相邻图像；
    根据所述一次滤波处理后的待处理图像和相邻图像的像素值，确定所述一次滤波处理后的待处理图像和相邻图像中的运动区域；
    计算所述一次滤波处理后的待处理图像和相邻图像的边界值；
    基于所述边界值，确定所述一次滤波处理后的待处理图像和相邻图像中的非边界的运动区域；
    对所述非边界的运动区域进行二次滤波处理，得到噪声处理后的待处理图像和相邻图像。
  3. 根据权利要求1所述的图像处理方法，其特征在于，所述计算所述第一图像金字塔与所述第二图像金字塔对应每层之间的图像层位移，包括：
    计算第一图像层的初始图像块位移，所述初始图像块位移为所述第一图像层中的第一图像块与对应的第二图像层中的第二图像块之间的位移，所述第一图像层为所述第一图像金字塔中的图像层，所述第二图像层为所述第二图像金字塔中的图像层，所述第一图像块为所述 第一图像层中预设大小的图像块，所述第二图像块为所述第二图像层中所述预设大小的图像块；
    计算所述初始图像块位移对应的初始衡量指标值；
    根据所述初始图像块位移和所述初始衡量指标值，确定所述第一图像金字塔与所述第二图像金字塔对应每层之间的图像层位移。
  4. 根据权利要求3所述的图像处理方法，其特征在于，所述计算第一图像层的初始图像块位移，包括以下至少一项：
    第一项：
    对所述第一图像层的前一层图像层的图像层位移进行插值计算，获得插值计算结果，所述前一层图像层为所述第一图像金字塔中，与所述第一图像层相邻、且分辨率低于所述第一图像层的图像层；
    根据所述插值计算结果，确定所述第一图像层的第一图像块位移，所述初始图像块位移包括所述第一图像块位移；
    第二项：
    对所述第一图像层的前一层图像层的图像层位移进行统计计算，获得统计计算结果，所述前一层图像层为所述第一图像金字塔中，与所述第一图像层相邻、且分辨率低于所述第一图像层的第一图像层；
    根据所述统计计算结果，确定所述第一图像层的第二图像块位移，所述初始图像块位移包括所述第二图像块位移；
    第三项：
    确定所述第一图像层中已确定图像块位移的图像块；
    根据所述已确定图像块位移的图像块，确定所述第一图像层的第三图像块位移，所述初始图像块位移包括所述第三图像块位移。
  5. 根据权利要求4所述的图像处理方法，其特征在于，所述计算所述初始图像块位移对应的初始衡量指标值，包括：
    确定所述第二图像层中，与所述第一图像层的初始图像块位移相同的第二图像块；
    根据所述第一图像层中第一图像块的像素值，以及所述第二图像块的像素值，计算所述初始图像块位移对应的初始衡量指标值。
  6. 根据权利要求5所述的图像处理方法，其特征在于，所述根据各所述图像层位移以及 所述图像层位移对应的衡量指标值，将所述相邻图像与所述待处理图像对齐，包括：
    基于所述初始衡量指标值，从所述初始图像块位移中确定所述第一图像层中第一图像块的图像块位移；
    基于所述第一图像层中第一图像块的图像块位移，确定所述第一图像金字塔与所述第二图像金字塔对应每层之间的图像层位移，以及所述图像层位移对应的衡量指标值；
    根据各所述图像层位移及所述图像层位移对应的衡量指标值，将所述相邻图像与所述待处理图像对齐。
  7. 根据权利要求5所述的图像处理方法，其特征在于，在所述基于所述第一图像层中第一图像块的图像块位移，确定所述第一图像金字塔与所述第二图像金字塔对应每层之间的图像层位移之后，还包括：
    确定所述第一图像层中与所述第一图像块相邻的预设邻域图像块；
    根据各所述预设邻域图像块的图像块位移及对应的衡量指标值，更新所述第一图像层中第一图像块的图像块位移，确定更新后的图像层位移。
  8. 根据权利要求7所述的图像处理方法，其特征在于，所述融合所述相邻图像与所述待处理图像，获取处理后的图像，包括：
    根据所述更新后的图像层位移对应的衡量指标值，确定所述相邻图像的初始权重；
    将所述待处理图像的预设权重与所述相邻图像的初始权重进行融合，得到所述相邻图像与所述待处理图像的融合权重；
    基于所述融合权重，融合所述相邻图像与所述待处理图像，获取处理后的图像。
  9. 根据权利要求8所述的图像处理方法，其特征在于，在所述获取处理后的图像之后，还包括：
    根据预设的像素值波动范围，对所述处理后的图像进行校正。
  10. 一种图像处理装置，其特征在于，所述装置包括：
    图像获取模块，用于获取待处理图像及所述待处理图像对应的相邻图像；
    图像金字塔构建模块，用于分别构建所述待处理图像和所述相邻图像的图像金字塔，所述待处理图像对应的图像金字塔为第一图像金字塔，所述相邻图像对应的图像金字塔为第二图像金字塔；
    计算模块，用于计算所述第一图像金字塔与所述第二图像金字塔对应每层之间的图像层位移，以及所述图像层位移对应的衡量指标值；
    图像对齐模块，用于根据各所述图像层位移以及所述图像层位移对应的衡量指标值，将所述相邻图像与所述待处理图像对齐；
    图像融合模块，用于融合所述相邻图像与所述待处理图像，获取处理后的图像。
  11. 一种计算机设备，包括存储器和处理器，所述存储器存储有计算机程序，其特征在于，所述处理器执行所述计算机程序时实现权利要求1至9中任一项所述的图像处理方法的步骤。
  12. 一种计算机可读存储介质，其上存储有计算机程序，其特征在于，所述计算机程序被处理器执行时实现权利要求1至9中任一项所述的图像处理方法的步骤。
PCT/CN2022/097077 2021-06-18 2022-06-06 图像处理方法、装置、计算机设备和存储介质 WO2022262599A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110681297.2A CN115496670A (zh) 2021-06-18 2021-06-18 图像处理方法、装置、计算机设备和存储介质
CN202110681297.2 2021-06-18

Publications (1)

Publication Number Publication Date
WO2022262599A1 true WO2022262599A1 (zh) 2022-12-22

Family

ID=84464029

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/097077 WO2022262599A1 (zh) 2021-06-18 2022-06-06 图像处理方法、装置、计算机设备和存储介质

Country Status (2)

Country Link
CN (1) CN115496670A (zh)
WO (1) WO2022262599A1 (zh)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120002890A1 (en) * 2010-07-05 2012-01-05 Apple Inc. Alignment of digital images and local motion detection for high dynamic range (hdr) imaging
CN103841296A (zh) * 2013-12-24 2014-06-04 哈尔滨工业大学 一种具有大范围旋转及平移运动估计的实时电子稳像方法
WO2017206656A1 (zh) * 2016-05-31 2017-12-07 努比亚技术有限公司 一种图像处理方法及终端、计算机存储介质
CN108573269A (zh) * 2017-10-24 2018-09-25 北京金山云网络技术有限公司 图像特征点匹配方法、匹配装置、电子设备及存储介质
CN111652818A (zh) * 2020-05-29 2020-09-11 浙江大华技术股份有限公司 一种基于金字塔的图像滤波方法、装置及存储介质
CN112308775A (zh) * 2020-09-23 2021-02-02 中国石油大学(华东) 水下图像拼接方法及装置

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9262684B2 (en) * 2013-06-06 2016-02-16 Apple Inc. Methods of image fusion for image stabilization
US20190205758A1 (en) * 2016-12-30 2019-07-04 Konica Minolta Laboratory U.S.A., Inc. Gland segmentation with deeply-supervised multi-level deconvolution networks
CN110070511B (zh) * 2019-04-30 2022-01-28 北京市商汤科技开发有限公司 图像处理方法和装置、电子设备及存储介质
CN111080683B (zh) * 2019-12-09 2023-06-02 Oppo广东移动通信有限公司 图像处理方法、装置、存储介质及电子设备

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120002890A1 (en) * 2010-07-05 2012-01-05 Apple Inc. Alignment of digital images and local motion detection for high dynamic range (hdr) imaging
CN103841296A (zh) * 2013-12-24 2014-06-04 哈尔滨工业大学 一种具有大范围旋转及平移运动估计的实时电子稳像方法
WO2017206656A1 (zh) * 2016-05-31 2017-12-07 努比亚技术有限公司 一种图像处理方法及终端、计算机存储介质
CN108573269A (zh) * 2017-10-24 2018-09-25 北京金山云网络技术有限公司 图像特征点匹配方法、匹配装置、电子设备及存储介质
CN111652818A (zh) * 2020-05-29 2020-09-11 浙江大华技术股份有限公司 一种基于金字塔的图像滤波方法、装置及存储介质
CN112308775A (zh) * 2020-09-23 2021-02-02 中国石油大学(华东) 水下图像拼接方法及装置

Also Published As

Publication number Publication date
CN115496670A (zh) 2022-12-20

Similar Documents

Publication Publication Date Title
US11790481B2 (en) Systems and methods for fusing images
CN110675404B (zh) 图像处理方法、图像处理装置、存储介质与终端设备
US9262684B2 (en) Methods of image fusion for image stabilization
EP2209090B1 (en) Image restoring apparatus and method thereof
CN108898567A (zh) 图像降噪方法、装置及系统
CN103514580B (zh) 用于获得视觉体验优化的超分辨率图像的方法和系统
US8803984B2 (en) Image processing device and method for producing a restored image using a candidate point spread function
JP5980294B2 (ja) データ処理装置、撮像装置、およびデータ処理方法
WO2020007320A1 (zh) 多视角图像的融合方法、装置、计算机设备和存储介质
US10762655B1 (en) Disparity estimation using sparsely-distributed phase detection pixels
CN113992861A (zh) 一种图像处理方法及图像处理装置
US11334961B2 (en) Multi-scale warping circuit for image fusion architecture
CN110650295A (zh) 图像处理方法及装置
US11798146B2 (en) Image fusion architecture
WO2017113917A1 (zh) 成像方法、成像装置和终端
CN117593511A (zh) 一种目标检测方法、装置、电子设备及介质
CN116017172A (zh) 一种Raw域图像的降噪方法及其装置、相机和终端
CN116567401A (zh) 摄像头模组的控制方法、摄像头模组及电子设备
EP3605450B1 (en) Image processing apparatus, image pickup apparatus, control method of image processing apparatus, and computer-program
WO2022262599A1 (zh) 图像处理方法、装置、计算机设备和存储介质
JP7025237B2 (ja) 画像処理装置およびその制御方法ならびにプログラム
JP2017103756A (ja) 画像データ処理装置及び方法
CN111080543A (zh) 图像处理方法及装置、电子设备及计算机可读存储介质
KR102150705B1 (ko) 영상 처리 장치 및 방법
US20220044372A1 (en) Image Fusion Architecture with Multimode Operations

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22824079

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22824079

Country of ref document: EP

Kind code of ref document: A1