TWI855372B - Image processing method, device, electronic equipment and medium - Google Patents

Abstract

The invention discloses an image processing method, device, electronic equipment and medium, belonging to the field of image processing technology. The image processing method comprises: determining N degradation parameters according to K1 preset images; performing degradation processing on the first image according to the N degradation parameters to obtain N degraded images; repeatedly calculating and training the image processing model based on the bull's eye chart image set to obtain a target image processing model; processing the second image according to the target image processing model to obtain a three-dimensional point cloud image corresponding to the second image.

Description

Image processing method, device, electronic equipment and medium

The present invention belongs to the field of image processing technology, and specifically relates to an image processing method, device, electronic equipment and readable storage medium.

As virtual reality technology matures, some electronic devices have the ability to output high-definition three-dimensional point cloud images.

When using electronic equipment to collect 3D images, the focal length of the optical module in the electronic equipment is usually fixed to unify the image accuracy. However, the optical module is limited by the size of the electronic equipment and the hardware cost, resulting in aberrations in the 3D point cloud images output by the electronic equipment, which affects the texture accuracy of the 3D point cloud images and reduces the image quality of the 3D point cloud images.

The purpose of the embodiments of the present invention is to provide an image processing method, apparatus, electronic device and medium, which can solve the problem of aberration in three-dimensional point cloud images, thereby reducing the image quality of the three-dimensional point cloud images.

In a first aspect, an embodiment of the present invention provides an image processing method, the method comprising: Based on K1 preset images, determining N degradation parameters, K1 and N are both positive integers greater than 1; Degrading the first image according to the N degradation parameters to obtain N degraded images; Repeatedly calculating and training the image processing model based on the bull's eye image set to obtain a target image processing model, the bull's eye image set including the first image and the N degraded images; Processing the second image according to the target image processing model to obtain a three-dimensional point cloud image corresponding to the second image.

In a second aspect, an embodiment of the present invention provides an image processing device, which includes: A determination module, used to determine N degradation parameters based on K1 preset images, where K1 and N are both positive integers greater than 1; A first processing module, used to perform degradation processing on the first image based on the N degradation parameters to obtain N degraded images; A training module, used to repeatedly calculate and train the image processing model based on a bull's eye image set to obtain a target image processing model, where the bull's eye image set includes the first image and the N degraded images; A second processing module, used to process the second image based on the target image processing model to obtain a three-dimensional point cloud image corresponding to the second image.

In a third aspect, an embodiment of the present invention provides an electronic device, which includes a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the method described in the first aspect.

In a fourth aspect, an embodiment of the present invention provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

In a fifth aspect, an embodiment of the present invention provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to execute programs or instructions to implement the method described in the first aspect.

In a sixth aspect, an embodiment of the present invention provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the method described in the first aspect.

In the embodiment of the present invention, N degradation parameters are determined based on K1 preset images; the first image is degraded based on the N degradation parameters to obtain N degraded images; the image processing model is repeatedly trained based on the bull's eye image set to obtain a target image processing model; the second image is processed based on the target image processing model to obtain a three-dimensional point cloud image corresponding to the second image. In the embodiment of the present invention, the image processing model is repeatedly trained using the bull's eye image set including the first image and the degraded image to obtain the target image processing model, which is used to repair the degraded image and output an image with higher texture precision. Furthermore, the target image processing model is used to process the input second image to obtain a three-dimensional point cloud image corresponding to the second image, thereby eliminating the aberration existing in the output three-dimensional point cloud image and improving the texture precision and image quality of the three-dimensional point cloud image.

In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention, but not to limit the present invention.

It should be noted that when an element is referred to as being "fixed on" or "disposed on" another element, it can be directly on the other element or indirectly on the other element. When an element is referred to as being "connected to" another element, it can be directly connected to the other element or indirectly connected to the other element.

It should be understood that the directions or positional relationships indicated by the terms "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside" and "outside" are based on the directions or positional relationships shown in the accompanying drawings and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific direction, be constructed and operate in a specific direction, and therefore should not be understood as a limitation on the present invention.

In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the quantity of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

In the present invention, unless otherwise clearly specified and limited, the terms "installation", "connection", "connection", "fixation" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two components or the interaction relationship between two components. For those with ordinary knowledge in this field, the specific meanings of the above terms in the present invention can be understood according to the specific circumstances.

The image processing method provided by the embodiment of the present invention is described in detail below through specific embodiments and application scenarios in conjunction with the accompanying drawings.

In the prior art, an image correction model can be trained and used to improve the texture accuracy of a three-dimensional point cloud image. However, in the repeated calculation training process of the image correction model, only each block structure or each image feature of the two-dimensional image corresponding to the photographed object is repeatedly calculated and processed, without considering the three-dimensional spatial features of the photographed object.

In this case, if the photographed object is an object, and the photographed object includes a near object and a distant object, wherein the distance between the near object and the electronic device is smaller than the distance between the distant object and the electronic device, then after processing the two-dimensional image corresponding to the photographed object using the image correction model, the distortion degree of the near object will be greater than that of the distant object, which reduces the image quality of the three-dimensional point cloud image.

In order to solve the above-mentioned technical problems, the embodiment of the present invention provides an image processing method. Please refer to Figure 1, which is a flow chart of the image processing method provided by the embodiment of the present invention. The image processing method provided by the embodiment of the present invention includes the following steps: S101, determining N degradation parameters based on K1 preset images.

The above degradation parameters are also called point spread function (PSF). PSF describes the response of an imaging system to a point light source. PSF is the pulse response of a focusing optical system.

The image processing method provided in this embodiment can be applied to electronic devices. This step is used to obtain N degradation parameters corresponding to the electronic device, where N is a positive integer greater than 1.

In this step, K1 preset images may be sampled and processed to obtain N degradation parameters, where K1 is a positive integer greater than 1. For a specific technical solution on how to determine the degradation parameters, please refer to the subsequent embodiments.

In an optional implementation, the above-mentioned N degradation parameters are preset.

S102: Perform degradation processing on the first image according to the N degradation parameters to obtain N degraded images.

In this step, after obtaining N degradation parameters, the preset first image is degraded using the N degradation parameters to obtain N degraded images, wherein the first image may be a plurality of high-definition images, and the distortion of the degraded image is greater than the distortion of the first image. For a specific implementation of how to degrade the first image to obtain N degraded images, please refer to the subsequent implementation examples.

S103, repeatedly calculating and training the image processing model based on the bull's eye image set to obtain a target image processing model.

The target training image set includes a first image and N degraded images, and the image processing model is a deep learning network model. For example, the image processing model is a convolutional neural network (CNN) model, or an artificial neural network (ANN) model, or other types of deep learning network models, which are not specifically limited herein.

In this step, the target training image set is used as training data for the image processing model to be trained, the image processing model to be trained is repeatedly trained, and the trained image processing model is determined as the target image processing model.

Optionally, during the Lth repetitive training process of the image processing model to be trained, if the loss function corresponding to the image processing model is less than a preset threshold, the training of the image processing model is determined to be completed. The loss function of the image processing model includes but is not limited to loss functions with image similarity or perceptual difference, such as L1 loss, L2 loss, contextual loss, and perceptual loss.

S104: Process the second image according to the target image processing model to obtain a three-dimensional point cloud image corresponding to the second image.

The second image may be an image input by a user, or an image obtained by other means, which is not specifically limited herein. Optionally, the second image may be an RGB image, or a Raw image.

In other embodiments, a depth map corresponding to the second image may also be obtained synchronously, wherein the depth map is used to represent the depth value corresponding to each pixel in the second image.

In this step, the target image processing model is used to process the second image to obtain a three-dimensional point cloud image corresponding to the second image. For specific implementation methods, please refer to the subsequent embodiments.

In the embodiment of the present invention, N degradation parameters are determined based on K1 preset images; the first image is degraded based on the N degradation parameters to obtain N degraded images; the image processing model is repeatedly trained based on the bull's eye image set to obtain a target image processing model; the second image is processed based on the target image processing model to obtain a three-dimensional point cloud image corresponding to the second image. In the embodiment of the present invention, the image processing model is repeatedly trained using the bull's eye image set including the first image and the degraded image to obtain the target image processing model, which is used to repair the degraded image and output an image with higher texture precision. Furthermore, the target image processing model is used to process the input second image to obtain a three-dimensional point cloud image corresponding to the second image, thereby eliminating the aberration existing in the output three-dimensional point cloud image and improving the texture precision and image quality of the three-dimensional point cloud image.

Optionally, determining N degradation parameters based on K1 preset images includes: Sampling the K1 preset images based on the distance dimension, field angle dimension, and wavelength dimension corresponding to the K1 preset images to obtain K1 degradation parameters; Interpolating the K1 degradation parameters to obtain K2 degradation parameters; Determining at least some of the K2 degradation parameters as N degradation parameters.

In this embodiment, the degradation parameters of the point light source in different distance dimensions, different field angle dimensions, and different wavelength dimensions can be measured. Specifically, the electronic device takes the point light source as the shooting object, shoots the point light source at different distances and/or different field angles, and obtains K1 preset images, wherein the shooting object corresponding to each image in the K1 preset images is the same, which is the point light source.

It should be understood that the above-mentioned point light source is an idealized particle point light source. Optionally, the above-mentioned point light source can be an object; the above-mentioned distance refers to the distance between the electronic device and the point light source; the above-mentioned field of view angle refers to the angle formed by the two edges of the maximum range through which the image of the object can pass through the lens with the lens of the optical module in the electronic device as the vertex.

Furthermore, each preset image is sampled and processed according to the distance dimension, field angle dimension and wavelength dimension of the preset image to obtain the degradation parameter corresponding to each preset image, that is, to obtain K1 degradation parameters.

Among them, the default image includes three color channels of R, G, and B. In order to improve the accuracy of sampling, for each color channel, the maximum quantum efficiency wavelength corresponding to the color channel is used as the sampling wavelength, and the default image is sampled to achieve wavelength dimension sampling.

For example, please refer to Figure 2, where the horizontal coordinate is used to represent the wavelength and the vertical coordinate is used to represent the quantum efficiency. In the application scenario of the preset image shown in Figure 2, the wavelength corresponding to the R channel at the maximum quantum efficiency is 600, and the preset image is sampled and processed at this wavelength.

In this embodiment, an interpolation method may be used to supplement the uncollected degradation parameters, so as to perform interpolation processing on K1 degradation parameters to obtain K2 degradation parameters, where K2 is a positive integer greater than K1.

Exemplarily, if the field angle dimension and wavelength dimension corresponding to the degradation parameter t1 are the same as the field angle dimension and wavelength dimension corresponding to the degradation parameter t2, the distance dimension corresponding to the degradation parameter t1 is 8, the distance dimension corresponding to the degradation parameter t2 is 10, and the specific value of the degradation parameter t1 is 10, and the specific value of the degradation parameter t2 is 20. For the degradation parameter t3, if the field angle dimension and wavelength dimension corresponding to the degradation parameter t3 are the same as the field angle dimension and wavelength dimension corresponding to the degradation parameter t2, and the distance dimension corresponding to the degradation parameter t3 is 9, the degradation parameter t3 can be increased by interpolation, and the distance value of the degradation parameter t3 is 15.

Furthermore, N degradation parameters are randomly selected from the K2 degradation parameters, and the first image is degraded using the N degradation parameters respectively.

Optionally, determining at least part of the K2 degradation parameters as N degradation parameters includes: establishing a degradation parameter table based on the K2 degradation parameters; selecting N degradation parameters from the degradation parameter table.

In this embodiment, after obtaining K2 degradation parameters by interpolation, a degradation parameter table can be established using the K2 degradation parameters, wherein the degradation parameter table is used to characterize the mapping relationship between the degradation parameters and the distance dimension, the field of view angle dimension, and the wavelength dimension.

Furthermore, N degradation parameters are randomly selected from the established degradation parameter table, and then the N degradation parameters are used to perform degradation on the first image respectively, so as to obtain N degraded images.

In some embodiments, after obtaining K2 degradation parameters, a degradation parameter table is not established, and N degradation parameters are directly selected from the K2 degradation parameters, and then the first image is degraded using the N degradation parameters to obtain N degraded images.

Since the degradation parameters are obtained based on sampling of the preset image from the distance dimension, the field angle dimension and the wavelength dimension, the degraded image obtained by degrading the first image using the degradation parameters has three-dimensional spatial characteristics. The degraded image is subsequently used as training data for the image processing model, so that the influence of the three-dimensional spatial characteristics of the photographed object on the output image is taken into account in the training process of the image processing model, so that the distant objects and near objects in the output image have the same texture precision.

Optionally, performing degradation processing on the first image according to the N degradation parameters to obtain N degraded images includes: Performing convolution processing on the first image according to the N degradation parameters to obtain N degraded images.

In this embodiment, the specific implementation method of performing convolution processing on the first image is to use N degradation parameters to perform convolution processing on the first image from the distance dimension, field angle dimension and wavelength dimension of the first image respectively to obtain N degraded images.

To facilitate understanding of the overall technical solution for model training in the image processing method provided by the embodiment of the present invention, please refer to Figure 3. As shown in Figure 3, S301, first measure K1 degradation parameters corresponding to the point light source at different distances, different field angles and different wavelength dimensions; S302, establish a degradation parameter table based on the K1 degradation parameters; S303, the high-definition image (i.e., the first image) is convolved with the degradation parameters to obtain a blurred image (i.e., the degraded image); S304, use the high-definition image and the blurred image as training data for the image processing model, repeatedly perform calculation training on the image processing model, and obtain the target image processing model.

Optionally, the target image processing model processes the second image to obtain a three-dimensional point cloud image corresponding to the second image, including: Determining M sub-images corresponding to the second image, and at least one degradation parameter corresponding to each of the M sub-images; Inputting the M sub-images and the at least one degradation parameter into the target image processing model to obtain M target sub-images; Converting the M target sub-images into a third image, and obtaining a three-dimensional point cloud image corresponding to the second image according to the third image.

As mentioned above, the second image can be an image input by a user, or an image obtained by other means. In this embodiment, after obtaining the second image, M sub-images corresponding to the second image and at least one degradation parameter corresponding to each sub-image are determined, where M is a positive integer greater than 1, and the M sub-images are obtained by converting the second image based on the color channel of the second image. For specific technical solutions on how to convert the second image and how to determine the degradation parameters corresponding to the sub-images, please refer to the subsequent embodiments.

In this embodiment, the above-mentioned M sub-images and at least one degradation parameter are used as inputs of the target image processing model, so that the target image processing module outputs M target sub-images.

In this embodiment, after obtaining M target sub-images, the M target sub-images are formatted and reorganized to obtain a third image, wherein the image format of the third image is the same as the image format of the second image, that is, if the second image is an RGB image, the third image is also an RGB image.

Furthermore, a three-dimensional projection is performed on each pixel in the third image according to the depth value corresponding to the pixel to obtain a three-dimensional point cloud image, wherein each pixel in the third image can be queried in the depth map to obtain the depth value corresponding to the pixel.

It should be noted that if the third image is a Raw image, it is necessary to perform image signal processing (ISP) on the third image to convert it into an RGB image, and then perform three-dimensional projection on each pixel in the third image to obtain a three-dimensional point cloud image.

Optionally, the determining of the M sub-images corresponding to the second image and at least one degradation parameter corresponding to each of the M sub-images includes: Determining the M sub-images corresponding to the second image; For any one of the M sub-images, based on the target degradation parameter table, determining the dimension information corresponding to each pixel in the sub-image, and obtaining at least one degradation parameter corresponding to the sub-image.

In this embodiment, the second image is converted to obtain M sub-images. For each sub-image, the dimension information corresponding to each pixel in the sub-image is queried in a preset target degradation parameter table to obtain at least one degradation parameter corresponding to the sub-image, wherein the target degradation parameter table is used to characterize the mapping relationship between the degradation parameter and the distance dimension, the field angle dimension and the wavelength dimension, and the dimension information includes at least one of the distance dimension, the field angle dimension and the wavelength dimension.

It should be understood that in the process of obtaining the second image, the depth map corresponding to the second image is obtained synchronously. As mentioned above, the depth map is used to characterize the depth value corresponding to each pixel in the second image. It should be understood that the depth value corresponding to each pixel is the distance dimension corresponding to each pixel. In this way, the depth value corresponding to each pixel of the sub-image can be input into the target degradation parameter table to obtain at least one degradation parameter corresponding to the sub-image.

Optionally, determining the M sub-images corresponding to the second image includes: When the second image is a first type of image, transforming the second image based on the color channel of the second image to obtain M sub-images; When the second image is a second type of image, inverse transforming the second image to obtain a fourth image, transforming the fourth image based on the color channel of the fourth image to obtain M sub-images.

The first type is a Raw image, and the second type is an RGB image. In this embodiment, the method for converting the second image is determined according to the image type of the second image.

One implementation is: when the image type of the second image is the first type, the second image is directly converted based on the color channel of the second image to obtain an R channel sub-image, a G channel sub-image, and a B channel sub-image.

Another implementation method is: when the image type of the second image is the second type, the second image is subjected to inverse gamma transformation, inverse color transformation and inverse white balance transformation to obtain a fourth image, and then the fourth image is transformed based on the color channel of the fourth image to obtain an R channel sub-image, a G channel sub-image and a B channel sub-image.

To facilitate understanding of the overall technical solution of using the target image processing model in the image processing method provided by the embodiment of the present invention, please refer to FIG. 4. As shown in FIG. 4, S401, a second image and a depth map corresponding to the second image are obtained; S402, the second image is converted into sub-images of different color channels; S403, a degradation parameter is determined according to the depth value corresponding to each pixel in the sub-image, that is, the depth value corresponding to the sub-image is queried in a preset target degradation parameter table to obtain the degradation parameter corresponding to the sub-image; S404, the sub-image and the degradation parameter are input into the target image processing model to obtain M target sub-images; S405, the M target sub-images are reorganized; S406, the reorganized bull's eye image is projected into three-dimensional space according to the depth value of each pixel to generate a point cloud.

The image processing device provided by the embodiment of the present invention is described in detail below through specific embodiments and application scenarios in conjunction with the accompanying drawings.

As shown in FIG5 , the training device 500 of the image processing model includes: A determination module 501, used to determine N degradation parameters based on K1 preset images; A first processing module 502, used to perform degradation processing on the first image based on the N degradation parameters to obtain N degraded images; A training module 503, used to repeatedly calculate and train the image processing model based on the bull's eye chart image set to obtain a target image processing model; A second processing module 504, used to process the second image based on the target image processing model to obtain a three-dimensional point cloud image corresponding to the second image.

Optionally, the determination module 501 is specifically used to: Sample the K1 preset images according to the distance dimension, field angle dimension and wavelength dimension corresponding to the K1 preset images to obtain K1 degradation parameters; Interpolate the K1 degradation parameters to obtain K2 degradation parameters; Determine at least part of the K2 degradation parameters as N degradation parameters.

Optionally, the determination module 501 is also specifically used to: Establish a degradation parameter table based on the K2 degradation parameters; Select N degradation parameters from the degradation parameter table.

Optionally, the first processing module 502 is specifically used to: Perform convolution processing on the first image according to the N degradation parameters to obtain N degraded images.

Optionally, the second processing module 504 is specifically used to: Determine M sub-images corresponding to the second image, and at least one degradation parameter corresponding to each of the M sub-images, where M is a positive integer greater than 1; Input the M sub-images and the at least one degradation parameter into the target image processing model to obtain M target sub-images; Convert the M target sub-images into a third image, and obtain a three-dimensional point cloud image corresponding to the second image based on the third image.

Optionally, the second processing module 504 is further specifically used to: Determine M sub-images corresponding to the second image; For any one of the M sub-images, based on the target degradation parameter table, determine the dimension information corresponding to each pixel in the sub-image, and obtain at least one degradation parameter corresponding to the sub-image.

Optionally, the second processing module 504 is further specifically used for: When the second image is a first type of image, transform the second image based on the color channel of the second image to obtain M sub-images; When the second image is a second type of image, perform inverse transformation processing on the second image to obtain a fourth image, and transform the third image based on the color channel of the fourth image to obtain M sub-images.

In the embodiment of the present invention, the image processing model is repeatedly trained using a bull's eye image set including the first image and the degraded image to obtain a target image processing model, which is used to repair the degraded image and output an image with higher texture precision. Furthermore, the target image processing model is used to process the input second image to obtain a three-dimensional point cloud image corresponding to the second image, thereby eliminating the aberration existing in the output three-dimensional point cloud image and improving the texture precision and image quality of the three-dimensional point cloud image.

The image processing device in the embodiment of the present invention can be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or other devices except the terminal. Exemplarily, the electronic device may be a mobile phone, a tablet computer, a laptop computer, a handheld computer, a vehicle-mounted electronic device, a mobile Internet device (MID), an augmented reality (AR)/virtual reality (VR) device, a robot, a wearable device, an ultra-mobile personal computer (UMPC), a mini laptop computer or a personal digital assistant (PDA), etc. It may also be a server, a network attached storage (NAS), a personal computer (PC), a television (TV), a teller machine or a self-service machine, etc., and the embodiments of the present invention are not specifically limited.

The image processing device in the embodiment of the present invention may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present invention.

The image processing device provided in the embodiment of the present invention can implement various processes implemented in the method embodiment of FIG. 1 . To avoid repetition, they will not be described again here.

Optionally, as shown in FIG6 , an embodiment of the present invention further provides an electronic device 600, including a processor 601, a memory 602, and a program or instruction stored in the memory 602 and executable on the processor 601. When the program or instruction is executed by the processor 601, the various processes of the training method embodiment of the above-mentioned image processing model are implemented, or the various processes of the above-mentioned image processing method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

It should be noted that the electronic devices in the embodiments of the present invention include the above-mentioned mobile electronic devices and non-mobile electronic devices.

FIG7 is a schematic diagram of the hardware structure of an electronic device implementing an embodiment of the present invention.

The electronic device 700 includes but is not limited to: a radio frequency unit 701, a network module 702, an audio output unit 703, an input unit 704, a sensor 705, a display unit 706, a user input unit 707, an interface unit 708, a memory 709, and a processor 710 and other components.

It is understood by those skilled in the art that the electronic device 700 may also include a power source (such as a battery) for supplying power to various components. The power source may be logically connected to the processor 710 through a power management system, so that the power management system can manage charging, discharging, and power consumption. The electronic device structure shown in FIG7 does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or arrange components differently, which will not be elaborated here.

The processor 710 is further used to determine N degradation parameters according to K1 preset images;

Performing degradation processing on the first image according to the N degradation parameters to obtain N degraded images;

Based on the bull's eye image set, the image processing model is repeatedly trained to obtain the target image processing model;

The second image is processed according to the target image processing model to obtain a three-dimensional point cloud image corresponding to the second image.

The processor 710 is further configured to sample the K1 preset images according to the distance dimension, the field angle dimension and the wavelength dimension corresponding to the K1 preset images to obtain K1 degradation parameters;

The K1 degradation parameters are interpolated to obtain K2 degradation parameters;

At least part of the K2 degradation parameters are determined as N degradation parameters.

The processor 710 is further used to establish a degradation parameter table based on the K2 degradation parameters;

Select N degradation parameters from the degradation parameter table.

The processor 710 is further used to perform convolution processing on the first image according to the N degradation parameters to obtain N degraded images.

The processor 710 is further configured to determine M sub-images corresponding to the second image, and at least one degradation parameter corresponding to each of the M sub-images;

Inputting the M sub-images and the at least one degradation parameter into the target image processing model to obtain M target sub-images;

The M target sub-images are converted into a third image, and a three-dimensional point cloud image corresponding to the second image is obtained according to the third image.

The processor 710 is further used to determine M sub-images corresponding to the second image;

For any sub-image among the M sub-images, dimension information corresponding to each pixel in the sub-image is determined based on the target degradation parameter table, and at least one degradation parameter corresponding to the sub-image is obtained.

The processor 710 is further configured to, when the second image is an image of the first type, convert the second image based on the color channel of the second image to obtain M sub-images;

When the second image is a second type image, the second image is inversely transformed to obtain a fourth image, and the fourth image is transformed based on a color channel of the fourth image to obtain M sub-images.

In the embodiment of the present invention, the image processing model is repeatedly trained using a bull's eye image set including the first image and the degraded image to obtain a target image processing model, which is used to repair the degraded image and output an image with higher texture precision. Furthermore, the target image processing model is used to process the input second image to obtain a three-dimensional point cloud image corresponding to the second image, thereby eliminating the aberration existing in the output three-dimensional point cloud image and improving the texture precision and image quality of the three-dimensional point cloud image.

It should be understood that in the embodiment of the present invention, the input unit 704 may include a graphics processing unit (GPU) 7041 and a microphone 7042, and the graphics processor 7041 processes the image data of the static picture or video obtained by the image capture device (such as a camera) in the video capture mode or the image capture mode. The display unit 706 may include a display panel 7061, and the display panel 7071 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 707 includes at least one of a touch panel 7071 and other input devices 7072. The touch panel 7071 is also called a touch screen. The touch panel 7071 may include two parts: a touch detection device and a touch controller. Other input devices 7072 may include but are not limited to a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be elaborated here.

The memory 709 can be used to store software programs and various data. The memory 709 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instruction required for at least one function (such as a sound playback function, an image playback function, etc.). In addition, the memory 709 may include a volatile memory or a non-volatile memory, or the memory 709 may include both a volatile memory and a non-volatile memory. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable ROM (PROM), an erasable programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM) or a flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct memory bus random access memory (DRRAM). The memory 709 in embodiments of the present invention includes but is not limited to these and any other suitable types of memory.

The processor 710 may include one or more processing units; optionally, the processor 710 integrates an application processor and a modulation and demodulation processor, wherein the application processor mainly processes operations related to the operating system, the user interface, and the application program, and the modulation and demodulation processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the above-mentioned modulation and demodulation processor may not be integrated into the processor 710.

The embodiment of the present invention also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, each process of the above-mentioned image processing method embodiment is implemented and the same technical effect can be achieved. To avoid repetition, it will not be described here.

The processor is the processor in the electronic device in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

The embodiment of the present invention further provides a chip, which includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned image processing method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

It should be understood that the chip mentioned in the embodiments of the present invention can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

The embodiment of the present invention provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned image processing method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

It should be noted that, in this article, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the method and device in the embodiment of the present invention is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in a reverse order according to the functions involved. For example, the described method may be performed in an order different from that described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

Through the description of the above implementation, a person with ordinary knowledge in the field can clearly understand that the above implementation method can be implemented by means of software plus a necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM/RAM, disk, optical disk), including a number of instructions for a terminal (which can be a mobile phone, computer, server, or network equipment, etc.) to execute the methods described in each embodiment of the present invention.

The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific embodiments, which are merely illustrative rather than restrictive. Under the inspiration of the present invention, a person with ordinary knowledge in the field can make many forms without departing from the scope of protection of the purpose of the present invention and the claims, all of which are within the protection of the present invention.

S101-S104: Steps S301-S304: Steps S401-S406: Steps 500: Training device 501: Determination module 502: First processing module 503: Training module 504: Second processing module 600: Electronic device 601: Processor 602: Memory 700: Electronic device 701: RF unit 702: Network module 703: Audio output unit 704: Input unit 7041: Graphics processor 7042: Microphone 705: Sensor 706: Display unit 7061: Display panel 707: User input unit 7071: Touch panel 7072: Other input devices 708: Interface unit 709: Memory 710: Processor

FIG. 1 is a flow chart of an image processing method provided by an embodiment of the present invention; FIG. 2 is a schematic diagram of wavelengths corresponding to different color channels of a preset image; FIG. 3 is one of the application flow charts of the image processing method provided by an embodiment of the present invention; FIG. 4 is a second application flow chart of the image processing method provided by an embodiment of the present invention; FIG. 5 is a structural diagram of an image processing device provided by an embodiment of the present invention; FIG. 6 is a structural diagram of an electronic device provided by an embodiment of the present invention; FIG. 7 is a hardware structural diagram of an electronic device provided by an embodiment of the present invention.

S101-S103: Steps

Claims (9)

An image processing method is characterized in that it includes: determining N degradation parameters based on K1 preset images, where K1 and N are both positive integers greater than 1; performing degradation processing on a first image based on the N degradation parameters to obtain N degraded images; repeatedly calculating and training an image processing model based on a bull's eye image set to obtain a target image processing model, where the bull's eye image set includes the first image and the N degraded images; processing a second image based on the target image processing model to obtain a three-dimensional image corresponding to the second image. Point cloud image; the determining N degradation parameters according to K1 preset images includes: sampling the K1 preset images according to the distance dimension, field angle dimension and wavelength dimension corresponding to the K1 preset images to obtain K1 degradation parameters; each image in the K1 preset images corresponds to the same photographed object; interpolating the K1 degradation parameters to obtain K2 degradation parameters, wherein K2 is a positive integer greater than K1; determining at least part of the K2 degradation parameters as N degradation parameters. The image processing method as described in claim 1 is characterized in that determining at least part of the K2 degradation parameters as N degradation parameters includes: establishing a degradation parameter table based on the K2 degradation parameters, the degradation parameter table is used to characterize the mapping relationship between the degradation parameters and the distance dimension, the field of view angle dimension and the wavelength dimension; selecting N degradation parameters from the degradation parameter table. The image processing method as described in claim 1 is characterized in that the step of performing degradation processing on the first image according to the N degradation parameters to obtain N degraded images includes: performing convolution processing on the first image according to the N degradation parameters to obtain N degraded images. The image processing method as described in claim 1 is characterized in that the second image is processed according to the target image processing model to obtain a three-dimensional point cloud image corresponding to the second image, including: determining M sub-images corresponding to the second image, and at least one degradation parameter corresponding to each of the M sub-images, where M is a positive integer greater than 1; inputting the M sub-images and the at least one degradation parameter into the target image processing model to obtain M target sub-images; converting the M target sub-images into a third image, and obtaining a three-dimensional point cloud image corresponding to the second image according to the third image. The image processing method as described in claim 4 is characterized in that the determining of the M sub-images corresponding to the second image and the at least one degradation parameter corresponding to each of the M sub-images comprises: determining the M sub-images corresponding to the second image; for any one of the M sub-images, based on a target degradation parameter table, determining the dimension information corresponding to each pixel in the sub-image, and obtaining at least one degradation parameter corresponding to the sub-image; wherein the target degradation parameter table is used to characterize the mapping relationship between the degradation parameter and the target dimension, and the target dimension includes at least one of the distance dimension, the field of view angle dimension and the wavelength dimension. The image processing method as described in claim 4 is characterized in that the determining of the M sub-images corresponding to the second image includes: when the second image is a first type of image, the second image is transformed based on the color channel of the second image to obtain the M sub-images; when the second image is a second type of image, the second image is inversely transformed to obtain a fourth image, and the fourth image is transformed based on the color channel of the fourth image to obtain the M sub-images. An image processing device, characterized in that the device comprises: a determination module, used to determine N degradation parameters according to K1 preset images, K1 and N are both positive integers greater than 1; a first processing module, used to perform degradation processing on a first image according to the N degradation parameters to obtain N degraded images; a training module, used to repeatedly calculate and train an image processing model based on a bull's eye image set to obtain a target image processing model, the bull's eye image set including the first image and the N degraded images; a second processing module, used to perform a training on an image processing model based on the target image processing model Process the second image to obtain a three-dimensional point cloud image corresponding to the second image; the determination module is specifically used to: According to the distance dimension, field angle dimension and wavelength dimension corresponding to the K1 preset images, sample the K1 preset images to obtain K1 degradation parameters; each image in the K1 preset images corresponds to the same photographed object; interpolate the K1 degradation parameters to obtain K2 degradation parameters, where K2 is a positive integer greater than K1; determine at least part of the K2 degradation parameters as N degradation parameters. An electronic device, characterized in that it includes a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the image processing method as described in any one of claims 1 to 6. A readable storage medium, characterized in that a program or instruction is stored on the readable storage medium, and when the program or instruction is executed by a processor, the steps of the image processing method described in any one of claim items 1 to 6 are implemented.