CN111861959A - An ultra-long depth of field and ultra-wide dynamic image synthesis algorithm - Google Patents

Abstract

The invention discloses an ultra-long depth-of-field and ultra-wide dynamic image synthesis algorithm, which specifically includes the following steps: S1: acquiring at least two images; S2: calculating the clarity C, saturation S and exposure degree E of each image respectively; S3 : Calculate the weight W of each image according to the clarity C, saturation S and exposure degree E of each image; S4: Perform weighted average processing on the weight W of all images to synthesize a super-long depth of field and super-wide dynamic image; This image synthesis algorithm can be directly synthesized on RGB images, and does not need to be processed in RAW format, which is convenient to process; this image synthesis algorithm has low computational complexity, does not require tone mapping, and can be processed in real time; this image synthesis algorithm also considers Sharpness, saturation and exposure, can synthesize ultra-long depth of field and ultra-wide dynamic images at the same time, and can better retain color information.

Description

An ultra-long depth of field and ultra-wide dynamic image synthesis algorithm

technical field

The invention relates to an image synthesis algorithm, in particular to an ultra-long depth of field and ultra-wide dynamic image synthesis algorithm.

Background technique

Due to the limitations of the physical and digital characteristics of the camera, the camera has a limited depth of field and a limited dynamic range, that is, a limited distance range that can be seen clearly and a limited light and dark range that can be distinguished. In some fields that require high depth of field and dynamic range, the limited depth of field and dynamic range cause a lot of trouble. For example, when the endoscopic camera system is used clinically, due to the limited depth of field, the working distance in the surgical field of view is greatly different. It is difficult to see clearly at the same time, and it is necessary to repeatedly adjust the working distance or focal length, which brings inconvenience to the operation; at the same time, due to the limited dynamic range, it is difficult to see objects with a large gap between light and dark at the same time, which brings discomfort to the doctor.

The existing high dynamic range (HDR) synthesis algorithm weights multiple RAW images with different exposures according to the exposure time, and the synthesized HDR image is represented by a matrix with a wide range of values. HDR images are mapped to 0~255 for correct color display. The computational complexity of this algorithm is high, and it is difficult to achieve the real-time display requirements of 50 and 60 frames of the camera system. In addition, this algorithm needs to be performed on RAW format images. For commonly used RGB images, it needs to be transferred to RAW format for operation, which further increases the computational complexity. Furthermore, the quality of the composite image is mainly related to the tone mapping method, and tone mapping is prone to problems such as loss of details, halos, and color casts. Most importantly, this algorithm can only solve the problem of limited dynamic range of the camera, but not the problem of limited depth of field.

Therefore, the existing technology still needs to be improved and developed.

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide an ultra-long depth-of-field and ultra-wide dynamic image synthesis algorithm, so as to provide a super-high-resolution output image with an ultra-long depth of field.

The technical scheme of the present invention is as follows: an ultra-long depth of field and ultra-wide dynamic image synthesis algorithm, which specifically includes the following steps:

S1: Acquire at least two images;

S2: Calculate the clarity C, saturation S and exposure E of each image respectively;

S3: Calculate the weight W of each image respectively according to the sharpness C, saturation S and exposure degree E of each image;

S4: Perform weighted average processing on the weights W of all the images to synthesize a super-long depth-of-field super-wide dynamic image.

In the super-long depth-of-field and super-wide dynamic image synthesis algorithm, in the S1, the images include a close-range clear image without exposure and a distant view clear image without underexposure.

The ultra-long depth of field and ultra-wide dynamic image synthesis algorithm, wherein, in the S2, the 3*3 Laplacian operator is used to slide over each pixel of each image to obtain the definition C of each pixel. (i,j).

The super-long depth-of-field super-wide dynamic image synthesis algorithm, wherein, the calculation formula of the clarity C(i,j) of each pixel point is:

f(i,j) is the gray value of the pixel whose coordinates are (i,j).

In the super-long depth-of-field super-wide dynamic image synthesis algorithm, in the S2, the saturation S(i, j) of each pixel is obtained by calculating the RGB value of each pixel of each image.

The super-long depth-of-field super-wide dynamic image synthesis algorithm, wherein, the calculation formula of the saturation S(i,j) of each pixel point is:

Among them, R(i,j) is the red component of the pixel with coordinates (i,j), G(i,j) is the green component of the pixel with coordinates (i,j), and B(i,j) is the coordinate is the blue component of the (i, j) pixel, and M(i, j) is the gray value of the (i, j) pixel.

In the super-long depth-of-field and super-wide dynamic image synthesis algorithm, the exposure degree E(i, j) of each pixel is obtained by calculating the RGB value of each pixel of each image.

The super-long depth-of-field super-wide dynamic image synthesis algorithm, wherein, the calculation formula of the exposure degree E(i,j) of each pixel point is:

Among them, G is the default value, R(i,j) is the red component of the pixel with coordinates (i,j), G(i,j) is the green component of the pixel with coordinates (i,j), B( i,j) is the blue component of the pixel whose coordinates are (i,j).

In the super-long depth-of-field super-wide dynamic image synthesis algorithm, in the step S3, after obtaining the clarity C, the saturation S and the exposure degree E of each pixel in each image, respectively calculate the content in each image. The weight W of each pixel point; the weight value W(i,j) of each pixel point is calculated according to C(i,j), S(i,j), B(i,j), and the calculation formula is:

。

.

， The super-long depth of field and super-wide dynamic image synthesis algorithm, wherein, in the S4, the weights of the pixels of the same coordinate in all the images are weighted and averaged one by one to obtain the pixels of the same coordinate in the super-long depth of field and super-wide dynamic images. The pixel value of the super-long depth of field and super-wide dynamic image is synthesized to obtain the super-long depth of field and super-wide dynamic image. The weighting formula of each pixel point is:

,

Among them, Wn is the weight of the pixel with coordinates (i, j) in the nth image, and In(i, j) is the pixel value of the pixel with coordinates (i, j) in the nth image.

Beneficial effects of the present invention: the present invention provides an ultra-long depth of field and ultra-wide dynamic image synthesis algorithm, which can be directly synthesized on RGB images without processing in RAW format, and the processing is convenient; the image synthesis algorithm has low computational complexity , does not require tone mapping, and can be processed in real time; this image synthesis algorithm considers sharpness, saturation and exposure at the same time, can synthesize ultra-long depth of field and ultra-wide dynamic images at the same time, and can better retain color information.

Description of drawings

FIG. 1 is a flow chart of the steps of an ultra-long depth and ultra-wide dynamic image synthesis algorithm in the present invention.

Detailed ways

Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals refer to the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary, only used to explain the present invention, and should not be construed as a limitation of the present invention.

In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", " rear, left, right, vertical, horizontal, top, bottom, inside, outside, clockwise, counterclockwise, etc., or The positional relationship is based on the orientation or positional relationship shown in the accompanying drawings, which is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, Therefore, it should not be construed as a limitation of the present invention. In addition, the terms "first" and "second" are only used for descriptive purposes, and should not be construed as indicating or implying relative importance or implying the number of indicated technical features. Thus, features defined as "first", "second" may expressly or implicitly include one or more of said features. In the description of the present invention, "plurality" means two or more, unless otherwise expressly and specifically defined.

In the description of the present invention, it should be noted that the terms "installed", "connected" and "connected" should be understood in a broad sense, unless otherwise expressly specified and limited, for example, it may be a fixed connection or a detachable connection Connection, or integral connection; it can be a mechanical connection, an electrical connection or can communicate with each other; it can be directly connected or indirectly connected through an intermediate medium, it can be the internal communication of two elements or the interaction of two elements relation. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific situations.

In the present invention, unless otherwise expressly specified and limited, a first feature "on" or "under" a second feature may include the first and second features in direct contact, or may include the first and second features Not directly but through additional features between them. Also, the first feature being "above", "over" and "above" the second feature includes the first feature being directly above and obliquely above the second feature, or simply means that the first feature is level higher than the second feature. The first feature is "below", "below" and "below" the second feature includes the first feature being directly below and diagonally below the second feature, or simply means that the first feature has a lower level than the second feature.

The following disclosure provides many different embodiments or examples for implementing different structures of the present invention. In order to simplify the disclosure of the present invention, the components and arrangements of specific examples are described below. Of course, they are only examples and are not intended to limit the invention. Furthermore, the present disclosure may repeat reference numerals and/or reference letters in different instances for the purpose of simplicity and clarity and not in itself indicative of a relationship between the various embodiments and/or arrangements discussed. In addition, the present disclosure provides examples of various specific processes and materials, but one of ordinary skill in the art will recognize the application of other processes and/or the use of other materials.

As shown in Figure 1, an ultra-long depth of field and ultra-wide dynamic image synthesis algorithm specifically includes the following steps:

S1: Acquire at least two images.

Wherein, at least one image image1 with a clear close-range view and no underexposure and at least one image image2 with a clear view and no underexposure are obtained respectively for synthesis.

S2: Calculate the sharpness C, saturation S and exposure E of each image separately.

Wherein, the definition C is slid over each pixel point of the image through the 3*3 Laplacian operator, and the definition C(i,j) of each pixel point is obtained, and the calculation formula is as follows:

f(i,j) is the gray value of the pixel whose coordinates are (i,j). The larger the C value, the higher the clarity.

Wherein, the saturation S is calculated by the RGB value of each pixel to obtain the saturation S(i, j) of each pixel, and the calculation formula is as follows:

Among them, R(i,j) is the red component of the pixel with coordinates (i,j), G(i,j) is the green component of the pixel with coordinates (i,j), and B(i,j) is the coordinate is the blue component of the (i, j) pixel, and M(i, j) is the gray value of the (i, j) pixel. The larger the S value, the higher the saturation.

Wherein, the exposure degree E is calculated by the RGB value of each pixel point to obtain the exposure degree E(i, j) of each pixel point, and the calculation formula is as follows:

Among them, G is the default value, which can be set to 0.2. The larger the E value, the better the exposure.

Among them, the value range of the above R(i,j), G(i,j), B(i,j), f(i,j) is 0~1, if not, it needs to be normalized.

S3: Calculate the weight W of each image respectively according to the sharpness C, saturation S and exposure level E of each image.

After obtaining the sharpness C, saturation S and exposure degree E of each pixel in each image, respectively calculate the weight W of each pixel in each image, and the weight value of each pixel W(i, j ) is calculated according to C(i,j), S(i,j), B(i,j), and the calculation formula is as follows:

S4: Perform weighted average processing on the weights W of all the images to synthesize a super-long depth-of-field super-wide dynamic image.

Assuming that the weights of the pixels at the same coordinates in the n images are W1...Wn, the weights of the pixels at the same coordinates in the n images are weighted and averaged to obtain the pixels of the same coordinates in the super-long depth of field and super-wide dynamic images. value, synthesize all the pixels in the obtained ultra-long depth of field and ultra-wide dynamic image to obtain an ultra-long depth of field and ultra-wide dynamic image, and the weighting formula of each pixel is as follows:

Among them: I1(i,j)...In(i,j) is the pixel value of the first to nth image whose coordinates are (i,j) pixels.

In the description of this specification, reference to the terms "one embodiment," "some embodiments," "exemplary embodiment," "example," "specific example," or "some examples", etc. A particular feature, structure, material, or characteristic described in this embodiment or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

It should be understood that the application of the present invention is not limited to the above examples. For those of ordinary skill in the art, improvements or transformations can be made according to the above descriptions, and all these improvements and transformations should belong to the protection scope of the appended claims of the present invention.

Claims (10)

1. An ultra-wide dynamic image synthesis algorithm with ultra-long depth of field is characterized by comprising the following steps:

s1: acquiring at least two images;

s2: respectively calculating the definition C, the saturation S and the exposure E of each image;

S3: respectively calculating the weight W of each image according to the definition C, the saturation S and the exposure E of each image;

s4: and carrying out weighted average processing on the weights W of all the images to synthesize the ultra-long depth-of-field ultra-wide dynamic image.

2. The ultra-long depth-of-field ultra-wide dynamic image synthesis algorithm as claimed in claim 1, wherein in the step S1, the image comprises a close-up clear and non-overexposed image and a far-up clear and non-underexposed image.

3. The ultra-wide dynamic image synthesis algorithm with ultra-long depth of field according to claim 1, wherein in S2, the definition C (i, j) of each pixel is obtained by sliding 3 × 3 laplacian across each pixel in each image.

4. The ultra-wide dynamic image synthesis algorithm with ultra-long depth of field according to claim 3, wherein the calculation formula of the definition C (i, j) of each pixel point is as follows:

f (i, j) is the gray value of the pixel point with the coordinate (i, j).

5. The ultra-wide dynamic image synthesis algorithm with ultra-long depth of field according to claim 1, wherein in S2, the saturation S (i, j) of each pixel is calculated from the RGB value of each pixel of each image.

6. The ultra-wide dynamic image synthesis algorithm with ultra-long depth of field according to claim 5, wherein the calculation formula of the saturation S (i, j) of each pixel point is as follows:

Wherein, R (i, j) is a red component of the pixel point with the coordinate (i, j), G (i, j) is a green component of the pixel point with the coordinate (i, j), B (i, j) is a blue component of the pixel point with the coordinate (i, j), and M (i, j) is a gray value of the pixel point with the coordinate (i, j).

7. The ultra-wide dynamic image synthesis algorithm with ultra-long depth of field according to claim 1, wherein the exposure degree E (i, j) of each pixel point is calculated by the RGB value of each pixel point of each image.

8. The ultra-wide dynamic image synthesis algorithm with ultra-long depth of field according to claim 7, wherein the calculation formula of the exposure degree E (i, j) of each pixel point is as follows:

wherein, G is a preset value, R (i, j) is a red component of a pixel point with coordinates (i, j), G (i, j) is a green component of the pixel point with coordinates (i, j), and B (i, j) is a blue component of the pixel point with coordinates (i, j).

9. The ultra-wide dynamic image synthesis algorithm with ultra-long depth of field according to claim 1, wherein in S3, after obtaining the definition C, the saturation S and the exposure E of each pixel in each image, the weight W of each pixel in each image is calculated; the weighted value W (i, j) of each pixel point is obtained by calculation according to C (i, j), S (i, j) and B (i, j), and the calculation formula is as follows:

。

10. The ultra-wide dynamic image synthesis algorithm with ultra-long depth of field according to claim 1, wherein in S4, the weights of the pixels at the same coordinate in all the images are weighted and averaged one by one to obtain the pixel value of the pixel at the same coordinate in the ultra-wide dynamic image with ultra-long depth of field, and all the pixels in the obtained ultra-wide dynamic image with ultra-long depth of field are synthesized to obtain the ultra-wide dynamic image with ultra-long depth of field, and the weighting formula of each pixel is as follows:

，

wherein, Wn is the weight of the pixel point with the coordinate (i, j) In the nth image, and In (i, j) is the pixel value of the pixel point with the coordinate (i, j) In the nth image.

Images