[go: up one dir, main page]

CN119364204B - Dual-gain HDR synthesis method and system for sCMOS sensors - Google Patents

Dual-gain HDR synthesis method and system for sCMOS sensors

Info

Publication number
CN119364204B
CN119364204B CN202411490650.9A CN202411490650A CN119364204B CN 119364204 B CN119364204 B CN 119364204B CN 202411490650 A CN202411490650 A CN 202411490650A CN 119364204 B CN119364204 B CN 119364204B
Authority
CN
China
Prior art keywords
gain
image
low
hdr
gray
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202411490650.9A
Other languages
Chinese (zh)
Other versions
CN119364204A (en
Inventor
王小庆
郑福
孙博奥
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
National Space Science Center of CAS
Original Assignee
National Space Science Center of CAS
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by National Space Science Center of CAS filed Critical National Space Science Center of CAS
Priority to CN202411490650.9A priority Critical patent/CN119364204B/en
Publication of CN119364204A publication Critical patent/CN119364204A/en
Application granted granted Critical
Publication of CN119364204B publication Critical patent/CN119364204B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/50—Control of the SSIS exposure
    • H04N25/57—Control of the dynamic range
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/50—Control of the SSIS exposure
    • H04N25/51—Control of the gain
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/70—SSIS architectures; Circuits associated therewith
    • H04N25/76—Addressed sensors, e.g. MOS or CMOS sensors
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/14—Picture signal circuitry for video frequency region
    • H04N5/20—Circuitry for controlling amplitude response
    • H04N5/202—Gamma control

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Transforming Light Signals Into Electric Signals (AREA)
  • Studio Devices (AREA)

Abstract

本发明提供了sCMOS传感器的双增益HDR合成方法及系统，涉及图像处理领域，该方法通过使用sCMOS传感器同时获取高增益和低增益两幅图像；对高增益和低增益图像进行灰度值线性关系分析；进行图像直方图统计，确定增益和曝光时间设置；对高增益和低增益图像进行灰度拉伸处理，根据灰度值关系合成HDR图像。通过对高增益和低增益图像进行灰度值线性关系分析及灰度拉伸处理，可以快速合成高动态范围图像，旨在消除图像配准问题并简化HDR图像的合成处理过程。通过对高增益和低增益图像进行灰度值线性关系分析及灰度拉伸处理，快速合成高动态范围图像。

The present invention provides a dual-gain HDR synthesis method and system for an sCMOS sensor, relating to the field of image processing. The method utilizes an sCMOS sensor to simultaneously acquire two images, one high-gain and one low-gain; performs a linear grayscale value analysis on the high-gain and low-gain images; performs image histogram statistics to determine gain and exposure time settings; and performs grayscale stretching on the high-gain and low-gain images to synthesize an HDR image based on the grayscale value relationship. By performing the grayscale value linear relationship analysis and grayscale stretching on the high-gain and low-gain images, a high dynamic range image can be rapidly synthesized, eliminating image registration issues and simplifying the HDR image synthesis process. By performing the grayscale value linear relationship analysis and grayscale stretching on the high-gain and low-gain images, a high dynamic range image can be rapidly synthesized.

Description

Dual-gain HDR synthesis method and system of sCMOS sensor
Technical Field
The invention relates to the field of image processing, in particular to a dual-gain HDR synthesis method and system of an sCMOS sensor.
Background
High Dynamic Range (HDR) imaging technology plays an increasingly important role in the field of modern image processing and computer vision. The main goal of HDR imaging techniques is to overcome the dynamic range limitations of conventional imaging systems, and to be able to capture and display details of the brightest and darkest portions of a scene simultaneously. Conventional imaging sensors often have difficulty in combining highlight and dark details in the scene with a single exposure, resulting in either overexposure (no details in the highlight portion) or underexposure (no details in the dark area) of the image. To address this problem, HDR imaging techniques have evolved.
Conventional HDR imaging methods are typically based on multiple exposure techniques, i.e. by exposing the same scene multiple times, acquiring multiple images at different exposure times, and synthesizing these images into one HDR image. Although effective, this approach has several significant drawbacks:
(1) Image registration is difficult, namely, because of the time required by multiple exposure, moving objects in a scene can cause inconsistent images, the complexity of image registration is increased, and artifacts such as ghosts (ghosting) are easy to appear.
(2) The processing complexity is high, and the multiple exposure method needs a complex image processing algorithm to align and synthesize images, thereby increasing the calculation cost and the processing time.
(3) The real-time performance is insufficient, multiple times of shooting are needed for multiple exposure, real-time HDR imaging is difficult to realize, and the application of the method in dynamic scenes is limited.
Aiming at the problem of image registration in the traditional multi-exposure HDR synthesis method, the dual-gain characteristic of the sCMOS sensor provides a new solution for HDR image synthesis. The sCMOS sensor can output two images with high gain and low gain at the same time, and can realize images with wide dynamic range through proper algorithm synthesis.
Disclosure of Invention
In view of this, in order to solve the problem of image registration in the traditional multiple exposure method and simplify the synthesis processing procedure of the HDR image, the invention provides a dual-gain HDR synthesis method and system of an sCMOS sensor, which can rapidly synthesize the image with high dynamic range by carrying out gray value linear relation analysis and gray stretching processing on the high-gain image and the low-gain image, aiming at eliminating the problem of image registration and simplifying the synthesis processing procedure of the HDR image. And the high-dynamic-range image is quickly synthesized by carrying out gray value linear relation analysis and gray stretching treatment on the high-gain image and the low-gain image.
The invention is realized by adopting the following technical scheme:
in a first aspect, the present invention provides a dual gain HDR synthesis method of an sCMOS sensor, the method comprising the steps of:
simultaneously acquiring two images of high Gain (GH) and low Gain (GL) using an sCMOS sensor;
carrying out gray value linear relation analysis on the high-gain image and the low-gain image;
Performing image histogram statistics, and determining gain and exposure time settings;
and carrying out gray stretching processing on the high-gain image and the low-gain image, and synthesizing an HDR image according to the gray value relation.
As a further aspect of the present invention, when the sCMOS sensor is used to simultaneously acquire two images of high gain and low gain, the high gain image is used to capture details of a darker portion of a scene, and the low gain image is used to capture details of a lighter portion of the scene.
As a further scheme of the invention, the inside of the sCMOS sensor is amplified by a dual gain channel and then quantized and output by an ADC converter, and an image with a higher dynamic range is synthesized outside the chip.
As a further scheme of the present invention, in the sCMOS sensor, if the voltage amplitude corresponding to the full well charge is V m, the output voltage range through the analog-to-digital converter is represented as [0, V m ], that is, the signal voltage entering the column bus is V e [0, V m ], the gain channel amplification factor is defined as G, KADC (V/DN) is defined as the ADC converter unit, and the expression of the output data D after the a bit precision quantization is:
When V is the minimum, i.e. system noise, at least the requirement is satisfied for the quantization of V into digital quantities to be read out At this time, the larger the gain G is, the smaller the signal data is read out.
As a further scheme of the invention, when the pixel does not reach saturation, the gray value relation of corresponding points of the high-gain image and the low-gain image is analyzed, and when the pixel is not saturated, the gray values output by the two gain channels are approximately equal to the set gain value ratio.
As a further aspect of the present invention, in the sCMOS sensor, when an image is read using two channels, the dynamic range of readable data is defined as:
Where max (D L) represents the maximum quantization value of the low-gain channel output, the maximum quantization value is 2 a-1,min(DH) represents the minimum data of the high-gain channel that can be quantized, the minimum data of the high-gain channel that can be quantized is 0, the dynamic range of the sensor readout image data with the dual-gain channel design is improved by 20×log (G H/GL) compared with the single channel, and the HDR image is obtained by synthesizing the LDR images of the two-channel output.
As a further aspect of the present invention, when performing image histogram statistics and determining gain and exposure time settings, if the gray scale range of the a bit is equally divided into b gray scales, where a and b are both even numbers, the number of dark pixels (gray scale value is 0) is defined (S low), the number of saturated pixels (gray scale value is 2 a -1) (S high), and the numbers of pixels located at the upper and lower portions in the histograms of the two images are calculated as:
Wherein P is the occupation of the number of pixels in the histogram, s is low gain, l is high gain, H is the image histogram, H is the gray level (1-b), N is the number of all pixels, when P t,s≈80％、Pt,l is about 80%, the gain and exposure time are adjusted, P t,s is the occupation of the number of pixels in the low gain histogram, H t,s (H) is the number of pixels in the gray level (1-b) in the low gain histogram, P t,l is the occupation of the number of pixels in the high gain histogram, and H t,l (H) is the number of pixels in the gray level (1-b) in the high gain histogram.
As a further scheme of the present invention, in the case that the pixel does not reach saturation, the gray value relationship of the corresponding points of the high gain image and the low gain image is analyzed as follows:
Hx,y＝k*Lx,y+b Hx,y<2a-1
Where H x,y represents the high-gain image gray value, L x,y represents the low-gain image gray value, k and b represent coefficients, each set of gain settings results in a set of fixed k and b, and x and y represent pixel coordinates.
As a further scheme of the present invention, when the gray stretching process is performed on the high-gain image and the low-gain image, and the HDR image is synthesized according to the gray value relationship, the idea of directly synthesizing useful information in the high-gain image and the low-gain image into the HDR image is combined, and when the following formula is satisfied:
Wherein a L represents a division point of the high-gain image histogram, a H represents a division point of the high-gain image histogram;
The regions of H x,y≤aL and L x,y≥aL are stretched separately using linear stretching to distribute the gray scale range of the fused image throughout [0,S ], then:
wherein H 'and L' are respectively expressed as H, L images after linear transformation of a certain gray level;
the synthesized image GHDR is represented as:
In a second aspect, the present invention also provides a dual gain HDR synthesis system of an sCMOS sensor for performing a dual gain HDR synthesis method of an sCMOS sensor, the dual gain HDR synthesis system comprising:
the dual-gain image acquisition module is used for simultaneously acquiring two images with high gain and low gain by using the sCMOS sensor;
the gray value linear relation analysis module is used for carrying out gray value linear relation analysis on the high-gain image and the low-gain image;
the image histogram statistics module is used for carrying out image histogram statistics and determining gain and exposure time settings;
the gray stretching module is used for carrying out gray stretching treatment on the high-gain image and the low-gain image;
And the HDR image synthesis module is used for synthesizing the HDR image according to the gray value relation.
The invention also includes a computer device comprising at least one processor and a memory communicatively coupled to the at least one processor, wherein the memory stores instructions executable by the at least one processor to cause the at least one processor to perform the dual gain HDR synthesis method of the sCMOS sensor.
The invention also includes a computer readable storage medium storing computer instructions for causing the computer to perform the dual gain HDR synthesis method of the sCMOS sensor.
Compared with the prior art, the dual-gain HDR synthesis method and system of the sCMOS sensor provided by the invention provide a high-efficiency and accurate HDR image generation scheme aiming at the limitations of the traditional HDR imaging method, especially the problems of difficult image registration, high processing complexity, insufficient real-time performance and the like in a multiple exposure method, and have the following beneficial effects:
1. The image registration problem is avoided. The invention utilizes the dual-gain characteristic of the sCMOS sensor to simultaneously acquire the high-gain image and the low-gain image at the same time, thereby fundamentally avoiding the image registration problem caused by multiple exposure. Since the two images are acquired within the same frame time, the problem of image inconsistency caused by the movement of objects in the scene is avoided, and the ghosts (ghosting) and artifacts (artifacts) caused by registration errors are eliminated.
2. The dynamic range is improved. According to the invention, through analyzing the gray value relation between the high-gain image and the low-gain image and carrying out corresponding gray stretching and synthesizing treatment, the system can realize high dynamic range imaging under single exposure. The high gain image captures low brightness details while the low gain image retains high brightness partial details, the organic combination of the two can cover a wider brightness range, so that the image can simultaneously reveal dark and bright details in the scene.
3. Simplifying the process flow. Compared with the traditional multiple exposure method, the method of the system does not need complex image registration and alignment processing, and simplifies the flow of HDR image synthesis. Through accurate gray value linear relation analysis and gray stretching processing, fusion of high-gain and low-gain images can be rapidly and effectively completed, the calculation complexity is reduced, and the processing efficiency is improved.
4. The real-time performance is enhanced. Since the sCMOS sensor is capable of simultaneously outputting high-gain and low-gain images within a single frame time, the method can support real-time HDR imaging. This is particularly important for applications in dynamic scenarios, such as in the fields of video monitoring, real-time scientific experimental imaging, real-time medical imaging, etc., where the response speed and imaging quality of the system can be significantly improved.
5. The image quality is improved. The invention can effectively improve the contrast and detail expressive force of the image through the gray stretching and synthesizing strategies. The combination of the high gain image and the low gain image not only preserves the details of the highlight and low light regions, but also avoids tone mapping problems common in conventional HDR image synthesis, making the final HDR image more visually natural and vivid.
6. The hardware cost is reduced. Compared with a multi-exposure camera system which needs to be specially designed, the sCMOS sensor-based dual-gain HDR imaging system utilizes the dual-gain output characteristic of the existing sensor, does not need additional hardware transformation or complicated mechanical shutter control, and reduces the hardware cost and complexity of the system.
7. System robustness is enhanced. Since the method of the present invention relies on the characteristics of the sCMOS sensor itself, rather than an externally complex mechanical structure or multiple exposure control, the robustness of the system in practical applications is significantly enhanced. The risk of mechanical failure is reduced, and the reliability and stability of the system are improved.
In summary, the dual-gain HDR synthesis method and system based on the sCMOS sensor provided by the invention realize an efficient, accurate and real-time HDR imaging scheme by utilizing the dual-gain reading characteristic of the sensor, overcome a plurality of defects of the traditional multi-exposure method, and have significant application value and technical advantages.
These and other aspects of the invention will be more readily apparent from the following description of the embodiments. It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention as claimed.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the related art, a brief description of the drawings is provided below, which are intended to provide a further understanding of the present invention and constitute a part of the specification, together with the embodiments of the present invention, serve to explain the present invention and not to limit the present invention. In the drawings:
fig. 1 is a flow chart of a dual gain HDR synthesis method of an sCMOS sensor according to an embodiment of the present invention.
Fig. 2 is a schematic diagram of an ideal dual gain channel relationship in a dual gain HDR synthesis method of an sCMOS sensor according to an embodiment of the present invention.
Fig. 3 is a graph showing the gray value relationship between the high gain channel and the low gain channel in the dual-gain HDR synthesis method of the sCMOS sensor according to the embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present invention more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
In some of the flows described in the specification and claims of the present invention and in the foregoing figures, a plurality of operations occurring in a particular order are included, but it should be understood that the operations may be performed out of order or performed in parallel, with the order of operations such as 101, 102, etc., being merely used to distinguish between the various operations, the order of the operations themselves not representing any order of execution. In addition, the flows may include more or fewer operations, and the operations may be performed sequentially or in parallel. It should be noted that, the descriptions of "first" and "second" herein are used to distinguish different messages, devices, modules, etc., and do not represent a sequence, and are not limited to the "first" and the "second" being different types.
Technical solutions in exemplary embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in exemplary embodiments of the present invention, and it is apparent that the described exemplary embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to fall within the scope of the invention.
Because of the difficult image registration of the traditional multi-exposure HDR synthesis method, the dual gain characteristic of the sCMOS sensor provides a new solution for HDR image synthesis. The sCMOS sensor can output two images with high gain and low gain at the same time, and can realize images with wide dynamic range through proper algorithm synthesis. According to the dual-gain HDR synthesis method and system of the sCMOS sensor, provided by the invention, the high-dynamic range image can be quickly synthesized by carrying out gray value linear relation analysis and gray stretching processing on the high-gain image and the low-gain image, so that the problem of image registration is solved, and the synthesis processing process of the HDR image is simplified. And the high-dynamic-range image is quickly synthesized by carrying out gray value linear relation analysis and gray stretching treatment on the high-gain image and the low-gain image.
The technical scheme of the invention is further described below with reference to specific embodiments:
Referring to fig. 1, fig. 1 is a flowchart of a dual-gain HDR synthesis method of an sCMOS sensor according to the present invention. The dual-gain HDR synthesis method of the sCMOS sensor provided by the embodiment of the invention comprises the following steps of:
step S10, simultaneously acquiring two images of high gain and low gain by using an sCMOS sensor.
And S20, carrying out gray value linear relation analysis on the high-gain image and the low-gain image.
And S30, carrying out image histogram statistics, and determining gain and exposure time settings.
Step S40, gray stretching processing is carried out on the high-gain image and the low-gain image, and the HDR image is synthesized according to the gray value relation.
In the dual-gain HDR synthesis method and system of the sCMOS sensor of the present embodiment, when the sCMOS sensor is used to simultaneously acquire two images of high gain and low gain, the high gain image is used to capture details of a darker portion of a scene, and the low gain image is used to capture details of a lighter portion of the scene. The inside of the sCMOS sensor is amplified by a dual gain channel and then quantized and output by an ADC converter, and an image with a higher dynamic range is synthesized outside the chip.
In this embodiment, if the voltage amplitude corresponding to the full-well charge is V m, the output voltage range through the analog-to-digital converter is represented as [0, V m ], that is, the signal voltage entering the column bus is V e [0, V m ], the gain channel amplification factor is defined as G, KADC (V/DN) is defined as the ADC converter unit, and the expression of the output data D after the a bit precision quantization is:
When V is the minimum, i.e. system noise, at least the requirement is satisfied for the quantization of V into digital quantities to be read out At this time, the larger the gain G is, the smaller the signal data is read out.
The relation diagram of the quantized data output by the dual gain channel and the electron number in the single pixel after photoelectric conversion is shown in fig. 2, the horizontal axis represents the electron number Q in the single pixel after photoelectric conversion, the vertical axis represents the value D quantized by the ADC, and for the image information, the horizontal axis is used for representing the gray value of the corresponding pixel point. G H denotes a high gain channel, G L denotes a low gain channel, and F Q denotes the number of full well charges. It can be seen from fig. 2 that in the case that the pixel does not reach saturation, the gray value relationship of the corresponding points of the high-gain image and the low-gain image is analyzed, and in the case that the pixel does not reach saturation, the gray values output by the two gain channels are approximately equal to the set gain value ratio.
The dynamic range of the data read out for a single channel is 2 a -1:1, in the sCMOS sensor of this embodiment, when the image is read out using two channels, the dynamic range of the readable data is defined as:
Where max (D L) represents the maximum quantization value output by the low-gain channel, the maximum quantization value is 2 a-1,min(DH) represents the minimum data quantifiable by the high-gain channel, the minimum data quantifiable by the high-gain channel is 0, the dynamic range of the image data read out by the sensor with the dual-gain channel design is improved by 20 x log (G H/GL) compared with the single channel, and the HDR image with better visual effect is obtained by synthesizing the LDR images output by the two channels.
In this embodiment, when performing image histogram statistics and determining gain and exposure time settings, if the gray scale range of abit is equally divided into b gray scales, where a and b are both even numbers, the number of dark pixels (gray scale value is 0) is defined (S low), the number of saturated pixels (gray scale value is 2 a -1) (S high), and the numbers of pixels located at the upper and lower portions in the histograms of the two images are calculated as:
Wherein P is the occupation of the number of pixels in the histogram, s is low gain, l is high gain, H is the image histogram, H is the gray scale (1-b), N is the number of all the pixels, and when P t,s≈80％、Pt,l is about 80%, the gain and the exposure time are adjusted.
Under the condition that the pixels do not reach saturation, the gray value relation of the corresponding points of the high-gain image and the low-gain image is analyzed as follows:
Hx,y＝k*Lx,y+b Hx,y<2a-1
Wherein H x,y represents a high-gain image gray value, L x,y represents a low-gain image gray value, k and b represent coefficients, a plurality of groups of image verification are used, a high-gain channel gray value relation curve of the high-gain image gray value and the low-gain channel gray value relation curve is drawn as shown in fig. 3, an abscissa is a low-gain brightness value, an ordinate represents a high-gain brightness value, and a conclusion is drawn from a plurality of groups of different gain combination corresponding relations that in an unsaturated area, the two represent obvious linear relations, and the relation is still established for shooting of different scenes, namely each group of gain settings is obtained, and a group of fixed k and b is obtained.
In this embodiment, gray stretching is performed on the high-gain image and the low-gain image, when synthesizing the HDR image according to the gray value relationship, the idea of directly synthesizing useful information in the high-gain image and the low-gain image into the HDR image is combined, and when the following formula is satisfied:
Then { SH u HL } can obtain an HDR image that retains the details in both figures and the synthesized image is smooth. The algorithm provided by the invention is a method for acquiring an original image by using Lapray, so that a H =s/2 can be directly set, and then the segmentation point a L of the low-gain image can be directly acquired according to a formula. Meanwhile, in order to better display the image effect, gray stretching is added in the fusion process, so that the effect of improving the image contrast is achieved.
The areas of H x,y≤aL and L x,y≥aL are stretched by linear stretching, so that the gray scale range of the fused image is distributed in the whole [0,S ], and then:
wherein H 'and L' are respectively expressed as H, L images after linear transformation of a certain gray level;
the synthesized image GHDR is represented as:
In the dual-gain HDR synthesis method of the sCMOS sensor, the dual-gain characteristic of the sCMOS sensor is utilized to simultaneously acquire high-gain and low-gain images at the same time, so that the problem of image registration caused by multiple exposure is fundamentally avoided. Since the two images are acquired within the same frame time, the problem of image inconsistency caused by the movement of objects in the scene is avoided, and the ghosts (ghosting) and artifacts (artifacts) caused by registration errors are eliminated. According to the invention, through analyzing the gray value relation between the high-gain image and the low-gain image and carrying out corresponding gray stretching and synthesizing treatment, the system can realize high dynamic range imaging under single exposure. The high gain image captures low brightness details while the low gain image retains high brightness partial details, the organic combination of the two can cover a wider brightness range, so that the image can simultaneously reveal dark and bright details in the scene.
Compared with the traditional multiple exposure method, the method of the system does not need complex image registration and alignment processing, and simplifies the flow of HDR image synthesis. Through accurate gray value linear relation analysis and gray stretching processing, fusion of high-gain and low-gain images can be rapidly and effectively completed, the calculation complexity is reduced, and the processing efficiency is improved. Since the sCMOS sensor is capable of simultaneously outputting high-gain and low-gain images within a single frame time, the method can support real-time HDR imaging. This is particularly important for applications in dynamic scenarios, such as in the fields of video monitoring, real-time scientific experimental imaging, real-time medical imaging, etc., where the response speed and imaging quality of the system can be significantly improved.
The invention can effectively improve the contrast and detail expressive force of the image through the gray stretching and synthesizing strategies. The combination of the high gain image and the low gain image not only preserves the details of the highlight and low light regions, but also avoids tone mapping problems common in conventional HDR image synthesis, making the final HDR image more visually natural and vivid. Compared with a multi-exposure camera system which needs to be specially designed, the sCMOS sensor-based dual-gain HDR imaging system utilizes the dual-gain output characteristic of the existing sensor, does not need additional hardware transformation or complicated mechanical shutter control, and reduces the hardware cost and complexity of the system. Since the method of the present invention relies on the characteristics of the sCMOS sensor itself, rather than an externally complex mechanical structure or multiple exposure control, the robustness of the system in practical applications is significantly enhanced. The risk of mechanical failure is reduced, and the reliability and stability of the system are improved.
It should be understood that although described in a certain order, the steps are not necessarily performed sequentially in the order described. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, some steps of the present embodiment may include a plurality of steps or stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of the steps or stages is not necessarily sequential, but may be performed alternately or alternately with at least a part of the steps or stages in other steps or other steps.
In one embodiment, the present invention provides a dual gain HDR synthesis system of an sCMOS sensor for performing the dual gain HDR synthesis method of the sCMOS sensor described above, the dual gain HDR synthesis method comprising:
the dual-gain image acquisition module is used for simultaneously acquiring two images with high gain and low gain by using the sCMOS sensor;
the gray value linear relation analysis module is used for carrying out gray value linear relation analysis on the high-gain image and the low-gain image;
the image histogram statistics module is used for carrying out image histogram statistics and determining gain and exposure time settings;
the gray stretching module is used for carrying out gray stretching treatment on the high-gain image and the low-gain image;
And the HDR image synthesis module is used for synthesizing the HDR image according to the gray value relation.
In this embodiment, the camera performs the steps of the dual gain HDR synthesis method of the sCMOS sensor, and therefore the operation of the camera will not be described in detail in this embodiment.
In summary, the dual-gain HDR synthesis method and system of the sCMOS sensor realize an efficient, accurate and real-time HDR imaging scheme by utilizing the dual-gain reading characteristic of the sensor, overcome a plurality of defects of the traditional multi-exposure method, and have remarkable application value and technical advantages.
In one embodiment, there is also provided in an embodiment of the invention a computer device comprising at least one processor, and a memory communicatively coupled to the at least one processor, the memory storing instructions executable by the at least one processor to cause the at least one processor to perform the steps of the dual gain HDR synthesis method of the sCMOS sensor.
In one embodiment, the present invention also provides a computer readable storage medium storing computer instructions for causing the computer to perform the steps of the dual gain HDR synthesis method of the sCMOS sensor.
Those skilled in the art will appreciate that implementing all or part of the above described methods in accordance with the embodiments may be accomplished by instructing the associated hardware by a computer program characterized by computer instructions, which may be stored on a non-transitory computer readable storage medium, which when executed may comprise the steps of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in embodiments provided herein may include at least one of non-volatile and volatile memory.
The non-volatile memory may include read-only memory, magnetic tape, floppy disk, flash memory, optical memory, etc. Volatile memory can include random access memory or external cache memory. By way of illustration, and not limitation, RAM can take many forms, such as static random access memory or dynamic random access memory.
The foregoing description of the preferred embodiments of the invention is not intended to be limiting, but rather is intended to cover all modifications, equivalents, and alternatives falling within the spirit and principles of the invention.

Claims (10)

1. A dual gain HDR synthesis method of an sCMOS sensor, comprising the steps of:
Simultaneously acquiring two images of high gain and low gain by using an sCMOS sensor;
carrying out gray value linear relation analysis on the high-gain image and the low-gain image;
Performing image histogram statistics, and determining gain and exposure time settings;
gray stretching processing is carried out on the high-gain image and the low-gain image, and an HDR image is synthesized according to the gray value relation;
The useful information in the high-low gain image is directly synthesized into an HDR image, and the high-low gain gray scale characteristic is combined, when the following formula is satisfied:
,
using linear stretching to respectively pair AndIs stretched in the region of (a).
2. The dual gain HDR synthesis method of an sCMOS sensor of claim 1 wherein the high gain image is used to capture details of darker portions of the scene and the low gain image is used to capture details of lighter portions of the scene when both high gain and low gain images are acquired simultaneously using the sCMOS sensor.
3. The dual-gain HDR synthesis method of an sCMOS sensor of claim 2, wherein the sCMOS sensor is internally amplified by dual-gain channels and then quantized by an ADC converter to output, and an image with a higher dynamic range is synthesized off-chip.
4. The method for dual gain HDR synthesis of sCMOS sensor according to claim 1, wherein if the voltage amplitude corresponding to full well charge in sCMOS sensor isThe output voltage range through the analog-to-digital converter is denoted as 0,I.e., the signal voltage entering the column bus is V e 0,Defining gain channel amplification as,For the ADC converter unit, the expression of the output data D after the a bit precision quantization is:
,
When (when) Is the minimum, i.e. the system noise, is the minimum, i.e. the system noiseQuantized to digital quantity readout, at least required to satisfyGain at this timeThe larger the value, the smaller the data of the signal are read out correspondingly.
5. The method for synthesizing dual gain HDR of sCMOS sensor according to claim 4, wherein the gray value relation between corresponding points of high gain image and low gain image is analyzed under the condition that pixel does not reach saturation, and the gray values outputted by two gain channels are approximately equal to the set gain value under the condition of unsaturation.
6. The method of dual gain HDR synthesis of an sCMOS sensor of claim 5, wherein in the sCMOS sensor, when the image is read with two channels, the dynamic range of the readable data is defined as:
,
In the formula, Representing the maximum quantized value of the low gain channel output, the maximum quantized value being,Representing the minimum data quantifiable by the high gain channel, which is 0, the dynamic range of the sensor readout image data with the dual gain channel design is improved compared to a single channelAn HDR image is obtained by synthesizing the LDR images output by the two channels.
7. The method of dual gain HDR synthesis for sCMOS sensor according to claim 6, wherein when making image histogram statistics and determining gain and exposure time settings, if an a bit gray scale range is equally divided into b gray scales, where a and b are both even numbers, the number of dark pixels is definedThe gray value is 0, and the gray value of the saturated pixel point isNumber of (3)The number of pixels located at the upper part and the lower part in the histograms of the two images is calculated as follows:
,
,
Wherein P is the occupation of the number of pixels in the histogram, s is low gain, l is high gain, H is the image histogram, H is the gray scale (1-b), N is the number of all pixels, when satisfying 、The gain and exposure time are adjusted; representing a occupation of the number of pixels in the low gain histogram; The number of pixel points representing the gray scale level (1-b) in the low gain histogram; representing a fraction of the number of pixels in the high gain histogram; the number of pixels representing the gray scale level (1-b) in the high gain histogram.
8. The method for dual gain HDR synthesis of an sCMOS sensor of claim 7 wherein, in the event that the pixel is not saturated, analyzing the gray value relationship between the corresponding points of the high gain image and the low gain image is:
,
In the formula, Representing the gray value of the high gain image,Representing low gain image gray values, k, b representing coefficients, each set of gain settings results in a set of fixed k, b.
9. The method for synthesizing the dual-gain HDR of the sCMOS sensor of claim 8, wherein when the high-gain image and the low-gain image are subjected to gray stretching processing and the HDR image is synthesized according to the gray value relationship, the gray range of the fused image is distributed over [0,S ], and then:
,
,
In the formula, 、Respectively indicated as H, L, and subjected to gray level linear transformation;
the synthesized image GHDR is represented as:
,
。
10. A dual gain HDR synthesis system of an sCMOS sensor for performing the dual gain HDR synthesis method of an sCMOS sensor of any of claims 1-9, the system comprising:
the dual-gain image acquisition module is used for simultaneously acquiring two images with high gain and low gain by using the sCMOS sensor;
the gray value linear relation analysis module is used for carrying out gray value linear relation analysis on the high-gain image and the low-gain image;
the image histogram statistics module is used for carrying out image histogram statistics and determining gain and exposure time settings;
the gray stretching module is used for carrying out gray stretching treatment on the high-gain image and the low-gain image;
The HDR image synthesis module is used for synthesizing an HDR image according to the gray value relation, wherein useful information in the high-low gain image is directly synthesized into the HDR image, and the gray characteristics of the high-low gain are combined, when the following formula is satisfied:
,
using linear stretching to respectively pair AndIs stretched in the region of (a).
CN202411490650.9A 2024-10-24 2024-10-24 Dual-gain HDR synthesis method and system for sCMOS sensors Active CN119364204B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202411490650.9A CN119364204B (en) 2024-10-24 2024-10-24 Dual-gain HDR synthesis method and system for sCMOS sensors

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202411490650.9A CN119364204B (en) 2024-10-24 2024-10-24 Dual-gain HDR synthesis method and system for sCMOS sensors

Publications (2)

Publication Number Publication Date
CN119364204A CN119364204A (en) 2025-01-24
CN119364204B true CN119364204B (en) 2025-09-16

Family

ID=94302418

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202411490650.9A Active CN119364204B (en) 2024-10-24 2024-10-24 Dual-gain HDR synthesis method and system for sCMOS sensors

Country Status (1)

Country Link
CN (1) CN119364204B (en)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106713907A (en) * 2017-02-21 2017-05-24 京东方科技集团股份有限公司 Display HDR image display performance evaluation method and device
CN115589471A (en) * 2022-09-14 2023-01-10 北京空间机电研究所 A Best Gain Ratio Discrimination Method Applicable to Large Dynamic Range Image Fusion

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106713907A (en) * 2017-02-21 2017-05-24 京东方科技集团股份有限公司 Display HDR image display performance evaluation method and device
CN115589471A (en) * 2022-09-14 2023-01-10 北京空间机电研究所 A Best Gain Ratio Discrimination Method Applicable to Large Dynamic Range Image Fusion

Also Published As

Publication number Publication date
CN119364204A (en) 2025-01-24

Similar Documents

Publication Publication Date Title
US20150022687A1 (en) System and method for automatic exposure and dynamic range compression
US8582814B2 (en) Image processing method and system
Akyüz Deep joint deinterlacing and denoising for single shot dual-ISO HDR reconstruction
CN103297701B (en) Formation method and imaging device
JP6312487B2 (en) Image processing apparatus, control method therefor, and program
CN101072303A (en) Image pickup device and image pickup apparatus
KR101023944B1 (en) Image processing apparatus and method
US7949182B2 (en) Combining differently exposed images of the same object
JP2020524422A (en) Imaging noise reduction system and method
CN113905185B (en) Image processing methods and devices
US20150221095A1 (en) Image processing apparatus, image processing method, and program
CN202535464U (en) Imaging device
US20050099508A1 (en) Dynamic range broadening method for a solid-state image sensor including photosensitive cells each having a main and a subregion
CN115082335B (en) A method for image denoising based on noise modeling
JP2023525846A (en) Information processing device and method
CN119364204B (en) Dual-gain HDR synthesis method and system for sCMOS sensors
JP7676160B2 (en) Imaging device, computer program, and storage medium
CN115601248A (en) HDR method, system, storage medium and computer equipment for real-time video
JP4903246B2 (en) Brightness correction circuit for image sensor
JP2014176065A (en) Image pickup device and method
JP2009302850A (en) Noise removal device of solid-state image sensor, image capturing apparatus, noise removal method of solid-state image sensor
CN114494080A (en) Image generation method and device, electronic equipment and storage medium
CN119417740B (en) SCMOS-based real-time high dynamic range image processing method and system
CN119729244B (en) Video data processing method, device, equipment and computer readable storage medium
US20240251172A1 (en) Method for processing pixel data, corresponding device and program

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant