CN119211494A - Method for acquiring parameters of ambient light and projection surface and spectral imaging device - Google Patents
Method for acquiring parameters of ambient light and projection surface and spectral imaging device Download PDFInfo
- Publication number
- CN119211494A CN119211494A CN202310762415.1A CN202310762415A CN119211494A CN 119211494 A CN119211494 A CN 119211494A CN 202310762415 A CN202310762415 A CN 202310762415A CN 119211494 A CN119211494 A CN 119211494A
- Authority
- CN
- China
- Prior art keywords
- spectrum
- projection surface
- ambient light
- set image
- light
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 76
- 238000000701 chemical imaging Methods 0.000 title claims abstract description 55
- 238000001228 spectrum Methods 0.000 claims abstract description 196
- 238000000985 reflectance spectrum Methods 0.000 claims description 37
- 239000003086 colorant Substances 0.000 claims description 25
- 230000003595 spectral effect Effects 0.000 claims description 23
- 238000002310 reflectometry Methods 0.000 claims description 18
- 230000004044 response Effects 0.000 claims description 16
- 239000000758 substrate Substances 0.000 claims description 8
- 238000013528 artificial neural network Methods 0.000 claims description 7
- 239000011159 matrix material Substances 0.000 claims description 7
- 230000035945 sensitivity Effects 0.000 claims description 3
- 238000013507 mapping Methods 0.000 claims description 2
- 238000010586 diagram Methods 0.000 description 23
- 238000000411 transmission spectrum Methods 0.000 description 12
- 230000000694 effects Effects 0.000 description 11
- 238000004364 calculation method Methods 0.000 description 9
- 238000001914 filtration Methods 0.000 description 8
- 230000003287 optical effect Effects 0.000 description 8
- 230000007613 environmental effect Effects 0.000 description 7
- 238000003384 imaging method Methods 0.000 description 7
- 230000008901 benefit Effects 0.000 description 6
- 238000012545 processing Methods 0.000 description 5
- 239000000463 material Substances 0.000 description 4
- 238000012360 testing method Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 239000002096 quantum dot Substances 0.000 description 3
- 238000011084 recovery Methods 0.000 description 3
- 230000036541 health Effects 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 239000002070 nanowire Substances 0.000 description 2
- 230000008447 perception Effects 0.000 description 2
- 239000002994 raw material Substances 0.000 description 2
- 238000007792 addition Methods 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- BJQHLKABXJIVAM-UHFFFAOYSA-N bis(2-ethylhexyl) phthalate Chemical compound CCCCC(CC)COC(=O)C1=CC=CC=C1C(=O)OCC(CC)CCCC BJQHLKABXJIVAM-UHFFFAOYSA-N 0.000 description 1
- 210000004556 brain Anatomy 0.000 description 1
- 230000036755 cellular response Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000000354 decomposition reaction Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000004744 fabric Substances 0.000 description 1
- 239000002061 nanopillar Substances 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 239000004038 photonic crystal Substances 0.000 description 1
- 238000012805 post-processing Methods 0.000 description 1
- 230000006798 recombination Effects 0.000 description 1
- 238000005215 recombination Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000002834 transmittance Methods 0.000 description 1
- 238000001429 visible spectrum Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N9/00—Details of colour television systems
- H04N9/12—Picture reproducers
- H04N9/31—Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM]
- H04N9/3179—Video signal processing therefor
- H04N9/3182—Colour adjustment, e.g. white balance, shading or gamut
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
Landscapes
- Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Chemical & Material Sciences (AREA)
- Analytical Chemistry (AREA)
- Biochemistry (AREA)
- General Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- Immunology (AREA)
- Pathology (AREA)
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Spectrometry And Color Measurement (AREA)
Abstract
The application relates to a method and a spectral imaging device for acquiring parameters of ambient light and a projection surface. The method for acquiring parameters of the environment light and the projection surface comprises the steps of determining a base light spectrum of a set image, wherein the set image comprises a plurality of color blocks with different brightness and any color, projecting the set image to a target area of the projection surface, acquiring a reflection spectrum of the set image reflected from the target area of the projection surface, and acquiring the environment light spectrum and the reflection spectrum of the projection surface based on the reflection spectrums corresponding to the color blocks of the set image. In this way, the parameters of the ambient light and the projection surface can be conveniently and accurately acquired.
Description
Technical Field
The present application relates to the field of optical processing technology, and more particularly, to a method and a spectral imaging apparatus for acquiring parameters of ambient light and a projection surface.
Background
A projection device, also known as a projector or projector, is an electronic device that can project image content onto a target area (e.g., a projection curtain area, a wall whitespace area, etc.). Currently, prior to projecting image content onto a target area by a projection device, the projection device needs to be dimmed to ensure accurate display.
The present two main external factors influencing the projection effect of the projection device, namely the environment light and the projection surface of the projected target area, so that the projection effect of the projection device can be obviously improved if the parameters of the environment light and the projection surface can be accurately acquired.
It is therefore desirable to provide a solution for acquiring parameters of the ambient light and the projection surface.
Disclosure of Invention
The embodiment of the application provides a method for acquiring parameters of ambient light and a projection surface and a spectrum imaging device, which can calculate the parameters of the ambient light and the projection surface based on reflection spectrums of set images comprising a plurality of color blocks in different areas of the projection surface, so that the parameters of the ambient light and the projection surface can be acquired conveniently and accurately.
According to an aspect of the present application, there is provided a method of acquiring parameters of ambient light and a projection surface, including determining a base light spectrum of a set image including a plurality of color patches of arbitrary colors of different brightnesses, projecting the set image to a target area of the projection surface and acquiring a reflection spectrum of the set image reflected from the target area of the projection surface, and acquiring the ambient light spectrum and the reflection spectrum of the projection surface based on the reflection spectrums corresponding to the plurality of color patches of the set image.
In the above method for acquiring parameters of the ambient light and the projection surface, the set image is a single image including a plurality of color patches of arbitrary colors of different brightnesses, or a plurality of images respectively including a plurality of color patches of arbitrary colors of different brightnesses, each of the plurality of images includes one or more color patches, and the one or more color patches are of the same or different brightnesses and the same or different colors.
In the method for acquiring the parameters of the environment light and the projection surface, determining the substrate light color of at least one set image comprises the steps of projecting the substrate light with a known spectrum to a target area of the projection surface, and acquiring the reflection spectrum of the substrate light in the target area of the projection surface as the substrate light spectrum.
In the above method of acquiring parameters of the ambient light and the projection surface, the RGB colors of the base light of the known spectrum are (0, 0) colors.
In the above method of acquiring parameters of the ambient light and the projection surface, the RGB colors of the base light of the known spectrum may be adjusted.
In the above method of acquiring parameters of the ambient light and the projection surface, the known spectrum of the base light is a spectrum information parameter obtained by color space mapping.
In the above method for acquiring parameters of the ambient light and the projection surface, the reflection spectrum of the set image includes the reflection spectrum of the base light, and the reflection spectrums of the plurality of color patches in the set image and the reflection spectrum of the ambient light.
In the method for acquiring parameters of the ambient light and the projection surface, acquiring the ambient light spectrum and the reflection spectrum of the projection surface based on the reflection spectrums corresponding to the plurality of color patches of the set image includes defining reflection spectrums of respective areas corresponding to the respective color patches of the plurality of color patches based on the reflection spectrum of the set image.
In the above method for acquiring parameters of the environmental light and the projection surface, the reflection spectrum of each color block is acquired by a spectral imaging sensor or is a preset spectrum.
In the above method for acquiring parameters of the ambient light and the projection surface, acquiring the ambient light spectrum and the reflection spectrum of the projection surface based on the reflection spectrums corresponding to the plurality of color patches of the set image includes:
The reflection spectrum relation of the area corresponding to each color block is determined as follows:
Reflectance spectrum n= (base light spectrum + ambient light spectrum) reflectivity;
reflectance spectrum 1= (region 1 spectrum+ambient light spectrum) ×reflectance;
Reflectance spectrum 2= (region 2 spectrum + ambient light spectrum) reflectivity;
......
reflectance spectrum n-1= (region n-1 spectrum + ambient light spectrum).
In the above method for acquiring parameters of the ambient light and the projection surface, acquiring the ambient light spectrum and the reflection spectrum of the projection surface based on the reflection spectrums corresponding to the plurality of color patches of the set image further includes:
Define the Loss function Loss as:
Loss= |reflectance spectrum 1- (region 2 spectrum + ambient light spectrum) |reflectance|+|reflectance spectrum 2- (region 2 spectrum + ambient light spectrum) |++ reflectance spectrum n-1- (region n-1 spectrum + ambient light spectrum) |+|reflectance spectrum n- (base light spectrum + ambient light spectrum) |reflectance|| where|·||indicates the norm of the vector or matrix;
The best estimate of the ambient light spectrum and the reflectivity of the target area of the projection surface is calculated by minimizing the Loss function Loss by means of a least squares method.
In the above method of acquiring parameters of the ambient light and the projection surface, the reflectance of the ambient light spectrum and the target area of the projection surface is solved from the relationship by a neural network modeling method.
According to another aspect of the present application, there is provided a spectral imaging device for use in a method of acquiring parameters of ambient light and a projection surface as described above for acquiring a base light spectrum of a set image and/or a reflection spectrum of said set image.
In the spectral imaging apparatus as described above, acquiring the base light spectrum of the setting image and/or the reflection spectrum of the setting image includes:
determining a relation:
Ck＝∫S(λ)ρ(λ)τ(λ)fk(λ)α(λ)dλ;
Wherein C k represents the pixel value of the kth channel of the spectral imaging device, S (lambda) represents the ambient light spectrum, rho (lambda) represents the reflectivity of a target area of a projection surface, tau (lambda) represents the transmissivity of a lens of the spectral imaging device, f k (lambda) represents the transmissivity of the kth structure of a spectral imaging sensor of the spectral imaging device, and alpha (lambda) represents the spectral sensitivity of the spectral imaging sensor of the spectral imaging device itself;
The above parameters satisfy the relation:
S(λ)ρ(λ)=X;
τ(λ)fk(λ)α(λ)=T;
Ck=Y;
Where X k is the spectral image value of the input spectrum at the kth structure and Y is the response of the output at the corresponding structure, resulting in:
Y=TX。
In the spectral imaging apparatus as described above, a known plurality of color patches of the set image are calibrated in advance to calculate an input light spectrum X, which is the base light and/or the reflected light, by T of the known spectral imaging apparatus and a response Y of an obtained output.
According to a further aspect of the present application there is provided a terminal device comprising a spectral imaging device as described above.
The method for acquiring the parameters of the environment light and the projection surface and the spectrum imaging device provided by the embodiment of the application can calculate the parameters of the environment light and the projection surface based on the reflection spectrums of the set images comprising a plurality of color blocks in different areas of the projection surface, thereby conveniently and accurately acquiring the parameters of the environment light and the projection surface.
Drawings
Various other advantages and benefits of the present application will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the application. It is evident that the figures described below are only some embodiments of the application, from which other figures can be obtained without inventive effort for a person skilled in the art. Also, like reference numerals are used to designate like parts throughout the figures.
Fig. 1 illustrates a schematic flow chart of a method of acquiring parameters of ambient light and a projection surface according to an embodiment of the application.
Fig. 2 is a schematic diagram of different brightness color blocks of a set image in a method of acquiring parameters of ambient light and a projection surface according to an embodiment of the present application.
Fig. 3 illustrates a schematic diagram of a first example of a reflection spectrum of a set image in a method of acquiring parameters of an ambient light and a projection surface according to an embodiment of the present application.
Fig. 4 illustrates a schematic diagram of a second example of a reflection spectrum of a set image in a method of acquiring parameters of an ambient light and a projection surface according to an embodiment of the present application.
Fig. 5 illustrates a schematic diagram of respective regions defined in a reflection spectrum of a set image in a method of acquiring parameters of an ambient light and a projection surface according to an embodiment of the present application.
Fig. 6 illustrates a schematic of the band response of a cone cell to a spectral image.
Fig. 7 illustrates a schematic configuration diagram of a spectral imaging apparatus according to an embodiment of the present application.
Fig. 8 illustrates a schematic diagram of the principle of spectral imaging of a spectral imaging device according to an embodiment of the present application.
Fig. 9 illustrates spectral information of an image (210,250,230) emitted by a projector.
Fig. 10 illustrates a schematic diagram of adjusting a projection display effect by spectral information of ambient light and information of reflectivity of a target area of a projection surface.
Fig. 11 illustrates a schematic diagram of a display of a spectral imaging device application according to an embodiment of the application.
Fig. 12 illustrates a schematic diagram of a spectral imaging device applied to the security field according to an embodiment of the present application.
Fig. 13 illustrates a schematic diagram of a spectral imaging apparatus applied to the military field according to an embodiment of the present application.
Fig. 14 illustrates a schematic diagram of a spectral imaging apparatus applied to the field of medical health according to an embodiment of the present application.
Fig. 15 illustrates a schematic diagram of a spectral imaging apparatus applied to the field of automotive electronics according to an embodiment of the present application.
Detailed Description
Hereinafter, exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. It should be apparent that the described embodiments are only some embodiments of the present application and not all embodiments of the present application, and it should be understood that the present application is not limited by the example embodiments described herein.
Schematic method
Fig. 1 illustrates a schematic flow chart of a method of acquiring parameters of ambient light and a projection surface according to an embodiment of the application.
As shown in fig. 1, the method of acquiring parameters of the projection surface and the ambient light according to the embodiment of the present application includes the following steps.
S110, determining a base light spectrum of a set image, wherein the set image comprises a plurality of color blocks with any colors and different brightness.
Here, the setting image may be a single image including a plurality of color patches of arbitrary colors of different brightnesses, or a plurality of images each including a plurality of color patches of arbitrary colors of different brightnesses. Also, each of the plurality of images may contain one or more color patches of any luminance and any color, for example, the plurality of color patches are of the same luminance or different luminance, or the plurality of color patches are of the same color or different colors, and the plurality of color patches may also be of the same luminance and different colors, or the same color and different luminance, or the like. That is, in the embodiment of the present application, it is only necessary that the set image includes a plurality of color patches of arbitrary colors of different brightnesses, and the number of images of the set image is not limited, nor are the brightnesses and colors of the color patches included in each image.
The base light spectrum of the set image may be acquired, for example, when the projection device is not on, which may appear as a background black, for example as shown in fig. 2, where fig. 2 is a schematic diagram of different brightness color blocks of the set image in a method of acquiring parameters of the ambient light and the projection surface according to an embodiment of the application. It should be noted that, in practice, when the projection apparatus projects the set image onto the target area of the projection surface (e.g., a wall surface or a cloth surface, etc.), the black area will have a certain brightness due to the brightness of the substrate of the projection apparatus and the presence of ambient light (e.g., in a non-full black environment), and each square area corresponding to a plurality of color blocks with different brightness in the set image will also have color shift.
Therefore, in the embodiment of the present application, it is necessary to first acquire the spectrum of the base light of the set image in advance, which may be acquired by calibration in advance, for example, specifically, by projecting light of the RGB color (0, 0) of the known spectrum onto the target area of the projection surface, and acquiring the reflection spectrum at that time, that is, acquiring the reflectivity of the base light. Of course, other known spectra of light may be preset for calculation.
Here, it will be understood by those skilled in the art that in the embodiment of the present application, only the spectrum of the base light needs to be known, and the color coordinate values of the color space of the base light are not limited, for example, any color coordinate values of the RGB color space or other color spaces may be used. Also, in embodiments of the present application, the color coordinates, e.g., RGB coordinates, of the known spectrum of the base light are adjustable, and the known spectrum of the base light may also be spectral information parameters mapped from a color space by other means.
Thus, in a method of acquiring parameters of ambient light and a projection surface according to an embodiment of the present application, determining a base light color of at least one set image includes projecting base light of a known spectrum onto a target area of the projection surface, and acquiring a reflection spectrum of the base light at the target area of the projection surface as the base light spectrum.
S120, projecting the set image to a target area of a projection surface, and acquiring a reflection spectrum of the set image reflected from the target area of the projection surface.
Specifically, when the setting image is projected to the target area of the projection surface, since the setting image includes a plurality of color patches of arbitrary colors of different brightnesses, the reflection spectrum of the base light and the reflection spectrum of each color patch are included in the reflection spectrum of the setting image, and the ambient light also has a reflection spectrum for the target area of the projection surface.
Therefore, in the method of acquiring parameters of the environmental light and the projection surface according to the embodiment of the present application, the reflectance spectrum of the set image includes the reflectance spectrum of the base light, and the reflectance spectrums of the plurality of color patches in the set image and the reflectance spectrum of the environmental light.
Specifically, in the embodiment of the present application, the measured reflectance spectrum image of the set image may be as shown in fig. 3, for example, the test area may be square, or circular, or may be any shape, or may have a certain interval or may be a continuous area, and the projected several colors may be white, or may be a combination of various colors, such as four rows of color patches, where the intensities from left to right decrease sequentially, and the various colors may be white, RGB, or may have other layouts. Here, fig. 3 illustrates a schematic diagram of a first example of a reflection spectrum of a set image in a method of acquiring parameters of ambient light and a projection surface according to an embodiment of the present application, and fig. 4 illustrates a schematic diagram of a second example of a reflection spectrum of a set image in a method of acquiring parameters of ambient light and a projection surface according to an embodiment of the present application.
And S130, acquiring an ambient light spectrum and a reflection spectrum of a projection surface based on the reflection spectrums corresponding to the plurality of color blocks of the set image.
First, for the reflectance spectra of the set image as shown in fig. 3 and 4, reflectance spectra of respective areas corresponding to respective color patches of the plurality of color patches are defined as shown in fig. 5, where 1, 2..n-1 is a main test color patch and n is a base color patch. The reflection spectrum of each color block can be acquired by a spectral imaging sensor, or the spectrum of the test light of each color block can also use a preset spectrum. Here, fig. 5 illustrates a schematic diagram of respective regions defined in a reflection spectrum of a set image in a method of acquiring parameters of an ambient light and a projection surface according to an embodiment of the present application.
Then, obtaining the reflection spectrum of each region corresponding to each color block, wherein the reflection spectrum relationship of the region corresponding to each color block is as follows:
Reflectance spectrum n= (base light spectrum+ambient light spectrum) ×reflectance
Reflectance spectrum 1= (region 1 spectrum+ambient light spectrum) ×reflectance
Reflectance spectrum 2= (region 2 spectrum+ambient light spectrum) ×reflectance
......
Reflectance spectrum n-1 = (region n-1 spectrum + ambient light spectrum) ×reflectance
In the above, in the embodiment of the present application, since the spectrum and reflectivity of the ambient light are the parameter amounts that change every time, they are also the main objects of the solution according to the scheme of the embodiment of the present application.
It can be seen that the above problem is a noisy linear process solution problem, and can be solved by using various solving algorithms, where the method is not limited to a method for obtaining a spectrum to be measured by solving a matrix equation, and is generally performed by adopting a least square method or a neural network modeling method.
For example, a Loss function Loss may be defined as:
Loss= |reflectance spectrum 1- (region 2 spectrum + ambient light spectrum) |reflectance|+|reflectance spectrum 2- (region 2 spectrum + ambient light spectrum) |reflectance|+|reflectance spectrum n-1- (region n-1 spectrum + ambient light spectrum) |reflectance|+|reflectance spectrum n- (base light spectrum + ambient light spectrum) |, wherein I I.I I represents a vector or the norm of the matrix.
Then, the optimal estimated value of the ambient light spectrum and the reflectivity of the target area of the projection surface can be calculated by obtaining the Loss minimum by the least square method.
Or solving through a neural network modeling method, and the neural network modeling method can realize spectrum calculation on the NPU, so that the high spectrum precision is ensured, and meanwhile, the method has the characteristic of low power consumption, and is very suitable for mobile terminals such as mobile phones. In addition, the method can be also used for calculating the reflectivity of a target area of an ambient light spectrum and a projection surface in spectrum imaging in the fields of household appliances, automobiles and the like.
Here, considering that the environment used by the user is not constant every time the user uses the device, the method for acquiring the parameters of the environment light and the projection surface according to the embodiment of the application may be performed every time the projection device is turned on, so as to facilitate calculation of the environment light spectrum and the reflectivity of the target area of the projection surface.
Of course, if the accuracy is to be ensured, the calculation of the method for acquiring the parameters of the ambient light and the projection surface according to the embodiment of the present application may be further performed to obtain the current ambient light spectrum and the reflection spectrum of the target area of the projection surface.
Thus, the method for acquiring parameters of the environmental light and the projection surface according to the embodiment of the application considers that the projected image of the projection device is influenced by the environmental light and the projected target area, including but not limited to the materials of wall coverings and the like, namely, the reflectivities of the projected target areas of different materials are different, so that the projected light actually seen by human eyes has various color cast problems after reflection, and the parameters of the environmental light and the projection surface are calculated based on the reflection spectrums of the set images comprising a plurality of color blocks in the different areas of the projection surface, so that the parameters of the environmental light and the projection surface are conveniently and accurately acquired.
Schematic spectral imaging apparatus
As described above, in the method of acquiring parameters of the ambient light and the projection surface according to the embodiment of the present application, it is necessary to acquire the spectrum of the base light and the reflection spectrum information of the set image using the spectral imaging sensor, where since the set image includes a plurality of color patches of arbitrary colors of different brightness, the spectral imaging sensor is required to acquire the spectrum information of different wavelength bands, so that the accuracy is more accurate. In the case of using RGB, RGBIR, RGBW camera or the like, even if the number of channels thereof is small, the spectrum information of the relatively rough ambient light and the information of the reflectivity of the target area of the projection surface can be obtained by the method of acquiring the parameters of the ambient light and the projection surface according to the embodiment of the present application.
However, since RGB, RGBIR, RGBW camera can perceive a certain limitation of the spectral interval of light, it is not able to solve the full-band response, as shown in fig. 6. Here, fig. 6 illustrates a schematic diagram of band response of cone cells to a spectral image.
That is, since the projector apparatus needs to calculate the ambient spectrum and the reflectance in order to achieve a better projection display effect, the conventional RGB camera, rgbhr, RGBW camera can calculate the ambient spectrum and the reflectance according to the method as described above, but if the RGB camera is used, the calculation of the ambient spectrum and the reflectance is limited to the QE section (sensing section) of the RGB camera, and the ambient light other than the sensing section of the RGB camera cannot be sensed. These ambient lights also affect human perception. Therefore, the complete visible spectrum is missing, so that the scheme according to the embodiment of the application has poor effect. For example, as shown in FIG. 6, if a light ray of about 450nm is received, the S cell response is strongest and M, L cells response is weaker, then the human brain "marks" such light ray as blue after post-processing the perceived information. It can be seen that the cone cells can only respond to light within about 400-700nm, so that the human eye can only see the color of light in this band, namely what is known as visible light, but the RGB camera provides a much smaller perceived area than the human eye. Further, if the QE curve limitation is imposed on the 3 kinds of perception sections of the RGB camera, the accuracy of the above-described solving process is also lowered.
Thus, in an embodiment of the application, a method of acquiring parameters of ambient light and a projection surface according to an embodiment of the application is implemented using a spectral imaging device.
Here, fig. 7 illustrates a schematic configuration diagram of a spectral imaging apparatus according to an embodiment of the present application. As shown in fig. 7, in the spectral imaging apparatus according to the embodiment of the present application, the optical system is optional, which may be an optical system such as a lens assembly, a dodging assembly, or the like. The filtering structure is a broadband filtering structure in the frequency domain or the wavelength domain. The passband spectra of different wavelengths of the filter structure are not identical. The filter structure may be a structure or material having filter characteristics such as a super surface, a photonic crystal, a nano-pillar, a multilayer film, a dye, a quantum dot, MEMS (microelectromechanical system), FP etalon, CAVITY LAYER (resonator layer), waveguide layer (waveguide layer), a diffraction element, or the like. For example, in the embodiment of the present application, the light filtering structure may be a light modulating layer in chinese patent CN201921223201.2, and the image sensor (i.e., photodetector array) may be a CMOS Image Sensor (CIS), a CCD, an array photodetector, or the like. In addition, the optional data processing unit may be a processing unit such as MCU, CPU, GPU, FPGA, NPU, ASIC, which may export data generated by the image sensor to the outside for processing.
Specifically, after the image sensor measures light intensity information, for example, the light intensity information is transmitted to the data processing unit for restoration calculation. The specific process is described as follows:
The intensity signal of the incident light at different wavelengths λ is denoted as x (λ), the transmission spectrum curve of the filter structure is denoted as T (λ), the filter (filter structure) has m groups of structural units thereon, the transmission spectrums of each group of structural units are different from each other, and the filter structure may be denoted as Ti (λ) as a whole (i=1, 2,3,..m). The corresponding physical pixels are arranged below each group of structural units, and the light intensity bi modulated by the light filtering structure is detected. In the embodiment of the present application, for example, one physical pixel, that is, one physical pixel corresponds to one group of structural units, but the present application is not limited thereto, and a plurality of physical pixels may be one group corresponding to one group of structural units. Thus, in the spectral imaging apparatus according to the embodiment of the present application, the plurality of groups of the structural units constitute one "spectral pixel". Further, at least one spectral pixel may be used to restore the image. It should be noted that the number of the effective transmission spectrums (transmission spectrums for spectrum recovery, called effective transmission spectrums) Ti (λ) of the filter structure may be inconsistent with the number of the structural units, the transmission spectrums of the filter structure are manually set, tested, or calculated according to a certain rule according to the requirement of identification or recovery (for example, the transmission spectrums of each structural unit passing the test are effective transmission spectrums), so the number of the effective transmission spectrums of the filter structure may be smaller than the number of the structural units, or may be larger than the number of the structural units, for example, a certain transmission spectrum curve is not necessarily determined by a group of structural units.
The relationship between the spectral distribution of the incident light and the measured value of the image sensor can be expressed by the following equation:
Yi=∫x(λ)*Ti(λ)*T(λ)dλ
Discretizing to obtain:
Yi=Σ(x(λ)*Ti(λ)*T(λ))
Where T (λ) is the response of the image sensor, noted as:
Yi(λ)=Xi(λ)*T(λ),
the above equation can be extended to a matrix form:
Where Ti (i=1, 2,3,.., m) is the response of the image sensor after the light to be measured passes through the filter structure, and corresponds to the light intensity measurement values of the image sensor corresponding to the m structural units, respectively. X is the system's response to light of different wavelengths, determined by two factors, the filter structure transmittance and the quantum efficiency of the image sensor. X is a matrix, each row vector corresponding to the response of a set of building blocks to incident light of a different wavelength, where the incident light is sampled discretely and uniformly, for a total of n sampling points. The number of columns of X is the same as the number of samples of the incident light. Here, X (λ) is the intensity of the incident light at different wavelengths λ, i.e. the spectrum of the incident light to be measured.
In some examples, the optical filtering structure may be formed directly on the upper surface of the image sensor, for example, quantum dots, nanowires, etc., which directly form an optical filtering structure or material (nanowire, quantum dot, etc.) on the photosensitive area of the sensor, where it may be understood that the raw material of the image sensor is processed to form the optical filtering structure on the upper surface of the raw material when the image sensor is processed to form the image sensor, and the transmission spectrum and the response of the image sensor are integrated, that is, it may be understood that the response of the detector and the transmission spectrum are the same curve, and that the relationship between the spectral distribution of the incident light and the measured light intensity of the image sensor may be represented by the following formula:
Yi=Σ(x(λ)*Ti(λ))
further, at least one filter structure for modulating incident light is disposed on the image sensor having the filter structure. It will be appreciated that the image sensor (i.e. photodetector array) in the first embodiment may be a CMOS Image Sensor (CIS), CCD, array photodetector, etc. in the second embodiment, an image sensor integrated with a filtering structure is substituted.
At this time, the relationship between the spectral distribution of the incident light and the light intensity measurement value of the image sensor can be expressed by the following equation:
Yi=∫x(λ)*Ti(λ)*Ri(λ)dλ
Discretizing to obtain:
Yi=Σ(x(λ)*Ti(λ)*Ri(λ))
That is, in this embodiment, yi (λ) =ti (λ) ×xi (λ)
Because each spectrum imaging device needs to be provided with an optical system besides a spectrum chip, and comprises various lenses and the like, as shown in fig. 8, the spectrum imaging is obtained by the principle that after a target is obtained by a spectrum imaging sensor, the target is converted into an electric signal through a lens, an optical filter and the spectrum chip of a camera to be processed, and finally, the spectrum information of a corresponding shot object is obtained. Here, fig. 8 illustrates a schematic diagram of the principle of spectral imaging of the spectral imaging apparatus according to the embodiment of the present application.
Specifically, when (210,250,230) is emitted from the photographing projector as shown in fig. 8, the spectrum information of the acquired image is shown in fig. 9, where r=0.66992968, g=1.0, and b= 0.70530216 are measured in the projection display. Here, fig. 9 illustrates spectral information of an image (210,250,230) emitted from the projector.
Accordingly, the display effect of the projection can be adjusted by continuously based on the acquired spectral information of the ambient light and the information of the reflectance of the target area of the projection surface, as shown in fig. 10. Here, fig. 10 shows the projection display effect after adjustment by the spectral information of the ambient light and the information of the reflectance of the target area of the projection surface, wherein the display effect may further include optimization of information of chromaticity, luminance, color temperature, and the like. Here, fig. 10 illustrates a schematic diagram of adjusting the projection display effect by the spectral information of the ambient light and the information of the reflectance of the target area of the projection surface.
In performing calculation of the ambient light spectrum and the reflectance in the method of acquiring the parameters of the ambient light and the projection surface as described above using the spectral imaging apparatus, the following formula is adopted:
Ck＝∫S(λ)ρ(λ)τ(λ)fk(λ)α(λ)dλ
Wherein C k represents the pixel value of the kth channel, S (lambda) represents the spectrum of ambient light, ρ (lambda) represents the reflectivity of the target area of the projection surface, τ (lambda) represents the transmissivity of the lens, f k (lambda) represents the transmissivity of the kth structure, and α (lambda) represents the spectral sensitivity of the spectral imaging sensor itself of the spectral imaging device.
In the calculation formula corresponding to the spectrum recovery:
S(λ)ρ(λ)=X;
τ(λ)fk(λ)α(λ)=T;
Ck=Y
Where X k is the spectral image value of the input spectrum at the kth structure. Y is the output response corresponding to the structure, integrated to:
Y=TX
in the embodiment of the application, the known color blocks of the set image can be calibrated in advance, and the input light X, namely the acquired reflection spectrum of each corresponding color block, is calculated through the T of the known spectrum imaging device and the obtained output response Y.
Here, only in the case of using a spectral imaging apparatus, the spectral reconstruction accuracy of the conventional RGB camera as described above is at least 7-8 times higher. The method for obtaining the spectrum to be measured by solving the matrix equation is generally carried out by adopting a least square method or a neural network modeling method, wherein the neural network modeling method can realize spectrum calculation on the NPU, ensures high spectrum precision and has the characteristic of low power consumption, so that the method can be applied to various terminal equipment.
For example, fig. 11 illustrates a schematic diagram of a display of a spectral imaging device application according to an embodiment of the application.
In addition, the spectral imaging device according to the embodiment of the application can be applied to the field of security and protection as shown in fig. 12, the spectral imaging device according to the embodiment of the application can be applied to the field of military, as shown in fig. 13, the spectral imaging device according to the embodiment of the application can be applied to the field of medical health as shown in fig. 14, and the spectral imaging device according to the embodiment of the application can be applied to the field of automobile electronics as shown in fig. 15.
The basic principles of the present application have been described above in connection with specific embodiments, but it should be noted that the advantages, benefits, effects, etc. mentioned in the present application are merely examples and not intended to be limiting, and these advantages, benefits, effects, etc. are not to be construed as necessarily possessed by the various embodiments of the application. Furthermore, the specific details disclosed herein are for purposes of illustration and understanding only, and are not intended to be limiting, as the application is not necessarily limited to practice with the above described specific details.
The block diagrams of the devices, apparatuses, devices, systems referred to in the present application are only illustrative examples and are not intended to require or imply that the connections, arrangements, configurations must be made in the manner shown in the block diagrams. As will be appreciated by one of skill in the art, the devices, apparatuses, devices, systems may be connected, arranged, configured in any manner. Words such as "including," "comprising," "having," and the like are words of openness and mean "including but not limited to," and are used interchangeably therewith. The terms "or" and "as used herein refer to and are used interchangeably with the term" and/or "unless the context clearly indicates otherwise. The term "such as" as used herein refers to, and is used interchangeably with, the phrase "such as, but not limited to.
It is also noted that in the apparatus, devices and methods of the present application, the components or steps may be disassembled and/or assembled. Such decomposition and/or recombination should be considered as equivalent aspects of the present application.
The previous description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other aspects without departing from the scope of the application. Thus, the present application is not intended to be limited to the aspects shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
The foregoing description has been presented for purposes of illustration and description. Furthermore, this description is not intended to limit embodiments of the application to the form disclosed herein. Although a number of example aspects and embodiments have been discussed above, a person of ordinary skill in the art will recognize certain variations, modifications, alterations, additions, and subcombinations thereof.
Claims (16)
1. A method of acquiring parameters of ambient light and a projection surface, comprising:
Determining a base light spectrum of a set image, the set image comprising a plurality of color patches of any color of different brightness;
projecting the set image onto a target area of a projection surface and acquiring a reflection spectrum of the set image reflected from the target area of the projection surface, and
And acquiring an ambient light spectrum and a reflection spectrum of a projection surface based on the reflection spectrums corresponding to the plurality of color blocks of the set image.
2. The method according to claim 1, wherein the set image is a single image including a plurality of color patches of arbitrary colors of different brightnesses, or a plurality of images each including a plurality of color patches of arbitrary colors of different brightnesses, each of the plurality of images including one or more color patches, and the one or more color patches being the same or different brightnesses and the same or different colors.
3. The method of claim 1, wherein determining the base color of the at least one set image comprises:
Projecting a substrate light of a known spectrum onto a target area of a projection surface, and
And obtaining a reflection spectrum of the substrate light in a target area of the projection surface to serve as the substrate light spectrum.
4. A method according to claim 3, wherein the RGB colors of the base light of known spectrum are (0, 0) colors.
5. A method according to claim 3, wherein the RGB colors of the base light of known spectrum are adjustable.
6. A method according to claim 3, wherein the known spectrum of the base light is a spectral information parameter obtained by color space mapping.
7. The method of claim 1, wherein the reflectance spectrum of the set image comprises a reflectance spectrum of the base light, a reflectance spectrum of a plurality of color patches in the set image, and a reflectance spectrum of ambient light.
8. The method of claim 1, wherein obtaining the ambient light spectrum and the reflection spectrum of the projection surface based on the reflection spectra corresponding to the plurality of color patches of the set image comprises:
And defining a reflection spectrum of each region corresponding to each color lump in the plurality of color lumps based on the reflection spectrum of the set image.
9. The method of claim 7, wherein the reflectance spectrum of each color patch is acquired by a spectral imaging sensor or is a preset spectrum.
10. The method of claim 7, wherein obtaining the ambient light spectrum and the reflection spectrum of the projection surface based on the reflection spectra corresponding to the plurality of color patches of the set image comprises:
The reflection spectrum relation of the area corresponding to each color block is determined as follows:
Reflectance spectrum n= (base light spectrum + ambient light spectrum) reflectivity;
reflectance spectrum 1= (region 1 spectrum+ambient light spectrum) ×reflectance;
Reflectance spectrum 2= (region 2 spectrum + ambient light spectrum) reflectivity;
......
reflectance spectrum n-1= (region n-1 spectrum + ambient light spectrum).
11. The method of claim 10, wherein obtaining the ambient light spectrum and the reflection spectrum of the projection surface based on the reflection spectra corresponding to the plurality of color patches of the set image further comprises:
Define the Loss function Loss as:
Loss= |reflectance spectrum 1- (region 2 spectrum + ambient light spectrum) |reflectance|+|reflectance spectrum 2- (region 2 spectrum + ambient light spectrum) |++ reflectance spectrum n-1- (region n-1 spectrum + ambient light spectrum) |+|reflectance spectrum n- (base light spectrum + ambient light spectrum) |reflectance|| where|·||indicates the norm of the vector or matrix;
The best estimate of the ambient light spectrum and the reflectivity of the target area of the projection surface is calculated by minimizing the Loss function Loss by means of a least squares method.
12. The method of claim 11, wherein the ambient light spectrum and the reflectivity of the target area of the projection surface are solved from the relationship by neural network modeling.
13. A spectral imaging device for use in a method of acquiring parameters of ambient light and a projection surface according to any one of claims 1 to 12 for acquiring a base light spectrum of a set image and/or a reflection spectrum of the set image.
14. The spectral imaging apparatus of claim 13, wherein acquiring a base light spectrum of a set image and/or a reflectance spectrum of the set image comprises:
determining a relation:
Ck＝∫S(λ)ρ(λ)τ(λ)fk(λ)α(λ)dλ;
Wherein C k represents the pixel value of the kth channel of the spectral imaging device, S (lambda) represents the ambient light spectrum, rho (lambda) represents the reflectivity of a target area of a projection surface, tau (lambda) represents the transmissivity of a lens of the spectral imaging device, f k (lambda) represents the transmissivity of the kth structure of a spectral imaging sensor of the spectral imaging device, and alpha (lambda) represents the spectral sensitivity of the spectral imaging sensor of the spectral imaging device itself;
The above parameters satisfy the relation:
S(λ)ρ(λ)=X;
τ(λ)fk(λ)α(λ)=T;
Ck=Y;
Where X k is the spectral image value of the input spectrum at the kth structure and Y is the response of the output at the corresponding structure, resulting in:
Y=TX。
15. The spectral imaging device of claim 14, wherein a known plurality of color patches of the set image are calibrated in advance to calculate an input light spectrum X from T of the known spectral imaging device and a response Y of an obtained output, the input light being the base light and/or the reflected light.
16. A terminal device comprising a spectral imaging device according to any of claims 13 to 15.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310762415.1A CN119211494A (en) | 2023-06-26 | 2023-06-26 | Method for acquiring parameters of ambient light and projection surface and spectral imaging device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310762415.1A CN119211494A (en) | 2023-06-26 | 2023-06-26 | Method for acquiring parameters of ambient light and projection surface and spectral imaging device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN119211494A true CN119211494A (en) | 2024-12-27 |
Family
ID=94042866
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310762415.1A Pending CN119211494A (en) | 2023-06-26 | 2023-06-26 | Method for acquiring parameters of ambient light and projection surface and spectral imaging device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN119211494A (en) |
-
2023
- 2023-06-26 CN CN202310762415.1A patent/CN119211494A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11193830B2 (en) | Spectrocolorimeter imaging system | |
| JP6352185B2 (en) | Fast display calibration using a multicolor camera calibrated colorimetrically based on spectrum | |
| TWI626433B (en) | Method for two-dimensional, spatially resolved measurement and imaging colorimeter system capable of said measurement | |
| US7616314B2 (en) | Methods and apparatuses for determining a color calibration for different spectral light inputs in an imaging apparatus measurement | |
| US9188841B2 (en) | Imaging device | |
| US11006088B1 (en) | Colour calibration of an imaging device | |
| US9595239B2 (en) | Color display calibration system | |
| CN109979382A (en) | Imaging system color correcting method and system under screen based on screen transmitted spectrum | |
| CN104933706B (en) | A kind of imaging system color information scaling method | |
| KR101705818B1 (en) | Apparatus, system and method for measuring luminance and chromaticity | |
| CN113848044B (en) | Method for detecting brightness and chrominance consistency of display screen | |
| US20050219380A1 (en) | Digital camera for determining chromaticity coordinates and related color temperature of an object and method thereof | |
| JP6225519B2 (en) | Measuring apparatus and measuring method | |
| CN119211494A (en) | Method for acquiring parameters of ambient light and projection surface and spectral imaging device | |
| CN119207253A (en) | Projection display optimization method, spectrum imaging device and terminal device | |
| JP6601632B2 (en) | Color measurement method | |
| CN113132694B (en) | Colourity-based color consistency correction method for multi-camera lens of colour camera | |
| CN119788821A (en) | Method and system for optimizing projection display | |
| Zhu et al. | Matched illumination | |
| TWI385363B (en) | Method of constructing light-measuring look-up table, light-measuring method, and light-measuring system | |
| Shrestha | Multispectral imaging: Fast acquisition, capability extension, and quality evaluation | |
| US20230082539A1 (en) | Illuminant correction in an imaging system | |
| CN115086581B (en) | Image sensing method and device based on spectrum chip | |
| CN116773524A (en) | Micro light emitting diode array and display panel detection method and device | |
| CN119252170A (en) | Method and system for determining the color coordinates of a light emitter of a matrix display |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |