CN115511972A - Calculation method of correction matrix, non-uniformity correction method and related device - Google Patents

Abstract

The application is applicable to the technical field of multispectral, and provides a calculation method of a correction matrix, a non-uniformity error correction method and a related device, wherein the calculation method comprises the following steps: acquiring a calibration multispectral image of a white board, and acquiring first data of each channel in the calibration multispectral image; acquiring a reference position of a multispectral image sensor; determining a reference region in the calibration multispectral image according to the reference position, and acquiring second data corresponding to each channel in the reference region; and calculating a correction matrix corresponding to each channel according to the first data and the second data. According to the scheme, the target correction matrix of each channel is calculated by calibrating the data of each channel in the multispectral image and the data corresponding to each channel in the reference area, so that the non-uniformity error of the multispectral camera can be corrected according to the target correction matrix, and the multispectral image obtained by the multispectral camera is more real.

Description

Calculation method of correction matrix, non-uniformity correction method and related device

technical field

The present application belongs to the field of multispectral technology, and in particular relates to a method for calculating a correction matrix, a method for correcting a non-uniformity error of a multispectral image, a device for calculating a correction matrix, a device for correcting a nonuniformity error for a multispectral image, a terminal and a computer Read storage media.

Background technique

The multispectral image sensor has more channels than the RGB sensor, but due to the more complicated manufacturing process, due to the manufacturing process (such as uneven coating of optical filters), the same channel at different spatial positions of the multispectral image sensor Channels, whose response curves to the spectrum are inconsistent, manifest as the non-uniformity of the response of the multi-spectral image sensor in the spatial domain, which in turn leads to non-uniformity errors in the generated multi-spectral image.

Contents of the invention

The embodiment of the present application provides a method for calculating a correction matrix, a method for correcting a non-uniformity error of a multispectral image, a device for calculating a correction matrix, a device for correcting a nonuniformity error for a multispectral image, a terminal, and a computer-readable storage medium, The above problems can be solved.

In the first aspect, the embodiment of the present application provides a calculation method of the correction matrix, which is used to correct the non-uniformity error of the multispectral image sensor, including: acquiring the calibrated multispectral image of the whiteboard, and obtaining each of the calibrated multispectral images The first data of the channel, the calibration multi-spectral image is collected by the multi-spectral image sensor; the reference position in the multi-spectral image sensor is obtained; wherein, the actual multi-channel response curve and the ideal multi-channel response curve of the multi-spectral image sensor at the reference position The coincidence degree between them is greater than the preset threshold value; determine the reference area in the calibration multispectral image according to the reference position, and obtain the second data of each channel in the reference area; according to the first data and the second data, calculate each Correction matrix corresponding to the channel.

In some embodiments, obtaining the first data of each channel in the calibrated multispectral image includes: extracting the channel subimage corresponding to each channel in the calibrated multispectral image, and extracting the data of each pixel in each channel subimage , to obtain the first data of each channel; wherein, the first data is expressed in matrix form. In some embodiments, the reference area includes one or more reference pixel points. When the number of reference pixel points is multiple, obtaining the second data of each channel in the reference area includes: respectively extracting the plurality of reference pixel points Data for each channel; multiple data for each channel are averaged to obtain second data for each channel in the reference region.

In some embodiments, the number of calibrated multispectral images is one, and calculating the correction matrix corresponding to each channel according to the first data and the second data includes: dividing the first data and the second data of each channel , to get the correction matrix corresponding to each channel. In some embodiments, the number of calibrated multispectral images is multiple, and the multiple calibrated multispectral images are multispectral images of the whiteboard collected when the multispectral image sensor is at multiple distances from the whiteboard. According to the first data and the second data, Calculating the target correction matrix corresponding to each channel includes: obtaining the first data of each channel in each calibrated multispectral image to obtain multiple first data; summing or averaging multiple first data to obtain The third data; obtain the second data of each calibrated multispectral image to obtain a plurality of second data; sum or average the plurality of second data to obtain the fourth data; compare the third data and the fourth data The correction matrix corresponding to each channel in the multispectral image sensor is obtained.

In the second aspect, the embodiment of the present application provides a method for correcting non-uniformity errors of multispectral images, including: acquiring an initial multispectral image; performing nonuniformity correction on the initial multispectral image according to a preset correction matrix to obtain the target A multispectral image; wherein, the correction matrix is calculated by the calculation method of the correction matrix in the first aspect above.

In some embodiments, the correction matrix includes a sub-correction matrix corresponding to each channel, and the non-uniformity correction is performed on the initial multispectral image according to the preset correction matrix to obtain the target multispectral image, including: according to the sub-correction matrix corresponding to each channel The correction matrix corrects the data of each channel in the initial multispectral image respectively to obtain the target multispectral image.

In a third aspect, an embodiment of the present application provides a correction matrix calculation device, including: a first acquisition unit, a second acquisition unit, a processing unit, and a calculation unit. The first acquisition unit is used to acquire the calibrated multispectral image of the whiteboard, and acquires the first data of each channel in the calibrated multispectral image, and the calibrated multispectral image is collected by the multispectral image sensor; the second acquisition unit is used to acquire the multispectral image A reference position in the image sensor; wherein, the coincidence degree between the actual multi-channel response curve and the ideal multi-channel response curve of the multi-spectral image sensor at the reference position is greater than a preset threshold; the processing unit is used to determine the calibration multi-spectrum according to the reference position The reference area in the image, and acquire the second data of each channel in the reference area; the calculation unit is used to calculate the correction matrix corresponding to each channel according to the first data and the second data.

In a fourth aspect, an embodiment of the present application provides a non-uniformity error correction device for a multispectral image, including an acquisition unit and a non-uniformity correction unit. The acquisition unit is used to acquire the initial multispectral image; the non-uniformity correction unit is used to perform non-uniformity correction on the initial multispectral image according to the preset correction matrix to obtain the target multispectral image; wherein, the correction matrix is passed through the first aspect The calculation method of the correction matrix is calculated.

In a fifth aspect, an embodiment of the present application provides a terminal, including a multispectral image sensor and a processor. A multispectral image sensor is used to acquire multispectral images. In one embodiment, the processor is configured to execute the calculation method of the correction matrix in the first aspect after receiving the multispectral image. In another embodiment, the processor is configured to execute the non-uniformity error correction method of the multispectral image in the second aspect after receiving the multispectral image.

In a sixth aspect, the embodiment of the present application provides a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium. In one embodiment, when the computer program is executed by the processor, the calculation method of the correction matrix in the first aspect above is realized. In another embodiment, when the computer program is executed by the processor, the method for correcting the non-uniformity error of the multispectral image as in the second aspect above is realized.

In the embodiment of the present application, by obtaining the calibrated multispectral image of the whiteboard and extracting the data of each channel in the calibrated multispectral image and the data corresponding to each channel in the reference area, the correction matrix of each channel is calculated, so as to obtain the The correction matrix, and then the non-uniformity error correction can be performed according to the multi-spectral image collected by the correction matrix, which reduces the error caused by the non-uniformity of the multi-spectral image sensor, and the obtained target multi-spectral image is more real.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the accompanying drawings that need to be used in the descriptions of the embodiments or the prior art will be briefly introduced below. Obviously, the accompanying drawings in the following description are only for the present application For some embodiments, those of ordinary skill in the art can also obtain other drawings based on these drawings without paying creative efforts.

FIG. 1 is a schematic structural diagram of a terminal provided in a first embodiment of the present application;

Fig. 2 is a schematic diagram of 9 channel sub-images extracted from 9-channel multispectral images;

FIG. 3 is a schematic flowchart of a non-uniformity error correction method for a multispectral image sensor provided in the second embodiment of the present application;

FIG. 4 is a schematic flowchart of a calculation method of a correction matrix provided in the third embodiment of the present application;

FIG. 5 is a schematic diagram of a non-uniformity error correction device for a multispectral image provided in a fourth embodiment of the present application;

Fig. 6 is a schematic diagram of a calculation device for a correction matrix provided by a fifth embodiment of the present application.

detailed description

In the following description, specific details such as specific system structures and technologies are presented for the purpose of illustration rather than limitation, so as to thoroughly understand the embodiments of the present application. It will be apparent, however, to one skilled in the art that the present application may be practiced in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.

It should be understood that when used in this specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and/or components, but does not exclude one or more other Presence or addition of features, wholes, steps, operations, elements, components and/or collections thereof.

It should also be understood that the term "and/or" used in the description of the present application and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

As used in this specification and the appended claims, the term "if" may be construed, depending on the context, as "when" or "once" or "in response to determining" or "in response to detecting ". Similarly, the phrase "if determined" or "if [the described condition or event] is detected" may be construed, depending on the context, to mean "once determined" or "in response to the determination" or "once detected [the described condition or event] ]” or “in response to detection of [described condition or event]”.

In addition, in the description of the specification and appended claims of the present application, the terms "first", "second", "third" and so on are only used to distinguish descriptions, and should not be understood as indicating or implying relative importance.

Reference to "one embodiment" or "some embodiments" or the like in the specification of the present application means that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the present application. Thus, appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in other embodiments," etc. in various places in this specification are not necessarily All refer to the same embodiment, but mean "one or more but not all embodiments" unless specifically stated otherwise. The terms "including", "comprising", "having" and variations thereof mean "including but not limited to", unless specifically stated otherwise.

FIG. 1 is a schematic diagram of a terminal provided in a first embodiment of the present application. The terminal in this embodiment includes a processor 11 , a memory 12 , a multispectral camera 13 and a computer program 14 stored in the memory 11 and operable on the processor 11 . Processor 11 is connected with memory 12 and multispectral camera 13 respectively, and processor 11 can store data to memory 12, such as image; Processor 11 can also call the data in memory 12, such as computer program 14, image etc., processor 11 The multispectral camera 13 can also be controlled to capture images.

The terminal 1 may include but not limited to a processor 11 , a memory 12 and a multispectral camera 13 . Those skilled in the art can understand that FIG. 1 is only an example of the terminal 1 and does not constitute a limitation to the terminal 1. It may include more or less components than those shown in the figure, or combine certain components, or different components, such as The terminal 1 may also include an input/output interface, a network access interface, a bus, etc.; for another example, the terminal 1 may not include the memory 12, the computer program 14 is burned in the processor 11, or the memory 12 is a cloud storage.

The processor 11 can be a central processing unit (Central Processing Unit, CPU), and can also be other general-purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), off-the-shelf Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor, or the processor may be any conventional processor, or the like.

The storage 12 may be an internal storage unit of the terminal 1, such as a hard disk or a memory of the terminal 1. The memory 12 can also be an external storage device of the terminal 1, such as a plug-in hard disk equipped on the terminal 1, a smart memory card (Smart MediaCard, SMC), a secure digital (Secure Digital, SD) card, a flash memory card (Flash Card), etc. . Further, the terminal 1 may also include both an internal storage unit of the terminal 1 and an external storage device. The memory 12 is used to store the computer program 14 and the data needed to execute the program, such as correction matrix, multi-spectral image and so on. The memory 11 can also be used to temporarily store data that has been output or will be output.

The multispectral camera 13 is used to collect multispectral images of the detected object, and the collected multispectral images can be stored in the memory 12 or directly transmitted to the processor 11 . The multispectral camera 13 can receive at least 4 different wavelengths of light and then generate a multispectral image, for example, the multispectral camera 13 can receive 4, 5, 6, 7, 8, 9, 16 or more different wavelengths of light and generate an image.

Fig. 2 shows a schematic diagram of sub-images corresponding to 9 channels extracted from a multi-spectral image. In the embodiment corresponding to FIG. 2 , the multispectral camera 13 can receive light in nine different wavelength bands and generate a multispectral image, which can be called a 9-channel multispectral image. The image obtained by extracting the one-channel data corresponding to each pixel in the 9-channel multispectral image is a one-channel sub-image, and similarly, two-channel sub-images to nine-channel sub-images can be obtained, ch1, ch2, and ch3 in Figure 2 , ch4, ch5, ch6, ch7, ch8 and ch9 are the extracted one-channel sub-image, two-channel sub-image, three-channel sub-image, four-channel sub-image, five-channel sub-image, six-channel sub-image, seven-channel sub-image image, eight-channel sub-image, and nine-channel sub-image.

Specifically, the multispectral camera 13 includes a lens and a multispectral image sensor, the lens is used to converge light to the multispectral image sensor, and the multispectral image sensor generates a corresponding multispectral image after receiving the light. The multispectral image sensor includes a filter array and a pixel array. The filter array is used to filter light. The pixel array is used to receive the filtered light and convert it into an electrical signal. The filter array includes a variety of different central wavelengths. Optical filters, which in turn enable multi-spectral image sensors to achieve multi-channel imaging. Due to reasons such as manufacturing processes (for example, uneven coating of optical filters), the same channel at different spatial positions of the multi-spectral image sensor has different effects on the light of the same wavelength band. The response curves are inconsistent, and this inconsistency can be understood to be caused by the non-uniformity of the multispectral image sensor in the spatial domain. This non-uniformity may lead to inconsistent color or spectral curves of the same object in different imaging spaces. Taking biopsy as an example, the effect of biopsy at the central location is good, while the effect of biopsy at other locations is slightly worse. Therefore, it is necessary to correct the non-uniformity error of the multispectral image sensor.

The computer program 14 is stored in the memory 12, and the computer program 14 includes programs such as a non-uniformity error correction program of a multispectral image and a calculation program of a correction matrix. The non-uniformity error correction program can be invoked by the processor 11 to execute a non-uniformity error correction instruction on the multispectral image, for example, implement the nonuniformity error correction method for the multispectral image shown in FIG. 3 . The calculation program of the correction matrix can be invoked by the processor 11 to execute the instruction of calculating the correction matrix for correcting the non-uniformity error of the multispectral image sensor, for example, realizing the calculation method of the correction matrix shown in FIG. 4 . In some embodiments, the non-uniformity error correction program and the calculation program of the correction matrix can also be respectively located in the two memories 12 .

Please refer to FIG. 3 . FIG. 3 is a schematic flowchart of a non-uniformity error correction method for a multispectral image provided in the second embodiment of the present application. Non-uniformity error correction methods for multispectral images can include:

S201: Acquire an initial multispectral image.

The initial multispectral image may be captured by the multispectral camera in real time, or may be an image stored in the memory after being captured. Wherein, the initial multispectral image may include data of at least four different bands, that is, the initial multispectral image is at least a four-channel multispectral image.

S202: Perform non-uniformity correction on the initial multispectral image according to a preset correction matrix to obtain a target multispectral image; wherein, the correction matrix can be calculated by the calculation method of the correction matrix as shown in FIG. 4 and stored in a memory.

The processor can recall a preset correction matrix from the memory, and the correction matrix includes a sub-correction matrix corresponding to each channel, and then perform non-uniformity on the data of each channel in each pixel in the initial multispectral image according to the sub-correction matrix Correction, the image generated after correction is the target multispectral image.

Take the initial multispectral image as an example of a 9-channel multispectral image for illustration. The multispectral data of the entire initial multispectral image is MSI_target(m,n,9), where MSI_target(m,n,9) is a three-dimensional matrix, m is the number of rows of pixels in the initial multispectral image, and n is the initial multispectral image. The number of columns of pixels in the spectral image, 9 is the data of 9 channels for each pixel. The correction matrix is Correct(m,n,9), Correct(m,n,9) is a three-dimensional matrix, m is the number of rows of pixels, n is the number of columns of pixels, and 9 is 9 channels for each pixel correction parameters. In one embodiment, the method for performing non-uniformity correction on the initial multispectral image is as follows:

MSI_target_correct(m,n,9)=MSI_target(m,n,9)./Correct(m,n,9)

Among them, ./ represents the corresponding division of data at corresponding points in the two matrices, and MSI_target_correct(m,n,9) is the multispectral data of the target multispectral image, which is also expressed in the form of a three-dimensional matrix.

In one embodiment, the correction matrix Correct(m,n,9) can be decomposed into Correct1(m,n,1), Correct2(m,n,2), Correct3(m,n,3), Correct4(m, n,4), Correct5(m,n,5), Correct6(m,n,6), Correct7(m,n,7), Correct8(m,n,8), Correct9(m,n,9) The sub-correction matrices corresponding to the 9 channels are all two-dimensional matrices, and then divide the 9 sub-correction matrices by the data of the sub-images of the 9 channels to obtain the corrected data of the 9 sub-channels, and then obtain the target multispectral image.

In the embodiment of the present application, non-uniformity correction is performed on the collected multi-spectral image by calling the correction matrix, which reduces the error caused by the non-uniformity and makes the generated target multi-spectral image more accurate.

In some embodiments, the above-mentioned correction matrix can be calculated by a calculation method of a correction matrix. As shown in FIG. 4, FIG. 4 is a schematic flowchart of a calculation method of a correction matrix provided in the third embodiment of the present application. The calculation method of the correction matrix may include the following steps:

S401: Acquire a calibrated multispectral image of the whiteboard, and acquire first data of each channel in the calibrated multispectral image.

The whiteboard has a good response to the light of each band. Obtaining the calibrated multispectral image of the whiteboard and then calculating the correction matrix based on the multispectral data of the whiteboard will result in a more accurate correction matrix. When the correction matrix needs to be calculated, the processor can control the multispectral camera to collect the calibrated multispectral image of the whiteboard. When collecting the calibrated multispectral image, it is necessary to ensure that the whiteboard completely covers the field of view of the multispectral camera. The entire picture of the image is a whiteboard, which avoids the interference of other items and is conducive to improving the accuracy of the subsequent correction matrix. In one embodiment, the whiteboard is a diffuse reflection whiteboard, and the reflectance of the diffuse reflection whiteboard for ultraviolet light, visible light and near-infrared light can reach more than 98%. The difference, and then the calibration matrix obtained by calibration will be more accurate.

After the calibrated multispectral image is acquired, the first data of each channel in the calibrated multispectral image is further acquired. Specifically, the channel sub-image corresponding to each channel in the calibration multispectral image can be extracted separately, and then the data of each pixel point in each channel sub-image can be extracted, then the first data of each channel can be obtained, and the first data is expressed in matrix form . The first data of each channel includes the data of each pixel in the calibration multispectral image in the channel, expressed in the form of a matrix with m rows and n columns, where m is the number of rows of pixels in the calibration multispectral image, n is the column number of pixels in the calibration multispectral image.

Continue to calibrate the multispectral image to a 9-channel image as an example for illustration. The extracted first data corresponding to the first channel to the ninth channel respectively are: ch ₁ (m,n)=MSI_1(m,n,1); ch ₂ (m,n)=MSI_1(m,n,2); ch ₃ (m,n)=MSI_1(m,n,3); ch ₄ (m,n)=MSI_1(m,n,4); ch ₅ (m,n)=MSI_1(m,n,5) ;ch ₆ (m,n)=MSI_1(m,n,6); ch ₇ (m,n)=MSI_1(m,n,7); ch ₈ (m,n)=MSI_1(m,n,8 ); ch ₉ (m,n)=MSI_1(m,n,9). Wherein, MSI_1(m,n,1) is the first data of a channel in the calibration multispectral image.

S402: Acquire a reference position in the multi-spectral image sensor; wherein, the coincidence degree between the actual multi-channel response curve and the ideal multi-channel response curve of the multi-spectral image sensor at the reference position is greater than a preset threshold.

The reference position is the position where the coincidence degree between the actual multi-channel response curve and the ideal multi-channel response curve in the multispectral image sensor is greater than a preset threshold, and the preset threshold can be 80%, 90%, 95%, etc., here Not to list them all. The ideal multi-channel response curve can be understood as the response curve of each channel to a wide-band spectrum when there is no non-uniformity error (for example, when the filter is uniformly coated), and the actual multi-channel response curve is each channel to a wide band in actual use Spectral response curves for the bands. It can be understood that the difference between the actual response curve of each channel corresponding to the reference position and the ideal response curve of each channel is within an allowable range, and the data obtained at the reference position is more accurate. Specifically, the reference position may be pre-stored in the memory, and the reference positions of different multispectral image sensors may be the same or different. The reference position is mainly determined by the manufacturing process of the multispectral image sensor. The reference position can be determined after the multispectral image sensor is produced. The non-uniformity error at the reference position is the smallest. For example, the filter in the central area of the multispectral image sensor The light sheet coating is relatively uniform, and the reference position can be the central coordinate point.

In one embodiment, the reference position is a coordinate point, which can be obtained through calibration. For example, a sample image is collected by a multi-spectral camera, and the channel response curve of each pixel on the sample image is obtained, and then the ideal pixel point of the most ideal channel response curve is found, and then the reference position of the multi-spectral image sensor can be determined. When there are multiple ideal pixel points, you can compare the 9-channel response curves of the pixels in a certain area around the ideal pixel point, and then find the most ideal pixel point as a reference position; or, you can also select one of the multiple points arbitrarily as a reference position; or a plurality of ideal pixel points can also be used as a reference position.

Since the method of using a single coordinate point may affect the accuracy of the correction matrix due to noise interference in the data of a single coordinate point, in another embodiment, the reference position can be one or more areas, and the multispectral image sensor in this area The actual multi-channel response curve within is closer to the ideal multi-channel response curve, and the specific location of the area can also be determined by calibration and other methods.

S403: Determine a reference area in the calibration multispectral image according to the reference position, and acquire second data of each channel in the reference area.

Since the calibrated multispectral image is generated by the multispectral camera, the reference area in the calibrated multispectral image can be determined according to the information of the reference position. It can be understood that the image data corresponding to the reference area is generated by the pixels at the reference position. When the reference area is a pixel point, the data of each channel in the pixel point may be extracted as the second area of each channel in the reference area. When the reference area includes multiple pixels, it is necessary to extract the data of each channel in each pixel, and each channel can obtain multiple data, and then average the multiple data of each channel to obtain the data of each channel in the reference area. As for the second data of the channel, it can be understood that the second data is not a matrix but a value. Wherein, the manner of acquiring the second data may refer to the manner of acquiring the first data in step S401, which will not be repeated here.

S404: Calculate a correction matrix corresponding to each channel according to the first data and the second data.

In one embodiment, the number of calibrated multispectral images is one. When the multispectral camera is at a preset distance from the whiteboard, the collected multispectral images of the whiteboard are compared with the first data and the second data of each channel. to get the correction matrix corresponding to each channel. For example, Correct(m,n,k)=ch _k (m,n)/ch _k (x,y), where Correct(m,n,k) is the correction matrix of channel k, and ch _k (m,n ) is the first data of channel k, expressed in matrix form, and ch _k (x, y) is the second data of channel k, expressed in numerical value. Wherein, the correction matrix corresponding to each channel is the sub-correction matrix of each channel described in the above embodiment of the non-uniformity correction method.

In another embodiment, in order to calculate the correction matrix more accurately, the multispectral camera collects a calibration multispectral image at multiple distances from the whiteboard, and multiple calibration multispectral images can be obtained. The first data of each channel in the multispectral image, and the second data of each channel in the reference area of each calibrated multispectral image. It can be understood that the number of first data of each channel is multiple, and the number of second data of each channel is multiple. When calculating the correction matrix, the multiple first data are summed or averaged to obtain the corresponding third data; summing or averaging a plurality of second data to obtain fourth data; and then dividing the third data and the fourth data to obtain a correction matrix corresponding to each channel in the multispectral image sensor.

Continue to take the 9-channel calibrated multispectral image as an example. A total of p calibrated multispectral images of whiteboards corresponding to p distances have been collected, and each channel has p second data. The third data can be calculated as follows:

ch ₁ (m,n)=MSI_1(m,n,1)+MSI_2(m,n,1)+...+MSI_p(m,n,1);

ch ₂ (m,n)=MSI_1(m,n,2)+MSI_2(m,n,2)+...+MSI_p(m,n,2);

ch ₃ (m,n)=MSI_1(m,n,3)+MSI_2(m,n,3)+...+MSI_p(m,n,3);

ch ₄ (m,n)=MSI_1(m,n,4)+MSI_2(m,n,4)+...+MSI_p(m,n,4);

ch ₅ (m,n)=MSI_1(m,n,5)+MSI_2(m,n,5)+...+MSI_p(m,n,5);

ch ₆ (m,n)=MSI_1(m,n,6)+MSI_2(m,n,6)+...+MSI_p(m,n,6);

ch ₇ (m,n)=MSI_1(m,n,7)+MSI_2(m,n,7)+...+MSI_p(m,n,7);

ch ₈ (m,n)=MSI_1(m,n,8)+MSI_2(m,n,8)+...+MSI_p(m,n,8);

ch ₉ (m,n)=MSI_1(m,n,9)+MSI_2(m,n,9)+...+MSI_p(m,n,9).

Each channel in the reference area corresponds to p second data, and the fourth data corresponding to channel 1 to channel 9 can be calculated as follows:

ch ₁ (x,y)=MSI_1(x,y,1)+MSI_2(x,y,1)+...+MSI_p(x,y,1);

ch ₂ (x,y)=MSI_1(x,y,2)+MSI_2(x,y,2)+...+MSI_p(x,y,2);

ch ₃ (x,y)=MSI_1(x,y,3)+MSI_2(x,y,3)+...+MSI_p(x,y,3);

ch ₄ (x,y)=MSI_1(x,y,4)+MSI_2(x,y,4)+...+MSI_p(x,y,4);

ch ₅ (x,y)=MSI_1(x,y,5)+MSI_2(x,y,5)+...+MSI_p(x,y,5);

ch ₆ (x,y)=MSI_1(x,y,6)+MSI_2(x,y,6)+...+MSI_p(x,y,6);

ch ₇ (x,y)=MSI_1(x,y,7)+MSI_2(x,y,7)+...+MSI_p(x,y,7);

ch ₈ (x,y)=MSI_1(x,y,8)+MSI_2(x,y,8)+...+MSI_p(x,y,8);

ch ₉ (x, y)=MSI_1(x,y,9)+MSI_2(x,y,9)+...+MSI_p(x,y,9).

Wherein, x and y refer to the position coordinates of the reference pixel when the reference area is the reference pixel.

Dividing the third data and the fourth data of each channel, the third data of each channel includes m*n data, and the fourth data has only one data, that is, dividing the m*n data by the fourth data, Then the correction matrix corresponding to each channel is obtained. For example, the correction matrix of 9 channels is specifically calculated as follows:

Correct(m,n,1)=ch ₁ (m,n)/ch ₁ (x,y);

Correct(m,n,2)=ch ₂ (m,n)/ch ₂ (x,y);

Correct(m,n,3)=ch ₃ (m,n)/ch ₃ (x,y);

Correct(m,n,4)=ch ₄ (m,n)/ch ₄ (x,y);

Correct(m,n,5)=ch ₅ (m,n)/ch ₅ (x,y);

Correct(m,n,6)=ch ₆ (m,n)/ch ₆ (x,y);

Correct(m,n,7)=ch ₇ (m,n)/ch ₇ (x,y);

Correct(m,n,8)=ch ₈ (m,n)/ch ₈ (x,y);

Correct(m,n,9)=ch ₉ (m,n)/ch ₉ (x,y).

In addition, in the above example, the calculation of the third data and the fourth data by means of summation is only one implementation mode, and in other implementation modes, the third data and the fourth data can also be calculated by means of averaging; the above In an example, when calculating the correction matrix, the first data is divided by the second data. In other examples, the second data may also be divided by the first data, which is not limited here.

Wherein, the acquired correction matrix of each channel may be stored in a memory so as to be invoked by the processor when executing the above non-uniformity error correction method. In one embodiment, after obtaining the correction matrix of each channel, the correction matrices of multiple channels can be combined to obtain a multi-channel correction matrix. The multi-channel correction matrix is a three-dimensional matrix, which can be understood as the above-mentioned non-uniformity The sub-correction matrix for each channel described in the correction method example. In addition, the correction matrix of each channel obtained in this embodiment can be applied to perform non-uniformity correction on multispectral images obtained from any shooting distance.

In the embodiment of the present application, by obtaining the calibrated multispectral image of the whiteboard and extracting the data of each channel in the calibrated multispectral image and the data corresponding to each channel in the reference area, the target correction matrix for each channel is calculated, thereby obtaining each The correction matrix of the channel, and then the non-uniformity error correction can be performed according to the multi-spectral image collected by the correction matrix, which reduces the error caused by the non-uniformity of the multi-spectral image sensor, and the obtained target multi-spectral image is more real.

It should be understood that the sequence numbers of the steps in the above embodiments do not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiment of the present application.

Please refer to FIG. 5 . FIG. 5 is a schematic diagram of a non-uniformity error correction device for a multispectral image provided by a fourth embodiment of the present application. Each included unit is used to execute each step in the embodiment corresponding to FIG. 3 . For details, refer to the relevant description in the embodiment corresponding to FIG. 3 . For ease of description, only the parts related to this embodiment are shown. Referring to FIG. 5 , the device 5 for correcting non-uniformity errors of multispectral images includes an acquisition unit 510 and a non-uniformity correction unit 520 .

Specifically, the acquisition unit 510 is used to acquire the initial multispectral image; the non-uniformity correction unit 520 is used to correct the non-uniformity of the initial multispectral image according to the preset correction matrix to obtain the target multispectral image; wherein, the correction matrix can be It is calculated by the calculation method of the correction matrix as shown in FIG. 4 .

Please refer to FIG. 6 . FIG. 6 is a schematic diagram of an apparatus for calculating a correction matrix provided by a fifth embodiment of the present application. Each included unit is used to execute each step in the embodiment corresponding to FIG. 4 . For details, refer to the relevant description in the embodiment corresponding to FIG. 4 . For ease of description, only the parts related to this embodiment are shown. Referring to FIG. 6 , the correction matrix calculation device 6 includes a first acquisition unit 610 , a second acquisition unit 620 , a processing unit 630 and a calculation unit 640 .

The first acquisition unit 610 is used to acquire the calibrated multispectral image of the whiteboard, and acquires the first data of each channel in the calibrated multispectral image, and the calibrated multispectral image is collected by a multispectral image sensor; the second acquisition unit 620 is used to acquire A reference position in the multi-spectral image sensor; wherein, the coincidence degree between the actual multi-channel response curve of the multi-spectral image sensor at the reference position and the ideal multi-channel response curve is greater than a preset threshold; the processing unit 630 is used to determine according to the reference position Mark the reference area in the multispectral image, and acquire the second data of each channel in the reference area; the calculation unit 640 is used to calculate the correction matrix corresponding to each channel according to the first data and the second data.

It should be noted that the information interaction and execution process between the above-mentioned devices/units are based on the same concept as the method embodiment of the present application, and its specific functions and technical effects can be found in the method embodiment section. I won't repeat them here.

The embodiment of the present application also provides a network device, which includes: at least one processor, a memory, and a computer program stored in the memory and operable on the at least one processor. When the processor executes the computer program, any of the above-mentioned Steps in various method embodiments.

The embodiment of the present application also provides a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the steps in the foregoing method embodiments can be realized.

An embodiment of the present application provides a computer program product. When the computer program product is run on a mobile terminal, the mobile terminal can implement the steps in the foregoing method embodiments when executed.

If the integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments in the present application can be completed by instructing related hardware through computer programs, and the computer programs can be stored in a computer-readable storage medium. When executed by a processor, the steps in the foregoing method embodiments can be realized. Wherein, the computer program includes computer program code, and the computer program code may be in the form of source code, object code, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device capable of carrying computer program codes to a photographing device/terminal, a recording medium, a computer memory, a read-only memory (ROM, Read-Only Memory), a random-access memory (RAM, Random Access Memory), electrical carrier signals, telecommunication signals, and software distribution media. Such as U disk, mobile hard disk, magnetic disk or optical disk, etc. In some jurisdictions, computer readable media may not be electrical carrier signals and telecommunication signals under legislation and patent practice.

In the above-mentioned embodiments, the descriptions of each embodiment have their own emphases, and for parts that are not detailed or recorded in a certain embodiment, refer to the relevant descriptions of other embodiments.

Those skilled in the art can appreciate that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed by hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be regarded as exceeding the scope of the present application.

In the embodiments provided in this application, it should be understood that the disclosed device/network device and method may be implemented in other ways. For example, the device/network device embodiments described above are only illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units Or components may be combined or may be integrated into another system, or some features may be omitted, or not implemented. In another point, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: it can still implement the foregoing embodiments Modifications to the technical solutions described in the examples, or equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the various embodiments of the application, and should be included in the Within the protection scope of this application.

Claims (11)

1. A method for calculating a correction matrix for correcting non-uniformity errors of a multi-spectral image sensor, comprising:

acquiring a calibration multispectral image of a white board, and acquiring first data of each channel in the calibration multispectral image, wherein the calibration multispectral image is acquired by a multispectral image sensor;

acquiring a reference position in the multispectral image sensor; wherein the coincidence degree between the actual multichannel response curve and the ideal multichannel response curve of the multispectral image sensor at the reference position is greater than a preset threshold value;

determining a reference region in the calibration multispectral image according to the reference position, and acquiring second data of each channel in the reference region;

and calculating a correction matrix corresponding to each channel according to the first data and the second data.

2. The method according to claim 1, wherein said obtaining first data for each channel in said calibrated multispectral image comprises:

extracting a channel sub-image corresponding to each channel in the calibration multispectral image, and extracting data of each pixel point in each channel sub-image to obtain the first data of each channel; wherein the first data is represented in a matrix form.

3. The method for calculating the correction matrix according to claim 1, wherein the reference region includes one or more reference pixels, and when the number of the reference pixels is plural, the obtaining the second data of each channel in the reference region includes:

respectively extracting data of each channel in the reference pixel points to obtain a plurality of data of each channel;

and averaging the plurality of data of each channel to obtain second data of each channel in the reference region.

4. The method according to claim 1, wherein the calibrating the number of multispectral images is one, and the calculating the correction matrix corresponding to each channel according to the first data and the second data comprises:

and dividing the first data and the second data of each channel to obtain a correction matrix corresponding to each channel.

5. The method of calculating a correction matrix according to claim 1, wherein said calculating a plurality of calibration multispectral images, each of said plurality of calibration multispectral images being a multispectral image of said whiteboard captured at a plurality of distances from said multispectral image sensor to said whiteboard, comprises:

acquiring the first data of each channel in each calibrated multispectral image to obtain a plurality of first data;

performing summation or averaging on a plurality of first data to obtain third data;

acquiring the second data of each calibration multispectral image to obtain a plurality of second data;

performing summation or averaging on the plurality of second data to obtain fourth data;

and dividing the third data and the fourth data to obtain a correction matrix corresponding to each channel in the multispectral image sensor.

6. A method for correcting non-uniformity errors in a multi-spectral image, comprising:

acquiring an initial multispectral image;

carrying out non-uniformity correction on the initial multispectral image according to a preset correction matrix to obtain a target multispectral image; wherein the correction matrix is calculated by the calculation method of the correction matrix according to any one of claims 1 to 5.

7. The method for correcting non-uniformity errors in a multi-spectral image according to claim 6, wherein said correction matrix comprises a sub-correction matrix for each channel, and said non-uniformity correcting said initial multi-spectral image according to a predetermined correction matrix to obtain a target multi-spectral image comprises:

and correcting the data of each channel in the initial multispectral image according to the sub-correction matrix corresponding to each channel to obtain the target multispectral image.

8. An apparatus for calculating a correction matrix, comprising:

the device comprises a first acquisition unit, a second acquisition unit and a control unit, wherein the first acquisition unit is used for acquiring a calibration multispectral image of a white board and acquiring first data of each channel in the calibration multispectral image, and the calibration multispectral image is acquired by the multispectral image sensor;

a second acquisition unit configured to acquire a reference position in the multispectral image sensor; wherein a coincidence degree between an actual multi-channel response curve and an ideal multi-channel response curve of the multi-spectral image sensor at the reference position is greater than a preset threshold value;

the processing unit is used for determining a reference region in the calibration multispectral image according to the reference position and acquiring second data of each channel in the reference region;

and the calculation unit is used for calculating a correction matrix corresponding to each channel according to the first data and the second data.

9. An apparatus for correcting a non-uniformity of a multispectral image, comprising:

an acquisition unit for acquiring an initial multispectral image;

the nonuniformity correction unit is used for performing nonuniformity correction on the initial multispectral image according to a preset correction matrix to obtain a target multispectral image; wherein the correction matrix is calculated by the calculation method of the correction matrix according to any one of claims 1 to 5.

10. A terminal, comprising:

the multispectral image sensor is used for collecting multispectral images;

a processor for receiving the multispectral image and performing the method for calculating the correction matrix according to any one of claims 1 to 5 or the method for correcting the non-uniformity errors of the multispectral image according to claim 6 or 7.

11. A computer-readable storage medium, in which a computer program is stored, which, when being executed by a processor, carries out a method of calculating a correction matrix according to any one of claims 1 to 5, or a method of correcting a non-uniformity error of a multispectral image according to claim 6 or 7.

Images