JP7450917B2 - Multispectral image capturing device, inspection device, and multispectral image capturing method - Google Patents

Description

The present invention relates to a multispectral image capturing device, an inspection device, and a multispectral image capturing method.

For example, Patent Documents 1 to 5 disclose multispectral image capturing devices. A multispectral image capturing device captures a multispectral image by capturing an image of an object using a multiband camera that can capture images of multiple bands having different spectral sensitivity characteristics.

Japanese Patent Application Publication No. 2005-181038 Japanese Patent Application Publication No. 2010-122080 Japanese Patent Application Publication No. 2012-138652 Japanese Patent Application Publication No. 2008-136251 International Publication No. 2016/031922

However, in conventional multispectral image capturing devices, there is a problem that the structure of the multiband camera is complicated.
An object of the present invention is to provide a multispectral image capturing device and a multispectral image capturing method that can obtain a multispectral image of a subject with a simple configuration. Another object of the present invention is to provide an inspection device that inspects a subject based on a multispectral image acquired by a multispectral image capturing device.

Below, means for solving the above problems and their effects will be described.
A multispectral image capturing device that solves the above problems is a multispectral image capturing device that is equipped with an image sensor that has a color filter having a plurality of different transmittance characteristics in each pixel. an optical filter having a spectral transmittance characteristic having a plurality of blocking regions and one or more transmitting regions within a non-visible light wavelength region, and disposed on an optical path between the image sensor and the subject; , separates a first imaging signal output from the image sensor into imaging signals for each color of the color filter, performs a matrix operation between the separated imaging signals, and generates a plurality of bands having imaging characteristics different from those of the color filter. a converter for converting into a second image pickup signal, at least one band of the plurality of bands of second image pickup signals has sensitivity in a non-visible light wavelength region.

According to this configuration, a plurality of multispectral images can be generated for one optical filter.
The multispectral image capturing device includes: a plurality of the optical filters having the spectral transmittance characteristics different from each other; a switching unit that switches between the plurality of optical filters disposed on the optical path among the plurality of optical filters; a sensor and a control section that controls the switching section, and the control section controls the switching section to sequentially switch one of the optical filters disposed on the optical path in a time-sharing manner, and also controls the switching section to By imaging a subject in time division, the image sensor outputs the first imaging signal for each of the plurality of time division images, and the conversion unit outputs the first imaging signal from the image sensor for each of the plurality of time division images. By separating the first imaging signal into imaging signals for each color filter and performing a matrix operation between the separated imaging signals, the second imaging signal of multiple bands having imaging characteristics different from those of the color filter is obtained. Convert. The number of bands of the second image pickup signal may be greater than the number of colors of the color filter.

According to this configuration, by switching the optical filters, the image sensor images the subject in a time-division manner for each optical filter, so it is possible to generate a multispectral image with a greater number of bands than the number of colors of the color filter.

The multispectral image capturing device includes a plurality of light sources with different emission spectra, and the number of the light sources that are emitted simultaneously among the plurality of light sources is smaller than the plurality of light sources, so that each light intensity is increased during one shot imaging period. The light may be emitted in a time-division manner while controlling.

According to this configuration, it is possible to irradiate the subject with light in the visible wavelength region and non-visible wavelength region while keeping the maximum allowable current of the power source low. Therefore, a multispectral image of multiple bands including at least one band in the non-visible wavelength region can be obtained. Furthermore, the spectral sensitivity characteristics of multiple bands obtained by the optical filter can be corrected using the emission spectrum of the light source, and with a higher degree of freedom, multispectral imaging characteristics can be realized.

The multispectral image capturing device performs image processing to extract a specific region in an image based on the second imaging signal of multiple bands obtained by imaging a reference subject given in advance, and The amplification factor of the second image pickup signal for each band may be set so that the integrated value of the second image pickup signal for each band in all areas other than the specific area is equal between bands.

According to this configuration, a multispectral image of multiple bands can be obtained as a pseudocolor image. For example, a user can visually recognize a multispectral image of multiple bands displayed on the display unit as a pseudo-color image.

An inspection device that solves the above problems inspects the subject based on the multispectral image capturing device and the second imaging signal of a plurality of bands output when the multispectral image capturing device images the subject to be inspected. and an inspection processing section.

According to this configuration, an inspection device that can inspect a subject using multispectral images of multiple bands can be provided with a simple configuration.
A multispectral image capturing method that solves the above problems is a multispectral image capturing method that uses an image sensor having a color filter with a plurality of different transmittance characteristics in each pixel, and uses one region or An optical filter having spectral transmittance characteristics having a plurality of blocking regions and one or more transmitting regions within a non-visible light wavelength region is disposed on an optical path between the image sensor and the subject, an imaging step in which the image sensor images a subject through the optical filter; a first imaging signal output from the image sensor is separated into imaging signals for each color of the color filter; and a matrix calculation is performed between the separated imaging signals. and a conversion step of converting into a plurality of bands of second imaging signals having imaging characteristics different from those of the color filter, at least one band of the plurality of bands of second imaging signals is in a non-visible light wavelength region. Has sensitivity.

According to this method, effects similar to those of a multispectral image capturing device can be obtained.

According to the present invention, a multispectral image of a subject can be obtained with a simple configuration.

FIG. 1 is a schematic diagram showing an inspection system including an inspection device according to a first embodiment. (a) is a block diagram of a general-purpose color camera and a graph showing the relationship between wavelength and relative sensitivity; (b) is a block diagram of a color camera without an IR cut filter and a graph showing the relationship between wavelength and relative intensity. A configuration diagram of a camera and a graph showing the relationship between wavelength and relative sensitivity. (a) is a graph showing the light transmittance characteristics of the optical filter F1, and (b) is a graph showing the relative sensitivity of each color of the image sensor. The graph which shows the relative sensitivity for each band of a 2nd image pick-up signal. (a) is a graph showing the light transmittance characteristics of the optical filter F2, and (b) is a graph showing the relative sensitivity of each color of the image sensor. (a) is a graph showing the relative sensitivity for each color of the first imaging signal, and (b) is a graph showing the relative sensitivity for each band of the second imaging signal. (a) is a graph showing the light transmittance characteristics of the optical filter F3, and (b) is a graph showing the relative sensitivity of each color of the image sensor. (a) is a graph showing the relative sensitivity for each color of the first imaging signal, and (b) is a graph showing the relative sensitivity for each band of the second imaging signal. (a) is a graph showing the light transmittance characteristics of the optical filter F4, and (b) is a graph showing the relative sensitivity of each color of the image sensor. (a) is a graph showing the relative sensitivity for each color of the first imaging signal, and (b) is a graph showing the relative sensitivity for each band of the second imaging signal. FIG. 2 is a block diagram showing the electrical configuration of a control system of the multispectral image capturing device. A timing chart showing multispectral image capturing control. FIG. 2 is a block diagram showing the functional configuration of the inspection device. (a) is a schematic diagram explaining a specific area of a reference image, and (b) is a schematic diagram explaining a process of calculating an amplification factor based on a background area other than the specific area of the reference image. FIG. 2 is a schematic diagram showing an inspection system including an inspection device according to a second embodiment. FIG. 2 is a block diagram showing the electrical configuration of a control system of the multispectral image capturing device. A timing chart showing multispectral image capturing control. (a) and (b) are graphs showing the relative sensitivity of each frame of the second imaging signal captured in the first and second frames, respectively. A graph showing the relative sensitivity of 6 bands obtained by combining two imaging signals. 5 is a timing chart showing imaging control in a modified example.

(First embodiment)
Hereinafter, a multispectral image capturing device and an inspection device equipped with this multispectral image capturing device will be described with reference to the drawings.

The inspection system 10 shown in FIG. 1 inspects the quality of the article 12 using a captured image of the article 12 as a subject. The inspection system 10 includes a conveyance device 13 that conveys the article 12, and an inspection device 11 that inspects the quality of the article 12 based on the image taken by a camera 30 of the article 12 conveyed by the conveyance device 13. . The inspection device 11 includes a multispectral image capturing device 15 that generates multispectral images of multiple bands, and an inspection processing section 70 that performs inspection using the multispectral images. The multispectral image capturing device 15 includes a camera 30 that captures an image of the article 12, which is an example of a subject being conveyed by the conveying device 13, and a control processing unit 40 that is electrically connected to the camera 30. At least a portion of the control processing unit 40 is configured by a computer. The computer includes an input device and a display section. In this embodiment, the transport device 13 is driven by a control section of the transport system that is communicably connected to the control system of the inspection device 11. Note that the control processing unit 40 may control the transport device 13.

As shown in FIG. 1, the conveyance device 13 includes a conveyor 16 for conveying the article 12, a sensor 17 for detecting the article 12 placed on the conveyor 16 and conveyed, and an inspection result detected by the inspection device 11. It is provided with a discharge device (not shown) that removes the article 12 from the line through which non-defective products flow if it is determined to be a non-defective product. The conveyor 16 may be a belt conveyor, a roller conveyor, etc. as long as it can convey the article 12, or it may be one that grips and conveys the article 12 or conveys the article 12 in a suspended state. The ejection device may have a configuration that pushes out the article 12 to remove it, or a configuration that blows the article 12 away using air force.

The inspection device 11 includes a multispectral image capturing device 15 that generates multispectral images of multiple bands, an inspection processing unit 70 that inspects the quality of the article 12 using the multispectral images of multiple bands, and a multispectral image and inspection results. and a display section 41 that displays. The inspection device 11 inspects the article 12 based on the second imaging signal S2 of multiple bands output when the multispectral image capturing device 15 images the article 12, which is the subject to be inspected. The display unit 41 may be a monitor connected to a computer, or a display provided on an operation panel.

The multispectral image capturing device 15 includes an illumination unit 20 that irradiates light onto the article 12, a camera 30 that captures an image of the article 12 as a subject, and a control processing unit 40. The control processing unit 40 includes a control unit 50 that controls the illumination unit 20 and the camera 30, and a conversion unit 60 that converts the captured image captured by the camera 30 into a multispectral image of multiple bands. The converter 60 converts the first image signal S1 captured by the camera 30 into a second image signal S2 representing a multispectral image of multiple bands.

The inspection processing section 70 inspects the quality of the article 12 using the multi-band multispectral image generated by the conversion section 60 that constitutes the multispectral image capturing device 15, and uses the multispectral image used for the inspection and the inspection results. It is displayed on the display section 41. Further, the multispectral image capturing device 15 displays the multiband multispectral image generated by the conversion unit 60 on the display unit 41.

As shown in FIGS. 1 and 3, the camera 30 includes an optical filter 31, a lens 32, and an image sensor 33. The optical filter 31 is arranged on the optical path between the object 12 and the image sensor 33. In the example shown in FIG. 1, the optical filter 31 is placed between the object 12 and the lens 32, but it may also be placed between the lens 32 and the image sensor 33.

The image sensor 33 receives an image of the object 12 through the optical filter 31 and the lens 32, and outputs a first image signal S1 according to the result of the received light. The first image signal S1 output from the image sensor 33 is input to the converter 60. The converter 60 converts the first image signal S1 into a second image signal S2 representing a multispectral image of multiple bands.

The inspection processing unit 70 shown in FIG. 1 inspects the quality of the object 12 using the multi-band multispectral image represented by the second image signal S2 converted by the conversion unit 60. The second imaging signal S2 of multiple bands is an image signal for displaying a multispectral image.

The illumination unit 20 shown in FIG. 1 may be a single light emitting unit with one type of emission spectrum, but in the first embodiment, a plurality of light emitting units are used in order to improve multispectral characteristics and performance. use The illumination unit 20 includes a plurality of light sources 21 to 24 having different emission spectra. The illumination unit 20 of this example includes a first light source 21, a second light source 22, a third light source 23, and a fourth light source 24. The plurality of light sources 21 to 24 are composed of, for example, LEDs. The illumination unit 20 is configured by combining a plurality of light sources 21 to 24 having different emission spectra. The control unit 50 causes the plurality of light sources 21 to 24 to emit light to irradiate illumination light including visible light and near-infrared light toward the article 12, which is the subject.

The multispectral image capturing device 15 includes an image sensor 33 having a color filter 34 having a plurality of different transmittance characteristics on a pixel basis. The color filter 34 is configured by arranging a plurality of R filters 34R, G filters 34G, and B filters 34B each having a plurality of different transmittance characteristics in pixel units.

Next, the configuration of the image sensor 33 included in the camera 30 will be described with reference to FIGS. 2(a) and 2(b). FIG. 2A shows a general-purpose color camera 200 that captures RGB images. The color camera 200 includes a lens 32 assembled to a lens barrel 30a, a near-infrared light cut filter 201 (hereinafter also referred to as IR cut filter 201) that blocks near-infrared light, and an image sensor 33. The image sensor 33 includes an R light receiving element 33R that receives red light that has passed through an R filter 34R and outputs an R imaging signal according to the amount of received light, and an R light receiving element 33R that receives red light that has passed through a G filter 34G and outputs an R imaging signal that corresponds to the amount of received light. It includes a G light receiving element 33G that outputs an imaging signal, and a B light receiving element 33B that receives blue light transmitted through a B filter 34B and outputs a B imaging signal according to the amount of received light. In the image sensor 33, the R light receiving element 33R, the G light receiving element 33G, and the B light receiving element 33B are arranged in a predetermined arrangement.

This image sensor 33 has RGB imaging characteristics in which near-infrared light is cut. The R light receiving element 33R, the G light receiving element 33G, and the B light receiving element 33B are sensitive to light in respective wavelength bands shown in the graph of FIG. 2(a). In this graph, the horizontal axis shows wavelength and the vertical axis shows relative sensitivity. The R light receiving element 33R has high sensitivity to light in the red (R) wavelength band shown in the graph in FIG. 2(a). The G light receiving element 33G has high sensitivity to light in the green (G) wavelength band shown in the graph in FIG. 2(a). The B light receiving element 33B has high sensitivity to light in the blue (B) wavelength band shown in the graph in FIG. 2(a).

FIG. 2(b) shows a color camera 250 obtained by removing the IR cut filter 201 from the general-purpose color camera 200 shown in FIG. 2(a). The image sensor 33 built into the color camera 250 has RGB imaging characteristics in which near-infrared light is not cut and includes a wavelength band of near-infrared light. The R light-receiving element 33R, the G light-receiving element 33G, and the B light-receiving element 33B are sensitive to light in the visible light wavelength region VA and invisible light wavelength region IVA (particularly in the near-infrared wavelength region) shown in the graph in FIG. 2(b). has.

FIG. 3 shows a schematic configuration of the camera 30 of this embodiment. As shown in FIG. 3, the camera 30 includes an optical filter 31 on the optical path between the image sensor 33 and the object 12. The camera 30 does not include the IR cut filter 201 shown in FIG. 2(a). The image sensor 33 has a configuration similar to that of the general-purpose color camera 200 shown in FIG. 2(a).

The image sensor 33 itself has sensitivity in the visible light wavelength region VA and the invisible light wavelength region IVA, as shown in the graph of FIG. 2(b). The camera 30 is configured, for example, by removing the IR cut filter 201 from the general-purpose color camera 200 shown in FIG. 2A and then attaching an optical filter 31 on the optical path. Note that the camera 30 is not limited to a configuration based on the general-purpose color camera 200.

The color filter 34 constituting the image sensor 33 is an RGB primary color filter, but may also be a complementary color filter of Mg, Ye, or Cy. Further, in addition to the RGB filter or the complementary color filter, an NIR filter that selectively transmits near-infrared light may be mixed. Further, the RGB filter may be an R, G1, G2, B filter, or the color filter 34 may be a combination of a complementary color filter and a primary color filter. Furthermore, three or more types of filters may be combined.

The optical filter 31 has a spectral transmittance characteristic having one or more blocking regions in the visible light wavelength region VA and one or more transmitting regions in the non-visible light wavelength region IVA. In order to obtain such spectral transmittance characteristics, the optical filter 31 is composed of one optical filter or two optical filters 31A and 31B as shown in FIG. 3. Since the optical filter 31 shown in FIG. 3 has one or more transmission regions within the invisible wavelength region IVA, the light of the invisible wavelength that has passed through the optical filter 31 from the object 12 is The light is received by an image sensor 33 shown in FIG. The optical filter 31 of this embodiment transmits at least light in a part of the wavelength band within the near-infrared wavelength region. The near-infrared wavelength region is a wavelength region of about 800 to about 2500 nm. Note that one optical filter 31 may be composed of three or more filters, but since the light transmittance decreases and it is necessary to increase the amount of light from the illumination section 20, it is preferable that the number is small from the viewpoint of power saving.

FIG. 4(a) shows an example of the spectral transmittance characteristics of the optical filter 31. In this embodiment, the optical filter 31 includes an optical filter F1 having a spectral transmittance characteristic shown in FIG. 4(a), an optical filter F2 having a spectral transmittance characteristic shown in FIG. 6(a), and an optical filter F2 having a spectral transmittance characteristic shown in FIG. An optical filter F3 having the spectral transmittance characteristic shown in FIG. 10A and an optical filter F4 having the spectral transmittance characteristic shown in FIG. 10(a) are used. The optical filter 31 used has a spectral transmittance characteristic suitable for the inspection, depending on the difference in the spectral reflectance characteristic of the object identified in the subject to be inspected.

The optical filter F1 shown in FIG. 4A has a spectral transmittance characteristic that has a blocking region in a wavelength region of about 620 nm or less in the visible light wavelength region VA, and a transmitting region in a wavelength region exceeding about 620 nm. In the non-visible light wavelength region IVA, the entire region shown is a transmission region. The optical filter F1 has a transmission region at least in the near-infrared wavelength region.

In the camera 30 shown in FIG. 3, when the optical filter 31 is an optical filter F1 having the spectral transmittance characteristics shown in FIG. Light with a wavelength of 620 nm or less is blocked, and light with a wavelength exceeding about 620 nm is transmitted. Therefore, even if the image sensor 33 itself has the sensitivity shown in FIG. 4(b), the image sensor 33 does not receive light having a wavelength of about 620 nm or less that is blocked by the optical filter 31. Therefore, as shown in the graph of FIG. 3, the image sensor 33 has substantial sensitivity in a wavelength region exceeding about 620 nm. In other words, the image sensor 33 has no sensitivity in a wavelength region of about 620 nm or less, but has sensitivity in a wavelength region of more than about 620 nm.

The R light-receiving element 33R constituting the image sensor 33 shown in FIG. Element 33B has sensitivity in the band designated B. Each light receiving element 33R, 33G, 33B receives the light that has passed through each filter 34R, 34G, 34B of the color filter 34, out of the light that has passed through the optical filter 31, according to its sensitivity. The image sensor 33 receives a first image signal S1 in which each image signal having an R value, a G value, and a B value according to the amount of light received by each of the light receiving elements 33R, 33G, and 33B is serially arranged in the order of a predetermined arrangement pattern. Output.

The converter 60 shown in FIG. 5 generates the second image signal S2 by converting the RGB values of the first image signal S1 input from the image sensor 33 shown in FIG. 3 into XYZ values. The graph shown in FIG. 5 shows three bands represented by XYZ values that constitute the second image pickup signal S2. The horizontal axis of the graph shows wavelength, and the vertical axis shows relative sensitivity.

As shown in FIG. 5, in the present embodiment, the peak of at least one of the three bands of the second imaging signal S2 is in the near-infrared wavelength region, which has a longer wavelength than the visible wavelength region VA. In particular, in this example, the peaks of the two bands are in the near-infrared wavelength region. That is, the conversion unit 60 converts the three-band RGB signal shown in the graph of FIG. 3 into a three-band XYZ signal having sensitivity to a wavelength band different from that of the RGB signal. As shown in the graph of FIG. 5, the band indicated by X has a peak at about 870 nm, the band indicated by Y has a peak at about 770 nm, and the band indicated by Z has a peak at about 670 nm. Each of the sensitivity peaks shown by the three XYZ bands is higher than the sensitivity of the other two bands, and each XYZ band is separated.

The optical filter F2 shown in FIG. 6(a) has a cutoff region in the wavelength range of about 580 to 730 nm where the transmittance is 0.1 or less, and a cutoff region in the wavelength range of about 520 nm or less where the transmittance is 0.7 or more. It has spectral transmittance characteristics with a transmission region in the range of 770 nm or more. In other words, the optical filter F2 has a blocking region within the wavelength range of approximately 580 to 700 nm within the visible light wavelength region VA, and a transmitting region within the wavelength range of approximately 770 nm or more within the non-visible wavelength region IVA. Has transmittance characteristics.

Therefore, even if the image sensor 33 itself has the relative sensitivity of the three bands of RGB shown in FIG. 6(b), the image sensor 33 has , has a substantial relative sensitivity as shown in the graph of FIG. 7(a).

Then, the converter 60 converts the three-band RGB signal shown in the graph of FIG. 7(a) into a three-band XYZ signal having sensitivity to a wavelength band different from that of the RGB signal.
As shown in FIG. 7B, the peak of one of the three XYZ bands constituting the second image pickup signal S2 is in the near-infrared wavelength region of the invisible wavelength region IVA. The peaks of the other two bands are in the visible wavelength region VA. That is, as shown in the graph of FIG. 7(b), the X band has a peak at about 780 nm, the Y band has a peak at about 530 nm, and the Z band has a peak at about 460 nm. Each peak of relative sensitivity shown by the three XYZ bands is higher than the relative sensitivity value of the other two bands at the wavelength of each peak, and each band of XYZ is separated.

Further, the optical filter F3 shown in FIG. 8(a) has a cutoff region in which the transmittance is 0.1 or less in a plurality of wavelength bands, and a transmission region in which the transmittance is 0.7 or more in a plurality of wavelength bands. It has spectral transmittance characteristics. In other words, the optical filter F3 has a spectral transmittance that has a plurality of blocking regions in the visible wavelength region VA and a plurality of transmitting regions in the near-infrared wavelength region within the invisible wavelength region IVA. have characteristics.

Therefore, even if the image sensor 33 itself has the relative sensitivity of the three RGB bands shown in FIG. 8(b), the image sensor 33 has a relative sensitivity of three bands of RGB shown in FIG. , has a substantial relative sensitivity as shown in the graph of FIG. 9(a).

Then, the converter 60 converts the three-band RGB signal shown in the graph of FIG. 9(a) into a three-band XYZ signal having sensitivity to a wavelength band different from that of the RGB signal.
As shown in FIG. 9B, the peak of one of the three XYZ bands constituting the second image pickup signal S2 is in the near-infrared wavelength region of the invisible wavelength region IVA. The peaks of the other two bands are in the visible wavelength region VA. That is, as shown in the graph of FIG. 9(b), the X band has a peak at about 960 nm, the Y band has a peak at about 670 nm, and the Z band has a peak at about 560 nm. Each peak of relative sensitivity shown by the three XYZ bands is higher than the relative sensitivity value of the other two bands at the wavelength of each peak, and each band of XYZ is separated.

Furthermore, the optical filter F4 shown in FIG. 10(a) has a blocking region in which the transmittance is 0.1 or less in a plurality of wavelength bands, and a transmission region in which the transmittance is 0.7 or more in a plurality of wavelength bands. It has spectral transmittance characteristics. In other words, the optical filter F4 has a spectral transmittance that has a plurality of blocking regions in the visible wavelength region VA and a plurality of transmitting regions in the near-infrared wavelength region within the invisible wavelength region IVA. have characteristics.

Therefore, even if the image sensor 33 itself has the relative sensitivity of the three bands of RGB shown in FIG. 10(b), the image sensor 33 has , has a substantial relative sensitivity as shown in the graph of FIG. 11(a).

Then, the converter 60 converts the three-band RGB signal shown in the graph of FIG. 11(a) into a three-band XYZ signal having sensitivity to a wavelength band different from that of the RGB signal.
As shown in FIG. 11(b), in the second image pickup signal S2, one of the three bands has a peak in the near-infrared wavelength region of the invisible wavelength region IVA. The peaks of the other two bands are in the visible wavelength region VA. That is, as shown in the graph of FIG. 11(b), the X band has a peak at about 870 nm, the Y band has a peak at about 800 nm, and the Z band has a peak at about 830 nm. Each peak of relative sensitivity shown by the three XYZ bands is higher than the relative sensitivity value of the other two bands at the wavelength of each peak, and each band of XYZ is separated.

Next, with reference to FIG. 12, the electrical configuration of the control system of the multispectral image capturing device 15 will be described. As shown in FIG. 12, the control section 50 includes a light emission control section 51 and an imaging control section 52. The light emission control section 51 is electrically connected to the power source 25 and receives power at a predetermined voltage from the power source 25 . The light emission control unit 51 performs light emission control to time-divisionally cause the first light source 21, second light source 22, third light source 23, and fourth light source 24 that constitute the illumination unit 20 to emit light.

The imaging control unit 52 inputs an imaging trigger signal based on a detection signal when the sensor 17 (see FIG. 1) detects the article 12. Upon receiving the imaging trigger signal, the imaging control section 52 outputs a light emission command signal LS to the light emission control section 51 and outputs an imaging command signal IS to the image sensor 33. When the light emission control unit 51 receives the light emission command signal LS, it causes the plurality of light sources 21 to 24 to emit light by combining them in a time-sharing manner. The image sensor 33 performs an imaging operation upon receiving the imaging command signal IS. As a result of the imaging operation, the image sensor 33 outputs the first imaging signal S1 to the conversion unit 60. The first imaging signal S1 is an RGB imaging signal.

Next, the operation timing of the multispectral image capturing device 15 will be explained with reference to FIG. 13. FIG. 13 shows the timing of the light emission control that causes the plurality of light sources 21 to 24 to emit light in a time-division manner, the optical filter 31 used, and the imaging operation of the image sensor 33 in the process of capturing one shot of an object. show.

The camera 30 captures one frame through one optical filter 31 in one shot of one subject. Furthermore, by causing the plurality of light sources 21 to 24 constituting the illumination unit 20 to emit light in a time-division manner, illumination with a necessary light intensity without exceeding the allowable maximum current value of the power source 25 can be obtained. At this time, the light emission control includes controlling the light emission timing of the plurality of light sources 21 to 24 and controlling the current value flowing through each of the plurality of light sources 21 to 24. In the imaging process of one frame, by causing all of the first light source 21, second light source 22, third light source 23, and fourth light source 24 to emit light in a time-sharing manner, the visible light wavelength range VA and the non-visible light wavelength range IVA are emitted. Illumination light having an emission spectrum is irradiated onto the subject.

Here, if the number of multiple light sources is J (however, J≧2), then J=4 in this example. The control unit 50 causes the J light sources 21 to 24 to emit light in a time-division manner while controlling the light intensity of each light source, by making the number of light sources that emit light at the same time smaller than J, and controlling the light intensity of each light source during one shot imaging period. The multispectral image capturing device 15 acquires a three-band multispectral image of the article 12 by the camera 30 capturing one frame for each article 12 . Note that J may be 1. Multispectral imaging is possible even with a single light source.

Next, the configurations of the converter 60 and the inspection processor 70 will be described with reference to FIG. 14.
As shown in FIG. 14, an image of the article 12 is formed on an imaging surface of an image sensor 33 through an optical filter 31 and a lens 32. The image sensor 33 outputs the first imaging signal S1 as the imaging result of the article 12 to the conversion unit 60. The first image signal S1 is a serial signal including an R image signal (red signal), a G image signal (green signal), and a B image signal (blue signal) from each of the light receiving elements 33R, 33G, and 33B. Note that the R imaging signal, the G imaging signal, and the B imaging signal are also simply referred to as an R signal, a G signal, and a B signal.

As shown in FIG. 14, the conversion section 60 includes an RGB separation section 61, an XYZ conversion section 62, a white balance section 63, and a high brightness color suppression section 64. The RGB separation unit 61 separates the first image signal S1 input from the image sensor 33 into an R image signal, a G image signal, and a B image signal.

The XYZ converter 62 converts the R, G, and B signals input from the RGB separator 61 into X, Y, and Z signals. Specifically, the XYZ conversion unit 62 converts the RGB values, which are the signal values of the R signal, G signal, and B signal, into an X signal, a Y signal, and a Z signal by performing a matrix operation. The XYZ converter 62 is given matrix coefficients. Here, the matrix used for matrix calculation is a 3×3 matrix. The XYZ converter 62 is given coefficients of a 3×3 matrix.

The XYZ converter 62 performs a matrix operation of multiplying the RGB values of the first image signal S1 by a 3×3 matrix specified using matrix coefficients, and converts the spectral characteristics different from the RGB values of the first image signal S1. It is converted into a second image pickup signal S2 expressed by XYZ. The matrix coefficient is a coefficient for dividing RGB of the first imaging signal S1 into multiple bands of XYZ of the second imaging signal S2.

Here, the calculation formula for converting the RGB signal, which is the first imaging signal S1, into the XYZ signal, which is the second imaging signal S2, is given by the following equation (1).

Here, a1 to a3, b1 to b3, and c1 to c3 are matrix coefficients, and Gx, Gy, and Gz are amplification factors.

The XYZ conversion unit 62 performs arithmetic processing of multiplying the RGB values by a 3×3 matrix in equation (1) above. The XYZ conversion section 62 outputs the XYZ values before being multiplied by the amplification factor to the white balance section 63.

Here, the number of colors of the color filter 34 in the image sensor 33 is assumed to be n (however, n is a natural number of 3 or more). The matrix calculation performed between the n imaging signals is an n×n matrix calculation. Matrix coefficients are set in the n×n matrix that can separate the imaging signal for each color of the first imaging signal S1 into n-band wavelength regions. In this example, the imaging signals for each color of the first imaging signal S1 are an R signal, a G signal, and a B signal, and the number of colors n is "3" (n=3). That is, the n×n matrix is a 3×3 matrix. Matrix coefficients are set in the 3×3 matrix that can improve the separation of the three bands.

For example, when the optical filter F1 shown in FIG. 4(a) is used, a1=-0.05, a2=- 0.1, a3=1, b1=-0.26, b2=2, b3=-1.5, c1=1, c2=-1.5, c3=0.4.

For example, when the optical filter F2 shown in FIG. 6(a) is used, a1= 1, a2=0, a3=0, b1=-0.6, b2=1, b3=-0.1, c1=-0.2, c2=-0.45, c3=1.

Note that the present invention is not limited to the n×n matrix operation, and may be an n×m (where m≠n) matrix operation. When the number of colors is "3", the number of colors is not limited to 3x3 matrix calculation, but 3x4 matrix calculation is performed to generate a 4-band multispectral image, or 3x2 matrix calculation is performed to generate a 4-band multispectral image. It is also possible to generate a multispectral image with a small number of bands.

The white balance section 63 multiplies the XYZ values from the XYZ conversion section 62 by the given X amplification factor Gx, Y amplification factor Gy, and Z amplification factor Gz, respectively. The white balance unit 63 multiplies the X value after XYZ conversion by the X amplification factor Gx, the Y value by the Y amplification factor Gy, and the Z value by the Z amplification factor Gz. In other words, the white balance unit 63 adjusts the white balance by performing an operation of multiplying the amplification factors Gx, Gy, and Gz by a 1×3 matrix using the amplification factors Gx, Gy, and Gz as matrix coefficients in the above equation (1). The white balance section 63 outputs the XYZ values with adjusted white balance to the high-luminance color suppression section 64. Note that white balance adjustment may be performed by fixing one signal level and adjusting the other two signal levels. For example, the Y signal may be fixed, and the X and Z signals may be adjusted.

The high-luminance color suppression unit 64 considers the luminance equal to or higher than the threshold value to be an achromatic color, and suppresses the color of that portion. In this way, the conversion unit 60 sequentially performs RGB separation processing, XYZ conversion processing, white balance processing, and high-luminance color suppression processing on the input first image signal S1, thereby converting the input first image signal S1 into a first image that can express an XYZ pseudo-color image. 2 outputs the imaging signal S2. The converting unit 60 outputs a three-band multispectral image, which is an XYZ pseudo-color image of the article 12, to the inspection processing unit 70.

Next, the inspection processing section 70 will be explained.
The inspection processing section 70 includes a specific area detection processing section 71 and a determination processing section 72.
The specific area detection processing unit 71 detects a specific area EA in the XYZ pseudo color image SI based on the given parameters. Here, the specific area EA refers to an area to be inspected that is an inspection target for determining the quality of the article 12. The specific area detection processing unit 71 calculates the number and area of specific areas EA detected in the subject area in the XYZ pseudo-color image SI. Specifically, the specific area detection processing unit 71 counts the number of specific areas EA detected based on the parameters using a counter (not shown), and determines the area of the detected specific area EA.

The determination processing unit 72 inspects the quality of the article 12 based on the number and area of the specific areas EA based on the given threshold value. The determination processing unit 72 is given, as thresholds, a number threshold used to determine the number of specific areas EA and an area threshold used to determine the area. For example, the determination processing unit 72 determines whether the number of specific areas EA exceeding the area threshold exceeds a number threshold. Alternatively, the determination processing unit 72 determines whether the number of specific areas exceeds the threshold and the total area of the areas of the specific areas EA exceeds the area threshold. The determination processing unit 72 determines that the product is a good product if the determination conditions for the inspection are met, and determines it to be a defective product if the determination conditions are not met.

Note that the method by which the determination processing unit 72 determines whether the specific area EA is a good product or a defective product based on the number and area can be changed as appropriate. For example, the determination processing unit 72 may determine that the article 12 is a good product or, conversely, that the article 12 is a defective product when the number of specific areas EA exceeding the area threshold exceeds a number threshold. Moreover, the quality of the article 12 may be determined only by the area of the specific area EA, or the quality of the article 12 may be determined only by the number.

Next, a method of setting the amplification factors of X, Y, and Z will be explained with reference to FIG.
A standard image of the article 12, which is the subject to be inspected, is captured in advance. The inspection processing unit 70 is configured by, for example, a part of a personal computer. For example, the inspector causes the display unit 41 such as a monitor to display the reference image SI shown in FIG. 15(a). The inspector manually encloses and specifies the specific area EA to be extracted using a pointing device such as a mouse. If the inspector wants to specify the region to be extracted using color data, he or she manually specifies which region of the color region expressed by XYZ chromaticity coordinates the color is to be extracted. The parameters are given by chromaticity coordinate threshold values that can specify which region of the color region represented by the XYZ chromaticity coordinates is to be extracted. Furthermore, if the area to be extracted is specified by a difference rather than a color, the parameter is given by a threshold value that allows the difference data to specify which area to extract. XYZ here is a value calculated from the XYZ amplification factor that was provisionally set before the official amplification factor was determined, and the parameters specified here are also provisional and are used to extract the specific area EA. This is the threshold of only

When the specific area EA is designated, the inspection processing unit 70 calculates a background area OA that is an area other than the specific area EA in the reference image SI shown in FIG. 15(b) as an initial setting process. Then, the inspection processing unit 70 adds the values within the region for each of the X, Y, and Z imaging signals of the background region OA to obtain each integral value ΣX, ΣY, and ΣZ. Further, the inspection processing unit 70 calculates amplification factors Gx, Gy, and Gz that satisfy ΣX*Gx=ΣY*Gy=ΣZ*Gz. Then, the inspection processing unit 70 initializes Gx, Gy, and Gz as the amplification factors of the X, Y, and Z signals. Note that the specific area EA may be a good area for determining a non-defective product, or a defective area for determining a defective product.

In this way, the inspection processing unit 70 detects the specific area EA in the reference image SI based on the second imaging signal S2 of multiple bands obtained by imaging the article 12, which is a reference object given in advance. Perform image processing to extract. Then, the inspection processing unit 70 generates the second image pickup signal for each band so that the integrated value of the image pickup signal for each band in the entire area of the background area OA other than the specific area EA in the reference image SI is equal between the bands. The amplification factor of S2 is calculated and set as the amplification factor of the second image pickup signal S2 for each band. The coefficients of the 3×3 matrix, the X, Y, Z amplification factors Gx, Gy, Gz, parameters, and threshold values are stored in a storage unit (not shown). Note that the process of calculating the amplification factors Gx, Gy, and Gz may be performed by the conversion unit 60 instead of the inspection processing unit 70.

Next, the functions of the multispectral image capturing device 15 and the inspection device 11 will be explained.
As shown in FIG. 1, a conveyor 16 of a conveyance device 13 conveys an article 12, which is a subject. When the sensor 17 detects the article 12, a trigger signal is input to the control unit 50. As shown in FIG. 12, the imaging control unit 52 that has received the trigger signal outputs a light emission command signal LS to the light emission control unit 51 and outputs an imaging command signal IS to the image sensor 33.

As shown in FIG. 13, the light emission control unit 51 causes the plurality of light sources 21 to 24 to emit light in a time-division manner so as not to exceed the allowable maximum current value of the power source 25. In this illumination process, by causing all of the first light source 21, second light source 22, third light source 23, and fourth light source 24 to emit light in a time-sharing manner, the emission spectrum is divided into the visible light wavelength region VA and the non-visible light wavelength region IVA. An object 12, which is a subject, is irradiated with illumination light having a value of .

Further, the image sensor 33 captures one frame of the article 12 through the optical filter 31 . The image sensor 33 outputs the first image signal S1 to the converter 60.
The conversion unit 60 converts the RGB values into XYZ values by separating the first image signal S1 into RGB and multiplying the separated RGB values by a 3×3 matrix. By multiplying these XYZ values by amplification factors Gx, Gy, and Gz, XYZ values with adjusted white balance are generated. Furthermore, the high-luminance color suppression unit 64 regards the luminance above the threshold value as an achromatic color and suppresses the color of that portion. In this way, the second imaging signal S2 representing the XYZ pseudo-color image is output from the conversion section 60 to the inspection processing section 70.

In the inspection processing section 70, a specific area detection processing section 71 detects a specific area EA in the XYZ pseudo color image based on the second imaging signal S2. For example, if the reference image SI shown in FIG. 15(a) is an XYZ pseudo color image captured during an examination, the specific area detection processing unit 71 detects the specific area EA. Specifically, the specific area detection processing unit 71 detects the area of the color specified by the parameter among the color areas expressed by the XYZ chromaticity coordinates as the specific area EA. Alternatively, if the parameter is given as a difference rather than a color, the specific area detection processing unit 71 detects the area specified by the parameter as the specific area EA. Then, the specific area detection processing unit 71 counts the number of specific areas EA and determines the area of the specific areas EA.

The determination processing unit 72 determines whether the inspection target value of the specific area EA exceeds the threshold value using the given threshold value. The determination processing unit 72 of this example determines whether the article 12 is good or bad by comparing the number of specific areas EA with a number threshold and comparing the area of the specific areas EA with the area threshold. If the inspection result shows that the article 12 is defective, the discharge device is driven to remove the defective article 12 from the conveyor 16.

In this embodiment, a multispectral image capturing method is adopted.
The multispectral image capturing method includes an imaging step and a transformation step. In the imaging step, the image sensor 33 images the object 12 through the optical filter 31. In the conversion step, the first image signal S1 outputted from the image sensor 33 is separated into image signals for each color of the color filter, and a matrix operation is performed between the separated image signals to obtain an image signal that has different image characteristics from those of the color filter. It is converted into a second image pickup signal S2 of multiple bands. At least one band of the second imaging signal S2 of multiple bands has sensitivity to the invisible light wavelength region IVA.

According to the first embodiment described in detail above, the following effects can be obtained.
(1) The multispectral image capturing device 15 includes an image sensor 33 having a color filter 34 having a plurality of different transmittance characteristics for each pixel, an optical filter 31, and a converter 60. The optical filter 31 has a spectral transmittance characteristic having one or more blocking regions in the visible light wavelength region VA and one or more transmitting regions in the non-visible light wavelength region IVA. 33 and the object 12, which is the object, on the optical path. The conversion unit 60 separates the first image signal S1 output from the image sensor 33 into image signals for each color of the color filter 34, performs matrix calculations between the separated image signals, and converts the first image signal S1 output from the image sensor 33 into an image signal different from that of the color filter 34. It is converted into a second image pickup signal S2 of multiple bands having characteristics. At least one band of the second imaging signal S2 of multiple bands has sensitivity to the invisible light wavelength region IVA. Therefore, a plurality of multispectral images can be generated for one optical filter 31. Furthermore, the spectral sensitivity characteristics of multiple bands obtained by the optical filter 31 can be corrected using the emission spectra of the light sources 21 to 24, and with a higher degree of freedom, multispectral imaging characteristics can be realized.

(2) The illumination unit 20 includes a plurality of light sources 21 to 24 having different emission spectra. The number of light sources that are emitted simultaneously among the plurality of light sources 21 to 24 is made smaller than the plurality of light sources, and the light sources are emitted in a time-division manner while controlling the light intensity of each light source during one shot imaging period. Therefore, it is possible to irradiate the subject with light in the visible light wavelength region VA and the invisible light wavelength region IVA while keeping the maximum allowable current of the power source low. Therefore, a multispectral image of multiple bands including at least one band in the invisible wavelength region IVA can be obtained. Furthermore, the spectral sensitivity characteristics of multiple bands obtained by the optical filter 31 can be corrected using the emission spectra of the light sources 21 to 24, and with a higher degree of freedom, multispectral imaging characteristics can be realized.

(3) The inspection processing unit 70 performs image processing to extract a specific region in the reference image SI based on the second imaging signal S2 of multiple bands obtained by imaging a reference subject given in advance. The amplification factors Gx, Gy, and Gz calculated so that the integrated values of the second image pickup signal S2 for each band in the entire area of the background area OA other than the specific area EA in the reference image SI are equal between bands are calculated for each band. is set as the amplification factor of the second image pickup signal S2. Therefore, a multispectral image of multiple bands can be obtained as a pseudocolor image. For example, a user can visually recognize a multispectral image of multiple bands displayed on the display unit 41 as a pseudo-color image.

(4) The inspection device 11 inspects the article 12 based on the multispectral image capturing device 15 and the second imaging signal S2 of multiple bands output when the multispectral image capturing device 15 images the article 12 to be inspected. The inspection processing section 70 is provided. Therefore, it is possible to provide an inspection device 11 with a simple configuration that can inspect a subject using multispectral images of multiple bands.

(5) A multispectral image capturing method using an image sensor 33 having a color filter 34 having a plurality of different transmittance characteristics for each pixel. An optical filter 31 having spectral transmittance characteristics having one or more blocking regions in the visible wavelength region VA and one or more transmitting regions in the non-visible wavelength region IVA is used as the image sensor 33. placed on the optical path between the camera and the subject. This method includes an imaging step in which the image sensor 33 images the article 12 through the optical filter 31, and a conversion step. In the conversion step, the first imaging signal S1 output from the image sensor 33 is separated into imaging signals for each color of the color filter 34, and a matrix operation is performed between the separated imaging signals to obtain imaging characteristics different from those of the color filter 34. The second image signal S2 is converted into a plurality of bands of second image signals S2. At least one band of the second imaging signal S2 of multiple bands has sensitivity to the invisible light wavelength region IVA. According to this method, effects similar to those of the multispectral image capturing device 15 can be obtained.

(Second embodiment)
Next, a second embodiment will be described with reference to FIGS. 16 to 19. The second embodiment differs from the first embodiment in that two frames of an object are captured by switching the optical filter 31 every frame. By acquiring the second imaging signals S2 of three hands per frame, a six-band multispectral image is acquired. Note that the other components of the inspection device 11 are basically the same as those of the first embodiment.

As shown in FIGS. 16 and 17, the multispectral image capturing device 15 includes a filter switching unit 36 as an example of a switching unit that switches the optical filter 31. A plurality of optical filters 31 are set in the filter switching section 36. The filter switching unit 36 switches one of the plurality of optical filters 31 set between the image sensor 33 and the article 12 that is the subject. In FIG. 1, two examples are shown in which the optical filters 31 to be switched are the first optical filter 311 and the second optical filter 312, but the number is not limited to two, and any number may be used. For example, the number of optical filters 31 to be switched may be three, four, or five. The filter switching unit 36 selects the spectral transmittance of the optical filters F1 to F4 (FIG. 4(a), FIG. 6(a), FIG. 8(a), FIG. 10(a)) as the optical filter 31 to be switched. A plurality of different characteristics are set switchably. Note that one optical filter 31 may be composed of one or more optical filters 31A, 31B (see FIG. 3).

As shown in FIG. 17, the control section 50 includes a filter selection control section 53 that controls the filter switching section 36. The imaging control section 52 outputs a switching command signal FS to the filter switching section 36. When the filter switching unit 36 receives the switching command signal FS, it controls switching of the filter switching unit 36 and places the instructed optical filter 31 at a position on the optical path between the image sensor 33 and the subject.

As shown in FIG. 18, the imaging control unit 52 images one subject in multiple frames. For example, the imaging control unit 52 performs first frame imaging and second frame imaging for one subject. The imaging control section 52 outputs a switching command signal FS to the filter selection control section 53 every time one frame of imaging is completed. When the filter selection control unit 53 receives the switching command signal FS, it drives the filter switching unit 36 to switch from the first optical filter 311 to the second optical filter 312. As a result, the second optical filter 312 is placed on the optical path between the subject and the image sensor 33. The imaging control unit 52 causes the image sensor 33 to perform imaging of the second frame.

During imaging, the light emission control unit 51 causes the first light source 21, second light source 22, third light source 23, and fourth light source 24 to emit light in a time-sharing manner. Thereby, the necessary illumination is distributed in time divisions so as not to exceed the allowable maximum current value of the power source 25. As shown in FIG. 18, for example, in the imaging process of the first frame, the first light source 21, the second light source 22, and the third light source 23 are caused to emit light in a time-sharing manner. As a result, the object 12 is irradiated with illumination light having an emission spectrum in the visible wavelength region VA and the non-visible wavelength region IVA. Further, in the imaging process of the second frame, the first light source 21, the third light source 23, and the fourth light source 24 are caused to emit light in a time-sharing manner. As a result, the object 12 is irradiated with illumination light having an emission spectrum in the visible wavelength region VA and the non-visible wavelength region IVA.

The image sensor 33 sequentially outputs the first image signal S1 captured in the first frame and the first image signal S1 captured in the second frame to the conversion unit 60. In the conversion unit 60, an RGB separation unit 61 separates the first image signal S1 into RGB, an XYZ conversion unit 62 performs matrix calculation, and a white balance unit 63 performs processing such as multiplication by an amplification factor, thereby converting the first image signal S1 into three bands. is converted into a second image pickup signal S2. The second imaging signal S2 based on the imaging of the first frame is, for example, a three-band signal represented by X1, Y1, and Z1 shown in FIG. 19(a). Further, the second imaging signal S2 based on the imaging of the second frame is, for example, a three-band signal represented by X2, Y2, and Z2 shown in FIG. 19(b). The converter 60 outputs the 6-band second image signal S2 shown in FIG. 19(c) based on the two types of second image signal S2 of 3 bands each. The converter 60 outputs a 6-band XYZ pseudo color image based on the 6-band second imaging signal S2 to the inspection processor 70. The number of bands of the second image pickup signal S2 is "6", which is greater than the number of colors of the color filter 34 in the optical filter 31, which is "3". In other words, the multispectral image capturing device 15 of this embodiment can acquire the second imaging signal S2 with a larger number of bands than the first embodiment.

The inspection processing unit 70 selects two bands that are easy to identify after inspection from among the six bands and performs inspection processing. The number of bands used in the inspection process is not limited to two bands, but may be three bands, four bands, five bands, or six bands. In the case of two bands, the inspection processing unit 70 determines the quality of the article 12 based on the difference between the two bands, or determines the quality of the article 12 based on the difference between the largest band and the smallest band. Furthermore, in the case of four bands, the inspection processing unit 70 determines the quality of the article 12 based on a comparison or difference between the sum of two bands and the sum of the other two bands among the four bands. The inspection processing unit 70 may perform the inspection process using only one band out of the six bands, or may adopt other appropriate methods for the inspection process content.

According to the second embodiment, in addition to the same effects as the first embodiment, the following effects can be obtained.
(6) A filter switching unit 36 that switches between a plurality of optical filters 31 having mutually different spectral transmittance characteristics and one optical filter 31 disposed on the optical path among the plurality of optical filters 31, an image sensor 33, and a filter switching unit 36; A control section 50 that controls the section 36 is provided. The control unit 50 controls the filter switching unit 36 to sequentially switch one optical filter 31 disposed on the optical path in a time-division manner, and causes the image sensor 33 to image the object 12 in a time-division manner. The image sensor 33 outputs a first imaging signal S1 for each of a plurality of time-divided images. The conversion unit 60 separates the first imaging signal S1 output from the image sensor 33 into imaging signals for each color of the color filter 34 for each of the plurality of time-divided images, and performs matrix calculation between the separated imaging signals. As a result, the second image pickup signal S2 is converted into a plurality of bands of second image pickup signals S2 having image pickup characteristics different from those of the color filter 34. The number of bands of the second image pickup signal S2 is greater than the number of colors of the color filter 34. Therefore, by switching the optical filters 31, the image sensor 33 images the object 12 in time division for each optical filter 31, so it is possible to obtain a multispectral image with a number of bands greater than the number of colors of the color filter 34. . For example, it is possible to improve inspection accuracy and increase the number of types of subjects that can be inspected.

The embodiment is not limited to the above, and may be modified in the following manner.
- As shown in FIG. 20, the subject may be imaged by the camera 30 using wide dynamic range processing. When capturing an image with the camera 30, if the brightness of the subject is matched, the background becomes saturated and becomes pure white. On the other hand, if you match the brightness of the background, the subject will appear completely black. Therefore, as shown in FIG. 20, one optical filter 31 is used to change the brightness of illumination for one subject, and images are taken in two frames. In the first frame, the first light source 21, the second light source 22, and the third light source 23 are caused to emit light in a time-division manner, the illumination is set brightly at the first light intensity, and an image is captured in accordance with the subject. Next, in the second frame, the first light source 21, the second light source 22, the third light source 23, and the fourth light source 24 are caused to emit light in a time-sharing manner, and the illumination is darkened with a second light intensity that is weaker than the first light intensity, and the background light is darkened. Take an image. Note that during the inspection, a part of the object 12 may be saturated by the direct reflected light from the illumination unit 20, causing shine. Wide dynamic imaging may be applied as a means to prevent this shine.

- When using the optical filter 31 that has transmission regions in the visible light wavelength region and the near-infrared light wavelength region, at least one band of the second imaging signal S2 of multiple bands is in the near-infrared light wavelength region. Bye. For example, other bands may have sensitivity not only in the visible wavelength region but also in the mid-infrared wavelength region, far-infrared wavelength region, and ultraviolet wavelength region.

- The configuration may be such that the wavelength region in which the peaks of multiple bands are located is not in the near-infrared wavelength region. The wavelength range in which the peaks of multiple bands are present may be any one of a mid-infrared wavelength range, a far-infrared wavelength range, and an ultraviolet wavelength range.

- The first light source 21, second light source 22, third light source 23, and fourth light source 24 constituting the illumination unit 20 may be caused to emit light all at once instead of being configured to emit light in a time-sharing manner.
- Image data (for example, RGB image data) based on the first image signal captured by the image sensor 33 through the optical filter 31 using the camera 30 is stored in a removable memory such as a USB memory. The image data stored in the removable memory may be read by a personal computer, and the CPU (conversion unit 60) of the personal computer may perform conversion processing including matrix calculation to generate a multispectral image of multiple bands. In other words, the device that performs the imaging step and the device that performs the conversion step may be separate devices. Multispectral images of multiple bands can also be acquired by such a multispectral image capturing method.

- At least one of the amplification factor and the parameter may be calculated by the inspection processing unit 70, or a value calculated in advance by another computer may be written into the memory of the multispectral image capturing device 15.

- The number of colors of the color filter 34 constituting the image sensor 33 is not limited to three or four, but may be five, six, seven, or eight colors. A filter may be used in which at least one of these colors does not transmit visible light but transmits invisible light. For example, the image sensor 33 may include a color filter including a NIR filter that transmits near-infrared light.

- The article 12, which is an example of a subject to be imaged or inspected, is not particularly limited. The article 12 may be, for example, a container such as a plastic bottle or a bottle, a food product, a beverage, an electronic component, an electric appliance, a daily necessities, a part, a member, or a raw material such as powder, granule, or liquid. It is sufficient that the article 12 can be inspected for quality using a multispectral image.

- The object 12 can be inspected using a multi-band multispectral image with at least one band in the non-visible light wavelength region IVA such as the near-infrared region, and an image captured only under visible light cannot be inspected. Any item that can inspect a difficult inspection target is sufficient. The article 12 may be, for example, a container containing liquid, food such as fruits and vegetables, plants such as flowers, semi-processed or processed foods of vegetable or animal origin, living organisms, and the like.

- The object to be imaged or inspected is not limited to the article 12, but may also be a photograph of a building, an aerial topographic photograph, a sky photograph, an astronomical photograph, a microscopic photograph, or the like.
- Using the multispectral image capturing device 15 of the second embodiment, the multispectral image capturing control shown in FIG. 13 may be performed, or the multispectral image capturing control shown in FIG. 20 may be performed.

- The multispectral image capturing device 15 may be configured as a separate device from the inspection processing section 70. The multispectral imaging device 15 may be used for purposes other than inspection.
- The arrangement pattern of the color filters constituting the image sensor 33 is not limited to the RGB Bayer arrangement, but may be any arrangement pattern such as a stripe arrangement.

- At least one of the control unit 50, the conversion unit 60, and the inspection processing unit 70 may be partially or entirely configured by software consisting of a computer that executes a program, or may be configured by hardware such as an electronic circuit. may be done.

(definition)
"At least one band is sensitive to the invisible wavelength region" means that the peak of sensitivity of at least one band is in the invisible wavelength region IVA. In other words, for at least one band, the sensitivity peak is in at least one of a region where the wavelength is longer than the wavelength of visible light and a region where the wavelength is shorter than the wavelength of visible light. Here, "a region having a wavelength longer than a visible light wavelength" refers to at least one of a near-infrared wavelength region, a mid-infrared wavelength region, and a far-infrared wavelength region. Moreover, "a region whose wavelength is shorter than the visible light wavelength" refers to an ultraviolet light wavelength region.

DESCRIPTION OF SYMBOLS 10... Inspection system, 11... Inspection device, 12... Article as an example of a subject, 13... Conveyance device, 15... Multispectral image capturing device, 16... Conveyor, 17... Sensor, 20... Illumination unit, 21... First light source , 22... Second light source, 23... Third light source, 24... Fourth light source, 25... Power supply, 30... Camera, 30a... Lens barrel, 31, F1 to F4... Optical filter, 31A, 32B... Optical filter, 311... First optical filter, 312... Second optical filter, 32... Lens, 33... Color image sensor (image sensor), 33R...R light receiving element, 33G...G light receiving element, 33B...B light receiving element, 34... Color filter, 34R ...R filter constituting an example of a color filter, 34G...G filter constituting an example of a color filter, 34B...B filter constituting an example of a color filter, 36...filter switching unit which is an example of a switching unit, 40...control Processing section, 41... Display section, 50... Control section, 51... Light emission control section, 52... Imaging control section, 53... Filter selection control section, 60... Conversion section, 61... RGB separation section, 62... XYZ conversion section, 63 ...White balance section, 64...High brightness color suppression section, 70...Inspection processing section, 71...Specific area detection processing section, 72...Determination processing section, S1...First imaging signal, S2...Second imaging signal, VA...Visible Optical wavelength range, IVA...invisible light wavelength range, Gx, Gy, Gz...amplification factor, LS...light emission command signal, IS...imaging command signal, SI...reference image, EA...specific area, OA...non-specific area, FS ...Switching command signal.

Claims (5)

A multispectral image capturing device including an image sensor having a color filter having a plurality of different transmittance characteristics on a pixel basis,
It has a spectral transmittance characteristic having one or more blocking regions in the visible wavelength region and one or more transmitting regions in the non-visible wavelength region, an optical filter placed on the optical path;
The first imaging signal output from the image sensor is separated into imaging signals for each color of the color filter, and a matrix operation is performed between the separated imaging signals to generate a plurality of bands having imaging characteristics different from those of the color filter. a conversion unit that converts into a second imaging signal;
At least one band of the plurality of bands of the second imaging signal has sensitivity in a non-visible light wavelength region,
Image processing is performed to extract a specific area in the image based on the second imaging signal of multiple bands obtained by imaging a reference subject given in advance, and A multispectral image capturing device , characterized in that an amplification factor of the second imaging signal for each band is set so that an integrated value of the second imaging signal for each band is equal between bands. a plurality of optical filters having the spectral transmittance characteristics different from each other;
a switching unit that switches one optical filter disposed on the optical path among the plurality of optical filters;
comprising a control unit that controls the image sensor and the switching unit,
The control unit controls the switching unit to sequentially switch one of the optical filters disposed on the optical path in a time-division manner, and causes the image sensor to image a subject in a time-division manner, so that the image sensor outputting the first imaging signal for each of the plurality of time-shared images;
The conversion unit separates the first image signal output from the image sensor into image signals for each color filter for each of the plurality of time-divided images, and performs a matrix operation between the separated image signals. , converting the second imaging signal into a plurality of bands having imaging characteristics different from those of the color filter;
2. The multispectral image capturing device according to claim 1, wherein the number of bands of the second imaging signal is greater than the number of colors of the color filter. Equipped with multiple light sources with different emission spectra,
2. The light source according to claim 1, wherein the number of light sources that are caused to emit light simultaneously among the plurality of light sources is smaller than the plurality of light sources, and the light sources are caused to emit light in a time-sharing manner while controlling the light intensity of each light source during one shot imaging period. The multispectral image capturing device according to claim 2. The multispectral image capturing device according to any one of claims 1 to 3 ,
An inspection apparatus comprising: an inspection processing unit that inspects the subject based on the second imaging signal of a plurality of bands output when the multispectral image capturing device images the subject. A multispectral image capturing method using an image sensor having a color filter having a plurality of different transmittance characteristics on a pixel basis,
An optical filter having spectral transmittance characteristics having one or more blocking regions in the visible wavelength region and one or more transmitting regions in the non-visible wavelength region is connected to the image sensor and the subject. placed on the optical path between
an imaging step in which the image sensor images a subject through the optical filter;
The first imaging signal output from the image sensor is separated into imaging signals for each color of the color filter, and a matrix operation is performed between the separated imaging signals to generate a plurality of bands having imaging characteristics different from those of the color filter. a conversion step of converting into a second imaging signal ;
an image processing step of performing image processing to extract a specific region in an image based on the second imaging signal of multiple bands obtained by imaging a reference subject given in advance;
At least one band of the plurality of bands of the second imaging signal has sensitivity in a non-visible light wavelength region,
The amplification factor of the second imaging signal for each band is set so that the integrated value of the second imaging signal for each band in all regions other than the specific region in the image is equal between bands. A multispectral image capturing method characterized by: