JP2009031226A - Image analysis apparatus and image generation apparatus - Google Patents

Abstract

Kind Code: A1 A light transmission and diffusion property of a semi-transparent object having an unknown shape is estimated with a simple device without depending on a background color.
An image analysis apparatus includes a first translucent object image, a second translucent object image having a different background brightness, and a third translucent object image having a background brightness of substantially zero. An image data buffer 21, a transmittance / color estimation unit 11 for estimating the transmittance of the translucent object based on the first translucent object image and the second translucent object image, and the third semitransparent object image. Based on the transparent object image, a diffuse reflectance estimating unit 13 that estimates the diffuse reflectance of the translucent object, and a transmittance that separates the transmissivity of the translucent object into a regular transmittance and a diffuse transmittance of the translucent object And a separation unit 14.
[Selection] Figure 1

Description

  The present invention relates to an image processing technique for expressing a translucent object more realistically.

  Conventionally, in computer graphics technology, the transmittance has been determined based on the shape of the object and the optical characteristics of the object, or has been determined based on empirical rules.

  Recently, a method for measuring the transmittance of each pixel from an object with unknown optical characteristics using a known color or a known pattern as a background has been proposed (for example, Non-Patent Documents 1 and 2).

A method for measuring optical characteristics using laser light has also been proposed (for example, Patent Document 1).
Japanese Patent Laid-Open No. 7-104458 Naoko Kazama, “Measurement of optical properties of transparent objects for CG and its representation method”, Master's thesis, Nara Institute of Science and Technology, NAIST-IS-MT9751030 NTT Life Environment Research Laboratories, “Developing a system that automatically generates 3D CG data for glossy and translucent objects,” NTT Research Results Annual Report, 2001

  Here, when a known background color is used as in Non-Patent Document 1, if the object to be measured and the background color have the same hue, the location may be erroneously recognized as being translucent. Also, the color of the object itself changes depending on the background color, and it is difficult to completely eliminate this erroneous recognition.

  On the other hand, when a known pattern is used as in Non-Patent Document 2, ray tracing is performed with the shape of the object as known, so that an error in the shape causes an error in ray tracing and may greatly affect the measurement result.

  Further, in the case of a method using laser light as in Patent Document 1, it is difficult to incorporate the apparatus into a three-dimensional image measurement apparatus because the apparatus becomes large-scale.

  Accordingly, an object of the present invention is to estimate the properties of light transmission and diffusion of a translucent object having an unknown shape with a simple device without depending on the background color.

  Another object of the present invention is to express the translucent object more realistically by using the property relating to transmission estimated from an image obtained by capturing the translucent object.

  An image processing apparatus according to an embodiment of the present invention provides a first translucent object image obtained by photographing a translucent object using a first background, and a second lightness different from that of the first background. A second translucent object image obtained by photographing the translucent object using the background of the first background, and a third background having a brightness different from that of the first and second backgrounds and substantially zero in brightness. Means for storing a third translucent object image obtained by photographing the translucent object, a pixel value of the translucent object region in the first translucent object image, and the second semitransparent object image. Based on the pixel value of the translucent object area in the transparent object image, the transmissivity estimating means for estimating the transmissivity of the translucent object for each pixel, and the translucent object area in the third translucent object image The diffuse reflectance of the translucent object is estimated for each pixel based on the pixel value of Based on the diffuse reflectance estimation means and the estimated transmissivity and diffuse reflectance of the translucent object for each pixel, regular transmissivity for estimating the transmissivity and diffuse transmissivity of the translucent object for each pixel Rate and diffuse transmittance estimating means, and means for storing each data of the estimated regular transmittance, diffuse transmittance, and diffuse reflectance for each pixel.

  In a preferred embodiment, the apparatus further comprises means for storing a first background image obtained by photographing the first background and a second background image obtained by photographing the second background, The transmittance estimation means uses the first background image to remove the influence of the first background from the first translucent object image, and uses the second background image to perform the second operation. The transmissivity of the translucent object may be estimated for each pixel based on an image obtained by removing the influence of the second background from the translucent object image.

In a preferred embodiment, the transmittance estimating means sets the pixel value of the processing target pixel of the first translucent object image to B ₁ and each pixel of the pixel corresponding to the processing target pixel of the second translucent object image. When the value is B ₂ , each pixel value of the first background image is B _Q1 , and the pixel value of the pixel corresponding to the processing target pixel of the second background image is B _Q2 , For the transparent object region, it is also possible to estimate the transmittance α for each pixel and for each of red, green, and blue.
B ₁ = (1−α) B _P + αB _Q1
B ₂ = (1−α) B _P + αB _Q2
B _P : Eigenvalue of pixel to be processed of translucent object

In a preferred embodiment, the transmittance estimation means may further estimate the eigenvalue _BP for each pixel in the region of the translucent object.

In a preferred embodiment, specular transmittance and diffuse transmittance estimating means, transmittance alpha, specular transmittance i, diffuse transmittance k _t, when the diffuse reflectance k _r,
If α> _{k r,} it estimates that _{k t = k r, i =} α-k t,
If α ≦ k _r, it may be estimated that k _t = α _and i = 0.

  An image processing apparatus according to an embodiment of the present invention includes a translucent object and a background object located behind the translucent object, and an overlapping region in which at least a part of the translucent object overlaps the background object. An apparatus for generating a composite image having: means for storing the three-dimensional shape data of the semi-transparent object and the three-dimensional shape data of the background object; the semi-transparent object and the background object designated in advance; Means for calculating the relative distance between the semi-transparent object and the background object in the superimposition region based on the relative positional relationship between the semi-transparent object and the semi-transparent object image based on the three-dimensional shape data of the semi-transparent object. Estimated from a first rendering means for generating image, a second rendering means for generating a background object image based on the three-dimensional shape data of the background object, and an image obtained by photographing the translucent object Means for storing data indicating the transmissivity of the translucent object, and means for generating the composite image based on the translucent object image and the background object image, the transmissivity of the translucent object The composition ratio of the semi-transparent object image is determined based on the image, and the image of the background object in the superimposed region in the composite image is blurred according to the relative distance between the semi-transparent object and the background object. Image generating means for generating an image.

  In a preferred embodiment, the image generation means generates a blurred image of the background object image of the superimposed region according to a relative distance between the translucent object and the background object, and transmits the translucent object. An overlap region of the composite image may be generated based on the translucent object image in which the composite ratio is determined based on the rate and the blurred image.

  In a preferred embodiment, the apparatus further comprises means for storing data indicating the regular transmittance and the diffuse transmittance of the translucent object, and the image generating means calculates the composition ratio of the background object image based on the regular transmittance. Determining a composition ratio of the blurred image based on the diffuse transmittance, a translucent object image in which the composition ratio is determined based on the transmittance of the semitransparent object, and a composition ratio based on the regular transmittance. An overlap region of the composite image may be generated based on the determined background object image and the blurred image in which the composite ratio is determined based on the diffuse transmittance.

  An image processing method according to an embodiment of the present invention includes a first translucent object image obtained by photographing a translucent object using the first background, and a second lightness different from that of the first background. A second translucent object image obtained by photographing the translucent object using the background of the first background, and a third background having a brightness different from that of the first and second backgrounds and substantially zero in brightness. A process of storing a third translucent object image obtained by photographing the translucent object using the storage means, a pixel value of the translucent object region in the first translucent object image, and the first Processing for estimating the transmissivity of the translucent object for each predetermined area based on the pixel value of the translucent object area in the second translucent object image, and the translucent in the third translucent object image Based on the pixel value of the object region, the diffuse reflectance of the translucent object is determined as the predetermined value. Based on the processing to estimate for each area, and the estimated transmissivity and diffuse reflectance of the predetermined area of the semitransparent object, the regular transmittance and diffuse transmittance of the translucent object A process for estimating each area is performed.

  A method for generating a composite image according to an embodiment of the present invention includes a translucent object and a background object located behind the translucent object, and at least a part of the translucent object overlaps the background object. A method for generating a composite image having a region, wherein the semi-transparent object three-dimensional shape data and the background object three-dimensional shape data are stored in a storage means, and the semi-transparent specified in advance Based on the relative positional relationship between the object and the background object, based on the process of calculating the relative distance between the translucent object and the background object in the overlap region, and based on the three-dimensional shape data of the translucent object A process of generating a translucent object image, a process of generating a background object image based on the three-dimensional shape data of the background object, and the translucent object estimated from an image obtained by photographing the translucent object Storing the data indicating the transmittance of the image in the storage means, and means for generating the composite image based on the translucent object image and the background object image, and based on the transmissivity of the translucent object The composition ratio of the semi-transparent object image is determined, and the image of the background object in the superimposed area in the composite image is blurred according to the relative distance between the semi-transparent object and the background object to generate the composite image. And processing.

  Hereinafter, an image processing apparatus according to an embodiment of the present invention will be described with reference to the drawings.

  The image processing apparatus according to the present embodiment includes an image analysis apparatus 1 and an image capturing apparatus 3 whose configurations are shown in FIG. 1, and an image generation apparatus 7 whose configuration is shown in FIG.

  The image analysis apparatus 1 and the image generation apparatus 7 are both configured by a predetermined computer system. The components or functions of the image analysis apparatus 1 and the image generation apparatus 7 described below are, for example, predetermined hardware and software ( It can be realized by a computer program.

  FIG. 1 shows a configuration of an image analysis device 1 and an image photographing device 3 according to the present embodiment.

  The image capturing apparatus 3 includes a ceiling illumination unit 31, an image sensor 32, a back illumination unit 33, a filter table 35, and a sample table 37.

  The ceiling illumination unit 31 includes fluorescent lamps 31a and 31b and an illumination diffusion plate 31c. The ceiling illumination unit 31 illuminates the translucent object 5 that is a sample disposed on the sample table 37 within the field of view of the image sensor 32.

  The back illumination unit 33 illuminates the filters 35 a to 35 c on the filter table 35 located on the optical axis of the image sensor 32 from the back side. Each of the filters 35a to 35c serves as a background of an image captured by the image sensor 32.

  FIG. 2 shows an example of the filter table 35. In the filter table 35, a plurality of (three in the figure) different density filters are arranged on a circular table. In this embodiment, a filter 35a having a density of 0% (all the backlight is transmitted), a filter 35b having a density of 1% to 99% (a part of the backlight is partially blocked), and a filter 35c having a density of 100% (all the backlight is blocked). Is installed. That is, when the filters 35a to 35c are used as the background, the brightness of the background is different. In particular, the lightness of the filter 35c is substantially zero. Hereinafter, the case where the filter 35a is used is called “white background”, the case where the filter 35b is used is called “gray background”, and the case where the filter 35c is used is called “black background”.

  The filter 35a having a concentration of 0% may be a cavity having no filter. Further, the filter 35b may have a density of 50%, for example.

  FIG. 3 shows an example of the sample table 37. The sample table 37 includes a sample region 37a in which a semitransparent object 5 having a semitransparent characteristic as a sample to be measured is placed on a circular table, and a standard diffuse reflector (gray card) 37b having a known diffuse reflectance. And a hollow region 37c.

  In the image capturing apparatus 3, the centers of the filter table 35 and the sample table 37 are installed on a common axis. The filter table 35 and the sample table 37 are rotatable about their central axes. The positions of the filter table 35 and the sample table 37 may be determined manually or may be controlled by a control device (not shown).

In this embodiment, the positions of the filter table 35 and the sample table 37 are determined, and the following images are taken. The captured image data is stored in the image data buffer 21.
Image 1: White background sample image (sample region 37a + filter 35a)
Image 2: Gray background sample image (sample region 37a + filter 35b)
Image 3: White background image (cavity region 37c + filter 35a)
Image 4: Gray background image (cavity region 37c + filter 35b)
Image 5: Black background image (cavity region 37c + filter 35c)
Image 6: Standard diffuse image (standard diffuse reflector 37b)

  Note that when these images are taken, the conditions for the ceiling illumination and the back illumination are all the same. Further, in the image 1 and the image 2, the semi-transparent object 5 is photographed with the same position. As a result, the same part of the translucent object 5 appears in different backgrounds in the pixels at the same positions in the image 1 and the image 2. Further, the difference between the image 1 and the image 3 is only whether a sample is present, and the background image is common. Similarly, the difference between the image 2 and the image 4 is only the presence of the sample, and the background image is common.

  Therefore, in each process described below, by comparing pixels at the same position in each image, it is possible to estimate the transmissivity and the like of a translucent object at a position corresponding to each pixel.

  The image analysis apparatus 1 will be described with reference to FIG. 1 again.

  The image analysis device 1 includes a transmittance / color estimation unit 11, a ceiling illumination estimation unit 12, a diffuse reflectance estimation unit 13, a transmittance separation processing unit 14, an image data buffer 21, and a color data storage unit 22. , A transmittance data storage unit 23, a ceiling illumination data storage unit 24, a diffuse reflectance data storage unit 25, a regular transmittance data storage unit 26, and a diffuse transmittance data storage unit 27.

  The image data buffer 21 stores image data photographed by the image photographing device 3. In the present embodiment, the image data of each of the images 1 to 6 described above is stored in the image data buffer 21.

  The transmittance / color estimation unit 11 estimates the transmittance and color of the translucent object 5 based on the image data stored in the image data buffer 21. The color and transmittance estimated by the transmittance / color estimation unit 11 are stored in the color data storage unit 22 and the transmittance data storage unit 23, respectively.

The transmittance / color estimation unit 11 estimates the transmittance α and the pixel value _BP unique to the translucent object 5 for each of R (red), G (green), and B (blue), for example. The RGB-specific transmittance α is stored in the transmittance data storage unit 23, and the pixel value _BP unique to the translucent object 5 is stored in the color data storage unit 22.

First, the RGB pixel value B of each image data stored in the image data buffer 21 is expressed by the following equation.
B = (1-α) B _P + αB _Q
B: Pixel value α: Transmittance B _P : Pixel value peculiar to semi-transparent object B _Q : Pixel value of background image

Here, the RGB pixel value B ₁ of the white background sample image (image 1) is expressed by equation (1).
B ₁ = (1−α) B _P + αB _Q1 (1)

Further, RGB specific pixel value B ₃ of white background image (image 3), since it was taken a white background image directly represented by the formula (2).
B ₃ = B _Q1 (2)

Similarly, RGB pixel values B ₂ and B ₄ of the white background sample image (image 2) and the gray background image (image 4) are expressed by equations (3) and (4), respectively.
B ₂ = (1−α) B _P + αB _Q2 (3)
B ₄ = B _Q2 (4)

The equation (1) to (4), the RGB-specific transmittance alpha, RGB separate translucent objects specific pixel value _{B P} is obtained for each pixel.

  The ceiling illumination estimation unit 12 estimates the influence of the ceiling illumination light on the image based on the image data of the standard diffusion image (image 6) stored in the image data buffer 21. Ceiling illumination data indicating the effect of the estimated ceiling illumination light on the image is stored in the ceiling illumination data storage unit 24.

Standard diffusion images (image 6), since the transmitted light does not exist, the RGB specific pixel value B ₆ is expressed by Equation (5).
B ₆ = k _N B _U (5)
B _U : Ceiling lighting data k _N : Diffuse reflectance of standard diffuser (known)

The equation (5), RGB separate ceiling illumination data _{B U} is determined for each pixel. The ceiling illumination data _BU is stored in the ceiling illumination data storage unit 24.

Diffuse reflectance estimation unit 13, based on the image data stored in the ceiling illumination data and image data buffer 21, estimates the diffuse reflectance k _r of the translucent objects 5. The estimated diffuse reflectance _kr is stored in the diffuse reflectance data storage unit 25.

Since a black background image (image 5) the transmitted light does not exist, the RGB specific pixel value B ₅ is represented by the formula (6).
B ₅ = k _r B _U (6)
k _r : diffuse reflectance

Therefore, from equation ceiling illumination data and equations obtained from (5) (6), RGB separate diffuse reflectance k _r is determined for each pixel. The RGB separate diffuse reflectance k _r of each pixel is stored in the diffuse reflectance data storage 25.

Transmittance separation processing section 14, based on the transmittance data stored in the transmission data storage unit 23, separates the RGB separate transmittance α in specular transmittance i and diffuse transmittance k _t for each pixel. For each separated pixel, the regular transmittance i and diffuse transmittance k _t for each RGB are stored in the regular transmittance data storage unit 26 and the diffuse transmittance data storage unit 27, respectively.

Here, the transmittance α, the regular transmittance i, and the diffuse transmittance _kt will be described with reference to FIG.

  As shown in FIG. 4A, in the case of a translucent object, part of the light incident from the back surface is transmitted to the front surface side. The ratio of the transmitted light intensity to the incident light intensity at this time is the transmittance α.

Transmission includes normal transmission in which incident light is transmitted without changing the direction and diffuse transmission in which incident light is transmitted while diffusing. Here, the transmittance alpha, specular transmittance i and diffuse transmittance k _t is generally satisfy the relationship of equation (7).
α = i + k _t (7)
Further, as shown in FIG. 4B, the light incident on the translucent object from the front is reflected while being diffused on the surface of the translucent object. The intensity of the diffuse reflected light to the intensity of the incident light is diffuse reflectance k _r.

Based on the above relation, in this embodiment, it separates the RGB separate transmittance α For transmittance separation processing unit 14 as follows each pixel specular transmittance i and diffuse transmittance k _t.

First, if the transmittance α is larger than the diffusion reflectance k _r (e.g. translucent objects such as when very thin), the transmitted light can be estimated and is as shown in Figure 5A. That is, diffuse transmission is small compared to relatively large regular transmission. Therefore, diffuse transmittance k _t is considered to be equal to the diffuse reflectance k _r. As a result, the transmittance α, the regular transmittance i, and the diffuse transmittance _kt are expressed by equations (8) and (9).
k _t = k _r (8)
i = α−k _t (9)
Where α> _kr

On the other hand, if the transmittance α is less diffuse reflectance k _r (for example, if a semi-transparent object is relatively thick, etc.), the transmitted light can be inferred that is as shown in Figure 5B. In other words, the regular transmission is extremely small and can be ignored, and all the transmitted light is considered to be diffusely transmitted. As a result, the transmittance α, the regular transmittance i, and the diffuse transmittance _kt are expressed by equations (10) and (11).
k _t = α (10)
i = 0 (11)
However, α ≦ k _r

  According to the image analysis apparatus 1 and the image capturing apparatus 3 according to the above-described embodiment, the properties relating to the transmission and diffusion of light of a translucent object having an unknown shape can be obtained with a simple apparatus without depending on the background color. Can be estimated.

  Next, FIG. 6 shows a configuration of the image generation apparatus 7 according to the present embodiment. The image generation apparatus 7 shown in the figure uses the transmissivity of the translucent object photographed by the above-described image photographing apparatus 3 and processed by the image analysis apparatus 1, and the like of the translucent object and a predetermined background object. A composite image is generated by superimposing the image. However, the image generation device 7 can also generate a composite image using the transmittance estimated using a device other than the image capturing device 3 and the image analysis device 1. At this time, the transmittance estimated from an image obtained by actually capturing a translucent object may be used.

  Further, when generating a composite image, the relative positional relationship between the translucent object and the background object is designated in advance. In this embodiment, a case where a translucent object is positioned in front of a background object and at least a part of both overlaps is targeted. In other words, if there is a region where the background object and the semi-transparent object overlap and a region where they do not overlap, the rendered image of the background object and the rendered image of the translucent object are respectively displayed for the non-overlapping region. Generate separately. For the region where the background object and the semi-transparent object overlap, the image generation device 7 synthesizes the image as described below.

  The image generation device 7 includes a background rendering unit 71, a translucent object rendering unit 72, a difference image generation unit 73, a blurred image generation unit 74, and an image composition unit 75. Further, the image generation device 7 has a background object shape data storage unit 81, a background object color data storage unit 82, a translucent object shape data storage unit 83, and a translucent object as storage units for storing data. Color data storage unit 22, transmission data storage unit 23, regular transmission data storage unit 26, diffuse transmission data storage unit 27, background object rendering image storage unit 84, and background object distance image storage Unit 85, translucent object distance image storage unit 86, translucent object rendering image storage unit 87, distance difference image storage unit 88, background object blurred image storage unit 89, and composite image storage unit 90 With.

  The shape data storage unit 81 of the background object stores, for example, shape data of a three-dimensional model of the background object expressed using polygons. The shape data of the three-dimensional model may be generated by measuring a background object using, for example, a three-dimensional scanner.

  The background object color data storage unit 82 stores, for example, the color data of each polygon of the shape data of the three-dimensional model stored in the shape data storage unit 81 of the background object.

  The translucent object shape data storage unit 83 stores shape data of a three-dimensional model expressed using polygons of the translucent object 5. The shape data of the three-dimensional model of the translucent object 5 may be measured in advance using the image sensor 32 of the image capturing device 3. Each polygon is associated with color data in the color data storage unit 22 of the translucent object in advance.

  The translucent object color data storage unit 22, the transmittance data storage unit 23, the regular transmittance data storage unit 26, and the diffuse transmittance data storage unit 27 are respectively a color data storage unit 22 and a transmittance of the image analysis apparatus 1. The data storage unit 23, the regular transmittance data storage unit 26, and the diffuse transmittance data storage unit 27.

  The background rendering unit 71 performs background object rendering processing based on the shape data and color data stored in the background object shape data storage unit 81 and the background object color data storage unit 82, respectively. For example, the background rendering unit 71 generates a rendering image of a background object arranged at a predetermined position with respect to a predetermined viewpoint. The generated rendered image of the background object is stored in the rendered image storage unit 84 of the background object. At the same time, the background rendering unit 71 generates a distance image of the background object when viewed from the same viewpoint, and stores it in the distance image storage unit 85 of the background object.

  The translucent object rendering unit 72 performs rendering processing of the translucent object 5 based on the shape data and color data stored in the translucent object shape data storage unit 83 and the translucent object color data storage unit 22, respectively. Do. For example, the translucent object rendering unit 72 generates a rendered image of the translucent object 5 arranged at a predetermined position with respect to the same viewpoint used for rendering the background object. The generated rendered image of the translucent object 5 is stored in the rendered image storage unit 87 of the translucent object. The translucent object rendering unit 72 further generates a distance image of the translucent object 5 when viewed from the same viewpoint, and stores it in the distance image storage unit 86 of the translucent object.

  The difference image generation unit 73 refers to the distance image storage unit 85 of the background object and the distance image storage unit 86 of the translucent object, and generates a difference image of the distance indicating the relative distance between the background object and the translucent object 5. And stored in the distance difference image storage unit 88. Here, since the positions of the background object and the translucent object are respectively determined in advance from a predetermined viewpoint, the relative positions of the background object and the translucent object are also determined in advance.

  The blur image generation unit 74 refers to the background object rendering image storage unit 84 and the distance difference image storage unit 88 to blur the image of the background object in the region where the background object and the translucent object 5 overlap. A blurred image is generated and stored in the blurred image storage unit 89 of the background object.

  The process of the blurred image generation unit 74 will be described with reference to FIG. First, the blurred image generation unit 74 prepares a predetermined blur filter (hereinafter simply referred to as “filter”) 740. In the example of the figure, the size of the blur filter 740 is 7 × 7 pixels, but the filter size is not limited to this.

  The blurred image generation unit 74 filters the background object rendering image 840 to generate a blurred image 890 of the background object. Here, coefficients of 0 to 1 are determined for each element of the filter 740. This coefficient varies in the manner described later. Accordingly, the blurred image generation unit 74 applies the filter 740 to the rendered image 840 so that the target pixel 841 of the rendered image 840 of the background object matches the central element 741 of the filter 740. Then, the pixel value of each pixel of the rendering image 840 covered by the filter 740 is multiplied by the coefficient of each element of the filter. The blurred image generation unit 74 sequentially performs this process on all the pixels of the rendering image 840, calculates the sum for each pixel of the value obtained by multiplying each pixel value and the filter coefficient, and blurs the background object. 890 is generated.

  Here, the coefficient of each element of the filter 740 is determined as follows. First, the coefficient of the center element 741 of the filter 740 is determined according to the value (relative distance d) of the corresponding pixel 881 of the difference image 880 corresponding to the target pixel 841. For example, as the relative distance d is smaller, the coefficient is closer to 1, and the degree of blur in the blurred image 890 is smaller. On the other hand, the greater the relative distance d, the closer the coefficient is to 0, and the degree of blur in the blurred image 890 increases.

  Surrounding elements other than the central element 741 of the filter 740 are determined according to the coefficient of the central element 741. For example, the coefficient of the center element 741 may be the maximum, and the coefficient may decrease as the distance from the center element 741 increases.

  For example, FIG. 8 is a diagram illustrating a coefficient distribution of the blur filter 740. That is, this figure shows the relationship between the coefficient and the distance from the central element 741. For example, when the relative distance d is smaller than the smallest first threshold (relative distance d: small), each coefficient may be determined as shown in FIG. 8A. Similarly, when the relative distance d is between the first threshold value and the second threshold value (second threshold value> first threshold value) (relative distance d: medium), as shown in FIG. May be determined. Further, when the distance difference is larger than the second threshold (relative distance d: large), each coefficient may be determined as shown in FIG. 8C.

  This is a composite image generated by the image generation device 7 that the background blur looks different depending on the relative distance between the translucent object and the background object when the translucent object is positioned in front of the background object. This is to reproduce. That is, when the distance between the translucent object and the background object is short, the degree of blur of the background object is small, and as the distance between the translucent object and the background object is large, the degree of blur of the background object becomes large. The blurred image generation unit 74 generates a blurred image 890 so as to be blurred according to the relative distance between the translucent object 5 and the background object.

  The image composition unit 75 refers to the rendering image storage unit 84 of the background object, the rendering image storage unit 87 of the translucent object, and the blurred image storage unit 89 of the background object, so that the semitransparent object 5 and the background object overlap each other. A composite image of the area is generated and stored in the composite image storage unit 90.

When performing image composition, the image composition unit 75 corrects the rendering image of the translucent object in the rendering image storage unit 87 of the translucent object using the transmittance α. That is, the image composition unit 75 multiplies (1-α) for each pixel of the rendering image of the translucent object using the corresponding transmittance α. Also, the image composition unit 75 corrects the rendered image of the background object in the background object rendered image storage unit 84 using the regular transmittance i. That is, the image composition unit 75 multiplies each pixel of the rendered image of the background object by the corresponding regular transmittance i. Further, the image synthesizing unit 75 is corrected using a diffuse transmittance k _t a blurred image of the background object blurred image storage unit 89 of the background objects. That is, the image synthesizing unit 75, for each pixel of the blurred image of the background object, multiplying corresponding diffuse transmittance k _t.

For each pixel, the image composition unit 75 renders the translucent object rendering image, the background object rendering image, and the background object blurring that have been corrected by the transmittance α, the regular transmittance i, and the diffuse transmittance k _t as described above. Each pixel value of the image is added to generate a composite image.

Incidentally, the image synthesizing unit 75 may generate the synthesized image without using the specular transmittance i and diffuse transmittance k _t. For example, the image composition unit 75 may generate a composite image using, for example, the transmittance α. That is, at this time, the image composition unit 75 determines the composition ratio of the translucent object image based on the transmittance α, and composes the background object image with the blurred image according to the relative distance d to generate a composite image. .

  Conventionally, the actual transmittance of the translucent object has not been taken into consideration, but according to the image generation device 7 according to the present embodiment, an image is generated using the transmittance estimated from the translucent object. Therefore, the translucency of the translucent object can be expressed more realistically.

  The above-described embodiments of the present invention are examples for explaining the present invention, and are not intended to limit the scope of the present invention only to those embodiments. Those skilled in the art can implement the present invention in various other modes without departing from the gist of the present invention.

1 shows a configuration of an image analysis device 1 and an image photographing device 3 according to an embodiment of the present invention. An example of the filter table 35 is shown. An example of the sample table 37 is shown. Transmittance alpha, is an illustration of a specular transmittance i and diffuse transmittance k _t. It is explanatory drawing of the process which isolate _| separates the transmittance | permeability (alpha) into the regular transmittance i and the diffuse transmittance kt. 1 shows a configuration of an image generation apparatus 7 according to an embodiment of the present invention. 7 is an explanatory diagram of processing of a blurred image generation unit 74. FIG. It is a figure which shows the coefficient distribution of a blur filter.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Image analysis apparatus 3 Image imaging device 5 Translucent object 7 Image generation apparatus 11 Transmittance / color estimation part 12 Ceiling illumination estimation part 13 Diffuse reflectance estimation part 14 Transmittance separation processing part 71 Background rendering part 72 Translucent object rendering part 73 Difference image generation unit 74 Image generation unit 75 Image composition unit

Claims (10)

Photographing the translucent object using a first translucent object image obtained by photographing the translucent object using the first background and a second background having a brightness different from that of the first background The obtained second translucent object image is obtained by photographing the translucent object using a third background having a lightness different from that of the first and second backgrounds and having substantially zero lightness. Means for storing three translucent object images;
Based on the pixel value of the semi-transparent object area in the first semi-transparent object image and the pixel value of the semi-transparent object area in the second semi-transparent object image, the transmittance of the semi-transparent object is defined as a pixel. A transmittance estimation means for estimating each,
Diffuse reflectance estimation means for estimating a diffuse reflectance of the translucent object for each pixel based on a pixel value of the translucent object region in the third translucent object image;
Based on the estimated transmissivity and diffuse reflectance for each pixel of the semi-transparent object, normal transmittance and diffuse transmittance estimation means for estimating the regular transmittance and diffuse transmittance of the semi-transparent object for each pixel. When,
Means for storing each data of the estimated regular transmittance, diffuse transmittance, and diffuse reflectance of each pixel. Means for storing a first background image obtained by photographing the first background and a second background image obtained by photographing the second background;
The transmittance estimating means uses the first background image to remove the influence of the first background from the first translucent object image, and uses the second background image to perform the second. The image processing apparatus according to claim 1, wherein the transmissivity of the translucent object is estimated for each pixel based on an image obtained by removing the influence of the second background from the translucent object image. The transmittance estimating means sets the pixel value of the processing target pixel of the first translucent object image to B ₁ , and sets the pixel value of the pixel corresponding to the processing target pixel of the second translucent object image to B ₂ , When each pixel value of the first background image is B _Q1 and the pixel value of the pixel corresponding to the processing target pixel of the second background image is B _Q2 , the region of the translucent object is expressed by the following equation: The image processing apparatus according to claim 2, wherein the transmittance α for each pixel and for each of red, green, and blue is estimated.
B ₁ = (1−α) B _P + αB _Q1
B ₂ = (1−α) B _P + αB _Q2
B _P : Eigenvalue of pixel to be processed of translucent object The image processing apparatus according to claim 3, wherein the transmittance estimation unit further estimates the eigenvalue _BP for each pixel in the region of the translucent object. Specular transmittance and diffuse transmittance estimating means, transmittance alpha, specular transmittance i, diffuse transmittance k _t, when the diffuse reflectance k _r,
If α> _{k r,} it estimates that _{k t = k r, i =} α-k t,
The image processing apparatus according to claim 1, wherein if α ≦ k _r , k _t = α _and i = 0 are estimated. An apparatus for generating a composite image including a translucent object and a background object located behind the translucent object, wherein at least a part of the translucent object overlaps the background object,
Means for storing the three-dimensional shape data of the translucent object and the three-dimensional shape data of the background object;
Means for calculating a relative distance between the translucent object and the background object in the superimposed region based on a relative positional relationship between the translucent object and the background object specified in advance;
First rendering means for generating a translucent object image based on the three-dimensional shape data of the translucent object;
Second rendering means for generating a background object image based on the three-dimensional shape data of the background object;
Means for storing data indicating the transmissivity of the translucent object estimated from an image of the translucent object;
Means for generating the composite image based on the translucent object image and the background object image, and determining a composite ratio of the translucent object image based on the transmissivity of the translucent object; An image processing apparatus, comprising: an image generation unit configured to generate an image of the background object in the superimposition region with a degree of blur corresponding to a relative distance between the translucent object and the background object. The image generating means includes
The background object image of the superimposition region is generated according to a relative distance between the translucent object and the background object, and a blurred image is generated, and the synthesis ratio is determined based on the transmittance of the translucent object. The image processing apparatus according to claim 6, wherein a superimposed region of the composite image is generated based on a transparent object image and the blurred image. Means for storing data indicating the regular transmittance and diffuse transmittance of the translucent object;
The image generating means includes
Determine the composition ratio of the background object image based on the regular transmittance, determine the composition ratio of the blurred image based on the diffuse transmittance,
A translucent object image with a composition ratio determined based on the transmissivity of the translucent object, a background object image with a composition ratio determined based on the regular transmittance, and a composition ratio based on the diffuse transmittance The image processing apparatus according to claim 7, wherein a superimposed region of the composite image is generated based on the determined blurred image. A computer program for image processing,
When executed on a computer,
Photographing the translucent object using a first translucent object image obtained by photographing a translucent object using the first background and a second background having a brightness different from that of the first background The obtained second translucent object image is obtained by photographing the translucent object using a third background having a lightness different from that of the first and second backgrounds and having substantially zero lightness. A process of storing the three translucent object images in the storage means;
Based on the pixel value of the semi-transparent object region in the first semi-transparent object image and the pixel value of the semi-transparent object region in the second semi-transparent object image, the transmittance of the semi-transparent object is defined as a pixel. Processing to estimate for each,
A process for estimating the diffuse reflectance of the translucent object for each pixel based on the pixel value of the translucent object region in the third translucent object image;
Based on the estimated transmissivity and diffuse reflectance of each pixel of the translucent object, a process of estimating the regular transmissivity and diffuse transmittance of the translucent object for each pixel is performed. A computer program. A computer program for generating a composite image including a translucent object and a background object located behind the translucent object, wherein at least a part of the translucent object overlaps the background object. ,
When executed on a computer,
Processing for storing the three-dimensional shape data of the translucent object and the three-dimensional shape data of the background object in a storage means;
A process of calculating a relative distance between the translucent object and the background object in the superimposed region based on a relative positional relationship between the translucent object and the background object specified in advance.
A process of generating a translucent object image based on the three-dimensional shape data of the translucent object;
Processing for generating a background object image based on the three-dimensional shape data of the background object;
A process for storing data indicating the transmissivity of the translucent object estimated from an image obtained by capturing the translucent object in a storage unit;
Means for generating the composite image based on the translucent object image and the background object image, and determining a composite ratio of the translucent object image based on the transmissivity of the translucent object; A computer program characterized by performing processing for generating an image of a background object in the superimposition area with a degree of blur according to a relative distance between the translucent object and the background object.

Images