JP6903348B2 - Image processing equipment, image processing methods and programs - Google Patents

Description

The present invention relates to an image processing apparatus, an image processing method and a program for generating a rubbing image of an object.

Takumoto is a classic technique that uses ink and paper to copy the unevenness of a structure. Takumoto has been widely used in the investigation of cultural properties because the recorded results can be confirmed at the site of investigating cultural properties such as stone monuments, the risk of re-investigation is low, and images close to the actual size can be obtained. It's being used.

On the other hand, acquisition of a rubbing book requires proficiency, it takes about 15 to 30 minutes for each object, and contact with the object is a prerequisite, so there is concern about collapse and stains. Takumoto also has many drawbacks, such as not being applicable to materials and woodblock prints that dislike water. Therefore, in recent years, various techniques have been proposed to replace the conventional rubbing.

For example, a method of recording a three-dimensional shape without contacting an object such as a cultural property has been proposed by using a 3D scanner or SfM / MVS (Structure from Motion / Multi-view stereo) technology. Patent Document 1 discloses a method and an apparatus for measuring a three-dimensional model such as a stone tool with a three-dimensional measuring instrument and creating an actual measurement drawing of the three-dimensional model.

Japanese Unexamined Patent Publication No. 2005-251043

However, the method of recording a three-dimensional shape using a 3D scanner or SfM / MVS technology has the following drawbacks. For example, an expensive imaging device and a high-performance computer are required to acquire and analyze shape data of an object. In order to adjust the analysis parameters of SfM / MVS according to the object, the user needs specialized knowledge. It takes a relatively long time (for example, about 6 hours at present) for the analysis process by a computer, and it is difficult to confirm the analysis result at the site of investigating cultural properties such as stone monuments.

As a result of checking the recorded three-dimensional shape data, if the data is missing or the analysis result is unclear, a re-survey is required at the survey site. However, for example, in the case of surveys at overseas archaeological sites or a huge number of surveys, the opportunities for conducting surveys on-site are limited. Therefore, the method of recording a three-dimensional shape using a 3D scanner or SfM / MVS technology and the technology of Patent Document 1 are not suitable for fieldwork for acquiring a rubbing book such as a stone monument at a research site. There is a demand for a method for quickly and easily acquiring an image of unevenness formed on the surface of an object (hereinafter referred to as a rubbing image).

An object of the present invention is to provide an image processing device for quickly and easily generating a rubbing image of an object.

The present invention for achieving the above object includes, for example, the following aspects.
(Item 1)
A background image acquisition unit that acquires a background image including irregularities formed on the surface of an object,
An oblique light image acquisition unit that acquires an oblique light image in which at least a part of the surface is irradiated with oblique light,
A luminance change image generation unit that generates a luminance change image representing a luminance change due to irradiation of the light based on the oblique light image and the background image.
A smoothed image generation unit that generates a smoothed image by applying a smoothing process to the luminance change image,
An enhanced image generation unit that generates an enhanced image in which the unevenness is emphasized based on the smoothed image and the brightness change image.
An image processing device.
(Item 2)
As the smoothing process, the smoothed image generation unit applies any one or a combination of a smoothing filter, a median filter, and a Gaussian filter to the brightness change image to generate the smoothed image. , Item 1. The image processing apparatus according to Item 1.
(Item 3)
Item 2. The image processing apparatus according to Item 1 or 2, wherein the brightness change image generation unit generates the brightness change image by the difference between the oblique light image and the background image.
(Item 4)
Item 3. The image processing apparatus according to Item 3, wherein the enhanced image generation unit generates the enhanced image by the difference between the smoothed image and the brightness change image.
(Item 5)
The emphasized image generation unit
A low-luminance enhanced image is generated by the difference between the smoothed image and a portion of the brightness-changing image whose brightness is lower than that of the smoothed image.
A high-luminance enhanced image is generated by the difference between the portion of the brightness-changing image having a higher brightness than the smoothed image and the smoothed image.
Item 4. The image processing apparatus according to Item 4, wherein the low-luminance-enhanced image and the high-luminance-enhanced image are combined to generate the enhanced image.
(Item 6)
The oblique light image acquisition unit acquires a plurality of the oblique light images having different light irradiation regions, and obtains the oblique light images.
Item 2. The image processing apparatus according to any one of Items 1 to 5, further comprising a emphasized image synthesizing unit for synthesizing a plurality of the enhanced images created for each of the oblique light images having different irradiation regions.
(Item 7)
Item 6. The image processing apparatus according to Item 6, wherein the enhanced image synthesizing unit comparatively brightly synthesizes a plurality of the enhanced images based on the brightness of the unevenness in the enhanced image.
(Item 8)
A background image acquisition step for acquiring a background image including irregularities formed on the surface of an object, and
An oblique light image acquisition step of acquiring an oblique light image in which at least a part of the surface is irradiated with oblique light, and
A brightness change image generation step of generating a brightness change image representing a brightness change due to irradiation of the light based on the oblique light image and the background image.
A smoothing image generation step of applying a smoothing process to the brightness change image to generate a smoothed image,
A highlight image generation step of generating a emphasized image in which the unevenness is emphasized based on the smoothed image and the brightness change image, and
Image processing methods, including.
(Item 9)
A program for operating a computer as each part of the image processing apparatus according to items 1 to 7.

According to the present invention, it is possible to provide an image processing device for quickly and easily generating a rubbing image of an object.

It is a figure for demonstrating the usage mode of the image processing apparatus which concerns on one Embodiment of this invention. It is a block diagram for demonstrating the function of the image processing apparatus which concerns on one Embodiment of this invention. It is a flowchart for demonstrating the procedure which performs image processing using the image processing apparatus which concerns on one Embodiment of this invention. This is an example of the acquired background image. This is an example of the acquired oblique light image. This is an example of the generated luminance change image. This is an example of the generated smoothed image. It is a graph which shows the distribution of the luminance value along a certain lateral position of the luminance change image and the smoothed image. This is an example of the generated emphasized image. It is a figure for demonstrating the emphasized image. It is a figure for comparing the change of the character shape between images. This is an example of a composited rubbing image. It is a schematic diagram which shows the state which the surface of an uneven surface is irradiated with oblique light from a light source. It is a figure for demonstrating two kinds of enhanced images, a low-luminance enhanced image and a high-luminance enhanced image, which are generated in the image processing method according to another embodiment. It is a figure for comparing the method of this invention with other conventional methods in 1st Example. It is an image of an object and a rubbing image according to the second embodiment. It is an image of an object and a rubbing image according to a third embodiment. It is an image of an object and a rubbing image according to a fourth embodiment. It is a rubbing image which concerns on the 5th Example.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the following description and drawings, the same reference numerals indicate the same or similar components, and thus duplicate description of the same or similar components will be omitted.
[Outline of Invention]

FIG. 1 is a diagram for explaining a usage mode of the image processing apparatus according to the embodiment of the present invention.

The image processing device 1 according to one embodiment is a device that generates an image (rubbed image) of unevenness 8 formed on the surface of an object 9. The surface of the object 9 is imaged by using the external image pickup device 2, and the captured image is transferred to the image processing device 1 by a wireless connection such as Bluetooth (registered trademark) or a wired connection.

The image processing device 1 uses two types of captured images, a background image and an oblique light image, to generate an enhanced image in which the unevenness 8 is emphasized as a rubbing image. The generated rubbing image is displayed, for example, on the display unit 15 of the image processing device 1. The background image is an image including the unevenness 8 formed on the surface of the object 9. The oblique light image is an image in which at least a part of the surface of the object 9 is irradiated with oblique light. The oblique light is partially applied to the surface of the object 9 by using the light source 3. In the present embodiment, the surface of the object 9 is irradiated with oblique light by the user of the image processing apparatus 1.

The object 9 is preferably a structure made of a natural material, and the unevenness 8 is a character or pattern formed on the structure. In the present embodiment, the object 9 is a stone monument, and the image processing device 1 generates an emphasized image as a rubbing image of the characters engraved on the stone monument 9.

The external image pickup device 2 is composed of a device such as a commercially available digital camera including an image pickup element such as a CMOS image sensor or a CCD image sensor. In the present embodiment, the external imaging device 2 is equipped with an optical lens 4 having an appropriate focal length, and the external imaging device 2 images the entire object 9 with the imaging position fixed by the tripod 5. In another embodiment, the external imaging device 2 may be configured by a video camera or the like that captures a moving image, and a still image including the surface of the object 9 may be cut out from the captured moving image and used.

As the light source 3, for example, a commercially available lighting fixture such as LED lighting can be used. Preferably, the wavelength of the light source 3 is a wavelength in the visible light region. As the wavelength of the light source 3, various wavelengths can be used depending on the light receiving characteristics of the external imaging device 2.
[Device configuration]

FIG. 2 is a block diagram for explaining the function of the image processing apparatus according to the embodiment of the present invention.

The image processing device 1 according to one embodiment includes a data processing unit 12, an auxiliary storage device 13, an input unit 14, a display unit 15, and a communication interface unit (communication I / F unit) 16. In the present embodiment, the image processing device 1 is configured by using a tablet terminal or a smartphone (hereinafter, referred to as a tablet terminal or the like). In another embodiment, the image processing device 1 can include an image pickup unit 11 instead of the external image pickup device 2. For the image pickup unit 11, for example, an image pickup element such as a CMOS image sensor provided in advance in a tablet terminal or the like can be used. In still another embodiment, a general-purpose computer such as a notebook PC can be used for the image processing device 1.

In the present embodiment, the image processing device 1 includes an auxiliary storage device 13, an input unit 14, a display unit 15, and a communication I / F unit 16 as hardware configurations. Although not shown, the image processing device 1 further includes a processor such as a CPU that performs data processing and a memory that the processor uses for a work area of data processing as a hardware configuration.

The auxiliary storage device 13 is a non-volatile storage device that stores an operating system (OS), various control programs, data generated by the programs, and the like. For example, a flash memory, an eMMC (embedded Multi Media Card), or an SSD. (Solid State Drive) etc. In the present embodiment, the auxiliary storage device 13 stores the background image 31, the oblique light image 32, the brightness change image 33, the smoothed image 34, the enhanced image 35, and the image processing program P.

The background image 31 is an image including the unevenness 8 formed on the surface of the object 9. The oblique light image 32 is an image when at least a part of the surface of the object 9 is irradiated with oblique light. The brightness change image 33 is an image showing a brightness change due to irradiation of at least a part of the surface of the object 9 with oblique light. The brightness change image 33 is generated based on the oblique light image 32 and the background image 31. The smoothed image 34 is an image obtained by applying a smoothing process to the luminance change image 33. The emphasized image 35 is an image in which the unevenness 8 formed on the surface of the object 9 is emphasized. The enhanced image 35 is generated based on the smoothed image 34 and the luminance change image 33.

In the present embodiment, the background image 31 and the oblique light image 32 are stored as, for example, an 8-bit (total of 256 gradations having a brightness value of 0 to 255) grayscale image. The brightness change image 33, the smoothed image 34, and the enhanced image 35 are also generated as an 8-bit grayscale image, and the Takumoto image is also generated as an 8-bit grayscale image.

The image processing program P is a computer program for realizing each unit 21 to 26 in the data processing unit 12 described later, which is a functional block by software. The image processing program P may be installed in the image processing device 1 via a network such as the Internet connected by the communication I / F unit 16. Alternatively, the image processing program P is installed in the image processing device 1 by causing the image processing device 1 to read a computer-readable non-temporary tangible recording medium such as a memory card on which the image processing program P is recorded. May be good. The image processing program P can be an application such as a tablet terminal.

The input unit 14 can be composed of, for example, a mouse or a keyboard, and the display unit 15 can be composed of, for example, a liquid crystal display, an organic EL display, or the like. In the present embodiment, the input unit 14 and the display unit 15 are integrated as a touch panel.

The communication I / F unit 16 transmits / receives data to / from an external device such as an external image pickup device 2 via a wired or wireless network. The communication I / F unit 16 may be various wireless or wired connections such as Bluetooth®, Wi-Fi®, and ZigBee®. In another embodiment, when the imaging unit 11 is used instead of the external imaging device 2, the communication I / F unit 16 may be omitted.

In the present embodiment, the image processing device 1 includes a data processing unit 12 as a software configuration. The data processing unit 12 is a functional block realized by the processor executing the image processing program P.

The background image acquisition unit 21 acquires the background image 31 including the unevenness 8 formed on the surface of the object 9. The acquired background image 31 is stored in the auxiliary storage device 13. In the present embodiment, the background image 31 is acquired via the external image pickup device 2. When the acquired background image 31 is, for example, RGB 24-bit full-color and not a grayscale image, the background image acquisition unit 21 can convert the acquired background image 31 into, for example, an 8-bit grayscale image and store it.

The oblique light image acquisition unit 22 acquires an oblique light image 32 in which at least a part of the surface of the object 9 is irradiated with oblique light. The acquired oblique light image 32 is stored in the auxiliary storage device 13. In the present embodiment, the oblique light image 32 is acquired via the external image pickup device 2. When the acquired oblique light image 32 is, for example, RGB 24-bit full-color and not a grayscale image, the oblique light image acquisition unit 22 can convert the acquired oblique light image 32 into, for example, an 8-bit grayscale image and store it. The oblique light is partially irradiated on the surface of the object 9 by using the light source 3, and the oblique light image acquisition unit 22 receives light according to the region of light irradiated on the surface of the object 9 by the light source 3. A plurality of oblique light images 32 having different irradiation regions are acquired.

The brightness change image generation unit 23 generates a brightness change image 33 showing a change in brightness due to irradiation of light based on the oblique light image 32 and the background image 31. The generated luminance change image 33 is stored in the auxiliary storage device 13. In the present embodiment, the brightness change image generation unit 23 generates the brightness change image 33 by the difference between the oblique light image 32 and the background image 31.

The smoothed image generation unit 24 applies a smoothing process to the luminance change image 33 to generate a smoothed image 34. The generated smoothed image 34 is stored in the auxiliary storage device 13. As a smoothing process, the smoothed image generation unit 24 applies any one or a combination of a smoothing filter, a median filter, and a Gaussian filter to the brightness change image 33 to generate a smoothed image 34. All three filters function as low-pass filters. In the present embodiment, the smoothed image generation unit 24 applies a smoothing filter to the luminance change image 33 to generate a smoothed image 34.

The enhanced image generation unit 25 generates an enhanced image 35 in which unevenness is emphasized, based on the smoothed image 34 and the brightness change image 33. The generated emphasized image 35 is stored in the auxiliary storage device 13. In the present embodiment, the enhanced image generation unit 25 generates the enhanced image 35 by the difference between the smoothed image 34 and the brightness change image 33.

The enhanced image synthesizing unit 26 synthesizes a plurality of enhanced images 35 created for each of the oblique light images 32 having different irradiation regions. In the present embodiment, the enhanced image synthesizing unit 26 comparatively brightly synthesizes a plurality of enhanced images 35 based on the brightness of the unevenness in the enhanced image 35. The emphasized image synthesizing unit 26 displays the combined emphasized image as a Takumoto image on, for example, a display unit 15. The enhanced image synthesizing unit 26 may store the synthesized enhanced image in, for example, an auxiliary storage device 13.
[Processing procedure]

FIG. 3 is a flowchart for explaining a procedure for performing image processing using the image processing apparatus according to the embodiment of the present invention.

In step S1 (background image acquisition step), the background image 31 including the unevenness 8 formed on the surface of the object 9 is acquired. Here, when the object 9 exists outdoors, for example, the natural light is substantially evenly applied to the entire object 9. An example of the acquired background image 31 is shown in FIG. In the present embodiment, the background image 31 is acquired only once at the beginning. In another embodiment, the background image 31 may be acquired each time the oblique light image 32 is acquired, and the latest one may be overwritten and used at all times.

In step S2, at least a part of the surface of the object 9 is irradiated with oblique light. In the present embodiment, the surface of the object 9 is irradiated with oblique light by the user of the image processing apparatus 1. The user uses the light source 3 to partially irradiate the surface of the object 9 with oblique light. The angle of the irradiation light with respect to the surface of the object 9 is preferably in the range of about 5 ° to 20 ° in the case of a substantially flat surface having a large radius of curvature. Here, the angle of the irradiation light is an angle formed by the surface of the object 9 and the optical axis of the irradiation light emitted from the light source 3. As the angle of the irradiation light increases and approaches 90 °, the irradiation light changes from oblique light to front light.

The angle of the irradiation light with respect to the surface of the object 9 does not have to be always constant. Even in the same irradiation region on the surface of the object 9, the surface of the object 9 can be irradiated with oblique light at different irradiation angles, whereby in step S3 described later, on the surface of the object 9. Even in the same irradiation region of, different oblique light images 32 can be acquired.

In step S3 (oblique image acquisition step), an oblique light image 32 in which at least a part of the surface of the object 9 is irradiated with oblique light is acquired. An example of the acquired oblique light image 32 is shown in FIG. It can be seen that the luminance value of the oblique light image 32 is generally increased as compared with the background image 31 by being irradiated with the oblique light. In the embodiment of the illustrated procedure, the light source 3 irradiates oblique light from the right side (upper side of FIG. 5) of the object 9. Diagonal light is partially illuminated on the surface of the object 9 by using the light source 3, and the oblique light image in which the light irradiation region differs depending on the region of light irradiated on the surface of the object 9 by the light source 3. 32 can be obtained.

In step S4 (luminance change image generation step), a luminance change image 33 representing a luminance change due to irradiation with light is generated based on the oblique light image 32 and the background image 31. FIG. 6 shows an example of the generated luminance change image 33.

In the present embodiment, the brightness change image 33 is generated by the difference between the oblique light image 32 and the background image 31. The image difference is a difference in the brightness value for each pixel (for example, a value in the range of 0 to 255 in the case of 8 bits), and for the pixel at the corresponding position in the oblique light image 32 and the background image 31, for example, the oblique light image 32. It is calculated by subtracting the brightness value of the background image 31 from the brightness value of. As a result, the change in the numerical value of the brightness value brightened by the oblique light irradiation is extracted from the brightness change image 33.

In step S5 (smoothing image generation step), a smoothing process is applied to the luminance change image 33 to generate a smoothed image 34. An example of the generated smoothed image 34 is shown in FIG. In the present embodiment, a smoothing filter is applied to the brightness change image 33 to generate a smoothed image 34. The smoothed image 34 is an image obtained by extracting the shape of the irradiated region by the oblique light irradiated on the surface of the object 9 by the light source 3.

The smoothed image 34 will be described with reference to FIG. FIG. 8 is a graph showing the distribution of luminance values along a certain lateral position of the image. (A) shows a graph 33g showing the distribution of the luminance values of the luminance change image 33, and (B) shows a graph 34g showing the distribution of the luminance values of the smoothed image 34. In addition, in (A), both the graph 33g and the graph 34g are shown for comparison.

As shown in the graph 33g of (A), in the luminance change image 33, the luminance value changes drastically for each position in the image, and a high frequency component and a low frequency component are mixed. The vertical amplitude of the graph 33g also changes drastically. In the embodiment of the illustrated procedure, the light source 3 irradiates oblique light from the right side (upper side of FIG. 7) of the object 9, and the brightness value also increases toward the right side of the graph 33g. The change in the brightness value reflects the change in the intensity of the reflected light due to the unevenness 8. For example, the brightness value is unlikely to increase at a position in the image corresponding to the shadow of the unevenness 8, and the brightness value increases at a position in the image where the unevenness 8 is directly irradiated by oblique light.

On the other hand, as shown in the graph 34g of (B), in the smoothed image 34, the change in the luminance value for each position in the image is smoothed by applying the smoothing process, and the smoothed image 34 is low. Only the frequency components are extracted. The change in the amplitude above and below the graph 34g is also gentle. As a result, the shape of the irradiation region due to the oblique light irradiated on the surface of the object 9 by the light source 3 is extracted from the smoothed image 34.

In step S6 (enhanced image generation step), an enhanced image 35 in which the unevenness 8 formed on the surface of the object 9 is emphasized is generated based on the smoothed image 34 and the brightness change image 33. An example of the generated emphasized image 35 is shown in FIG.

In the present embodiment, the enhanced image 35 is generated by the difference between the smoothed image 34 and the brightness change image 33. The image difference is calculated by, for example, subtracting the brightness value of the brightness change image 33 from the brightness value of the smoothed image 34 for the pixels at the positions corresponding to the smoothed image 34 and the brightness change image 33. As a result, the luminance value of the portion corresponding to the shadow of the unevenness 8 is extracted from the enhanced image 35.

The enhanced image 35 will be described with reference to FIG. FIG. 10 is a diagram for explaining the emphasized image. (A) and (B) are graphs showing the distribution of luminance values along a certain lateral position of an image. In (A), a graph 33g (black) showing the distribution of the brightness values of the brightness change image 33 and the brightness values of the smoothed image 34 with respect to the positions of the brightness change image 33 and the smoothed image 34 in the corresponding images. A graph showing the distribution, 34 g (gray), is shown. In (B), a graph 35 g (gray) showing the distribution of the luminance values of the emphasized image 35 is shown. (C) is an enlarged view of a part of the luminance change image 33, and the graph 33g shown in (A) is the distribution of the luminance values along the broken line LINE in (C). Similarly, (D) is an enlarged view of a part of the emphasized image 35, and the graph 35g shown in (B) is the distribution of the luminance values along the broken line LINE in (D). Reference numeral S shown in (A) to (D) is a portion on the broken line LINE corresponding to the main shadow of the unevenness 8.

With reference to (A), as shown in the graph 33g, the brightness value of the portion corresponding to the shadow S of the unevenness 8 is as low as less than about 20, and in the conventional technique, the portion corresponding to the shadow S of the unevenness 8 is formed. Extraction with a high contrast ratio compared to the surroundings has not been achieved. On the other hand, as shown in the graph 34g, in the smoothed image 34, the luminance value of the portion of the unevenness 8 corresponding to the shadow S is smoothed to the same luminance value as the surroundings. In the present invention, the enhanced image 35 is generated by subtracting the luminance change image 33 containing the high-frequency component, which is the basis for creating the smoothed image 34, from the smoothed image 34. As a result, the luminance value is as low as less than 20 in the graph 33g of the luminance change image 33, and the low luminance value buried in comparison with the graph 34g of the smoothed image 34 is found in the graph 35g of the enhanced image 35 after the subtraction process. It is inverted based on the brightness value of the smoothed image 34 and extracted as a high brightness value.

As shown in the graph 35g of (B), among the pixels of the brightness change image 33, the pixels having a brightness value higher than the brightness value of the smoothed image 34 are extracted as zero by the subtraction process. To. Further, since the brightness value of the pixels outside the irradiation region is approximately zero in the first place and remains approximately zero even after being smoothed by the smoothing process, even if the brightness change image 33 is subtracted from the smoothed image 34, the irradiation region The brightness value of the outer pixel is extracted while remaining substantially zero (that is, black).

In the luminance change image 33 shown in FIG. 6, the luminance value outside the irradiation region not irradiated with the light from the light source 3 and the luminance value of the portion corresponding to the shadow of the unevenness 8 in the irradiation region are both low. Therefore, when the brightness change image 33 is used to extract the pixels of the shadow portion of the unevenness 8 included in the irradiation region based on the brightness value, the pixels outside the irradiation region are also erroneously extracted.

On the other hand, in the present invention, the brightness change image 33 is subjected to a smoothing process to generate a smoothed image 34, and the brightness change image 33 is subtracted from the generated smoothed image 34 to obtain the brightness outside the irradiation region. It is possible to separate the value from the brightness value of the shadow portion of the unevenness 8 included in the irradiation area.

FIG. 11 is a diagram for comparing changes in glyphs between images. (A) is a background image 31, (B) is an oblique light image 32, (C) is a brightness change image 33, and (D) is an enhanced image 35. In each of the images (A) to (D), the part corresponding to the "Namu Amida Butsu" of the characters "Namu Amida Butsu" is shown. In the emphasized image 35 shown in (D), as compared with the images shown in (A) to (C), the portion corresponding to the shadow of the unevenness 8 is extracted with a higher contrast ratio than the surroundings, and the characters are displayed. The shape of is also clearly extracted.

In step S7, the processing is repeated from step S2 until the entire surface area of the object 9 desired by the user is processed (Yes in step S7), the irradiation area is changed, and a plurality of emphasized images 35 are generated one after another. ..

In step S8 (enhanced image composition step), a plurality of enhanced images 35 created for each of the oblique light images 32 having different irradiation regions are combined. In the present embodiment, a plurality of emphasized images 35 are comparatively brightly combined based on the brightness of the unevenness in the enhanced image 35. Comparative bright composition is a method of image composition, and is used, for example, when imaging the trajectory of a star. Various known methods can be used for comparative bright synthesis.

The combined plurality of emphasized images 35 are displayed, for example, on the display unit 15 as a Takumoto image of the object 9. An example of the synthesized rubbing image is shown in FIG. If the surface region of the object 9 desired by the user is only the first first region, the image composition process in this step is omitted, and the one emphasized image 35 generated in step S6 remains as it is. This is a rubbing image of the object 9.
[effect]

As described above, according to the image processing apparatus according to the embodiment of the present invention, it is possible to quickly and easily generate a rubbing image of an object. The user can easily and quickly generate a rubbing image of an object without requiring specialized knowledge.

According to the present invention, a rubbing image is created using an image obtained by capturing an object 9 using an external imaging device 2 or an imaging unit 11. As a result, unlike the conventional rubbing using ink and paper, it is possible to generate a rubbing image without touching the object, and even for materials that may collapse or become dirty, woodblock prints that dislike moisture, etc. You can create a rubbing image.

Further, according to the present invention, it is not necessary to prepare a dedicated expensive device, and a commercially available device can be used to easily and easily generate a rubbing image. Both the digital camera and the LED lighting, such as the tablet terminal exemplified in the above-described embodiment, are commercially available devices. The procedure for generating the Takumoto image is also simple and easy. For example, an interval timer is set in a digital camera, and images of the object 9 are acquired one after another at predetermined time intervals, and the object 9 is partially irradiated with oblique light by LED lighting or the like to obtain an irradiation area. The rubbing image can be generated by changing the above one after another.

Further, according to the present invention, a digital camera and LED lighting such as a tablet terminal constituting an image processing device have a portable size and weight, and the time required to create a rubbing image is short, so that a stone monument or the like can be used. Recorded results can be quickly confirmed at the site of investigating cultural properties. As a result of checking the recording result, even if the rubbing image has a missing part or an unclear part, an additional emphasized image can be immediately generated for the missing part or the unclear part at the site. This also reduces the risk of re-investigation.

The present invention is also suitable for fieldwork of cultural property research. For example, by using an image pickup element provided in advance in a tablet terminal or the like instead of a digital camera, the image processing device has a simpler configuration. For example, by installing the program (smartphone application) of the present invention on a tablet terminal or the like, the tablet terminal or the like owned by the participants of the fieldwork can be used as the image processing device of the present invention. As a result, fieldwork participants can quickly and easily generate and record Takumoto images such as stone monuments scattered in the participants' hometowns without requiring specialized knowledge. In addition, for example, it is possible to easily create a digital database of local history that collects images of Takumoto such as stone monuments existing in the local area.
[Other forms]

Although the present invention has been described above by the specific embodiment, the present invention is not limited to the above-described embodiment.

In the above embodiment, the object 9 is a stone monument, and the image processing device 1 generates the emphasized image 35 as a rubbing image of the characters engraved on the stone monument 9, but the rubbing image generated by the image processing device 1. Is not limited to this. For example, the image processing device 1 may generate a rubbing image of a pattern engraved on the woodblock with the woodblock as the object 9, and may generate a rubbing image of the mural painting engraved on the cave with the cave as the object 9. An image may be generated, or a rubbing image of the appearance of the stone wall may be generated with the stone wall such as a castle wall or a moat as the object 9. In the enhanced image 35, the unevenness 8 of the object 9 is emphasized, but the unevenness 8 to be emphasized may be concave or convex.

In the above embodiment, the object 9 is a structure made of a natural material, but the object 9 may be a structure, and may be an artificial material such as a synthetic resin or a living body such as a cell. In the above embodiment, the size of the object 9 is also a size that can be visually observed with the naked eye of about several tens of centimeters to several meters, but the object 9 corresponds to the performance of the resolution of the external imaging device 2 and the optical lens 4. Any size may be sufficient as long as the unevenness 8 formed on the surface of the object 9 can be discriminated.

In the above embodiment, the brightness change image 33 is generated by subtracting the brightness value of the background image 31 from the brightness value of the oblique light image 32, but the subtraction at the time of generating the image is by addition using the complement representation. You may go. For example, by inverting all bits of the brightness value of the background image 31 expressed in binary and then adding 1 in binary, the brightness value of the background image 31 is expressed in complement and the brightness of the oblique light image 32. The brightness change image 33 can be generated by adding the value and the brightness value of the background image 31 represented by the complement. As a result, the brightness change image 33 can be generated by adding the images. Similarly, the enhanced image 35 can be generated by adding the brightness value of the smoothed image 34 and the brightness value of the brightness change image 33 represented by the complement.

In the above embodiment, the brightness change image 33 is generated by subtracting the brightness value of the background image 31 from the brightness value of the oblique light image 32, but the order of subtraction when generating the image is not limited. For example, even if the brightness change image 33 is generated by inverting the brightness values of the background image 31 and the oblique light image 32 and subtracting the oblique light image 32 with the inverted brightness value from the background image 31 with the inverted brightness value. Good. Thereby, the order of subtracting the images when the brightness change image 33 is generated can be changed. Similarly, the enhanced image 35 is obtained by reversing the brightness values of the brightness change image 33 and the smoothed image 34 and subtracting the smoothed image 34 having the inverted brightness value from the brightness change image 33 in which the brightness value is reversed. It may be generated. Inversion of the luminance value means, for example, in the case of an 8-bit grayscale image, the luminance value is converted into a value obtained by subtracting the luminance value from the maximum gradation value of 255.

In the above embodiment, the brightness change image 33 is generated by the difference between the oblique light image 32 and the background image 31, but the process of generating the brightness change image 33 is not limited to this. The brightness change image 33 may be generated based on the oblique light image 32 and the background image 31. For example, the brightness change image 33 may be generated by further applying various image processing filters to the image obtained by the difference between the oblique light image 32 and the background image 31. Similarly, the enhanced image 35 may be generated by further applying various image processing filters to the image obtained by the difference between the smoothed image 34 and the brightness change image 33.

In the above embodiment, the background image 31 of the object 9 acquires the first captured image, and the luminance change image 33 is generated using the first captured background image 31, but the background image 31 Acquisition is not limited to one. For example, the user of the image processing device 1 alternately images the background image 31 and the oblique light image 32 by using the light source 3 and the external image pickup device 2 or the image pickup unit 11, and the image processing device 1 obtains a plurality of acquired images. A brightness change image 33 may be generated based on the background image 31 and the plurality of oblique light images 32, for example, based on a pair of the oblique light image 32 and the background image 31 captured immediately before the oblique light image 32. The pair of the background image 31 and the oblique light image 32 used when generating the luminance change image 33 is also not limited to the pair that are adjacent in time.

In the above embodiment, after changing the irradiation area to generate a plurality of emphasized images 35 one after another, the generated plurality of emphasized images 35 are collectively combined, but the enhanced image 35 is generated in step S6. Each time, it may be sequentially combined with the emphasized image 35 that has already been combined in the iterative processing so far.

In the above embodiment, the enhanced image 35 (hereinafter, low-luminance enhancement) in which the luminance value of the portion corresponding to the shadow of the unevenness 8 is extracted by subtracting the luminance value of the luminance change image 33 from the luminance value of the smoothed image 34. A Takumoto image is generated by generating (referred to as an image) and synthesizing the generated plurality of emphasized images 35, but the enhanced image 35 to be combined is not limited to one type of low-luminance emphasized image. For example, by subtracting the brightness value of the smoothed image 34 from the brightness value of the brightness change image 33, the brightness value of the portion corresponding to the bright portion of the unevenness 8 is extracted as another type of enhanced image (enhanced image ( Hereinafter referred to as a high-luminance enhanced image) may be generated, and the low-luminance enhanced image and the high-luminance enhanced image may be combined. A takumoto image may be generated by synthesizing a plurality of emphasized images obtained by synthesizing a low-luminance-enhanced image and a high-luminance-enhanced image.

FIG. 13 is a schematic view showing a state in which oblique light is irradiated from a light source to the surface of the uneven surface. When the unevenness 8 is irradiated with oblique light, the portion indicated by the symbol S of the unevenness 8 is not sufficiently irradiated with the light from the light source and becomes a shadow, and the portion indicated by the symbol L of the unevenness 8 is the light from the light source. It is irradiated and becomes bright.

In the image processing method according to another embodiment, a low-luminance-enhanced image in which the portion indicated by the reference numeral S is emphasized and a high-luminance-enhanced image in which the portion indicated by the reference numeral L is emphasized are generated, and these low-luminance-enhanced images and the image are generated. The high-luminance enhanced image is combined to generate the enhanced image 35. In the image processing apparatus according to the other embodiment, the enhanced image generation unit 25 generates two types of enhanced images, a low-luminance enhanced image and a high-luminance enhanced image. The enhanced image generation unit 25 generates a low-brightness enhanced image by the difference between the smoothed image 34 and the portion of the brightness change image 33 whose brightness is lower than that of the smoothed image 34, and from the smoothed image 34 of the brightness change image 33. A high-luminance enhanced image is generated by the difference between the high-luminance portion and the smoothed image 34. The enhanced image generation unit 25 synthesizes the low-luminance enhanced image and the high-luminance enhanced image to generate the enhanced image 35.

Two types of emphasized images will be described with reference to FIG. FIG. 14 is a diagram for explaining two types of enhanced images, a low-luminance-enhanced image and a high-luminance-enhanced image, which are generated in the image processing method according to another embodiment.

(A) to (C) are graphs showing the distribution of luminance values along a certain lateral position of an image. In (A), a graph 35 g (gray) showing the distribution of the luminance values of the low-luminance-enhanced image from which the luminance values of the portion S corresponding to the shadow of the unevenness 8 are extracted is shown. In (B), a graph 35 g (black) showing the distribution of the luminance values of the high-luminance-enhanced image is shown, in which the luminance values of the portions corresponding to the bright portions L of the unevenness 8 are extracted. (C) is a graph obtained by synthesizing the graph 35g (gray) of the low-luminance-enhanced image shown in (A) and the graph 35g (black) of the high-luminance-enhanced image shown in (B). FIG. (D) is an enlarged view of a part of the enhanced image generated by synthesizing the low-luminance-enhanced image and the high-luminance-enhanced image. The graph 35g shown in (A) is the same as the graph 35g shown in FIG. 10 (B). As shown in (D), when the low-luminance-enhanced image and the high-luminance-enhanced image are combined, a single irradiation of diagonal light to the unevenness 8 produces a vertical line of a character character formed by the unevenness 8. Can be extracted.

As described above, in the image processing method according to the other embodiment, two types of enhanced images, a low-luminance-enhanced image and a high-luminance-enhanced image, are generated from the same pair of the background image and the oblique light image, and these are combined. This makes it possible to generate an emphasized image 35 having a high complement ratio of the character shape of the character composed of the unevenness 8. This makes it possible to generate a rubbing image with a smaller number of images. Further, the mode of generating these two types of emphasized images and synthesizing them is useful, for example, when there is little noise on the surface of the object and the characters composed of irregularities are complicated. The fact that the noise on the surface of the object is small means that, for example, the proportion of irregularities other than the irregularities constituting the characters is small on the surface of the object.

In the above embodiment, the image processing device 1 and the external imaging device 2 have independent configurations, but the image processing device 1 and the external imaging device 2 may be integrated. The light source 3 may be further integrated with this integrated configuration.

In the above embodiment, the surface of the object 9 is partially irradiated with oblique light by the user using the light source 3, but the light source 3 may be mounted on, for example, a drone. Further, the irradiation work may be automated by controlling the drone from the image processing device 1. The drone may be further equipped with an external imaging device 2. That is, one or both of the light source 3 and the external imaging device 2 may be mounted on the drone to control either or both of the irradiation position and the imaging position from the image processing device 1.

Some or all of the functions of the data processing unit 12 and the data items in the auxiliary storage device 13 may be cloud-based in a server device (not shown) connected via the communication I / F unit 16. .. For example, the brightness change image generation unit 23, the smoothing image generation unit 24, the emphasis image generation unit 25, and the enhancement image composition unit 26 may be provided in the server device. In this case, the image processing device 1 transmits the background image 31 and the oblique light image 32 to the server device via the I / F unit 16. The brightness change image generation unit 23, the smoothing image generation unit 24, the emphasis image generation unit 25, and the enhancement image composition unit 26 of the server device are based on the received background image 31 and oblique light image 32, and the brightness change image 33 and smoothing are performed. The image 34 and the emphasized image 35 are generated, and the plurality of emphasized images 35 are combined. The server device may transmit the combined enhanced image to the image processing device 1 via the I / F unit 16, and the image processing device 1 may display the received combined enhanced image on the display unit 15.

In the above embodiment, the functional blocks 21 to 26 constituting the data processing unit 12 are realized by software, but some or all of the functional blocks 21 to 26 may be realized as hardware. The processing of the functional blocks 21 to 26 constituting the data processing unit 12 does not have to be processed by a single processor, and may be distributed and processed by a plurality of processors.

Examples of the present invention are shown below to further clarify the features of the present invention.

In the following examples, MacBook Pro (registered trademark) 2013 was used for the image processing device 1, and Nikon (registered trademark) D810 was used for the external imaging device 2. A wireless transmitter was connected to the external image pickup device 2, and the captured image was transferred to the image processing device 1 by Wi-Fi (registered trademark). A penlight type LED lighting was used as the light source 3. Partial irradiation of the surface of the object 9 with oblique light was performed by the user using the light source 3.

In the first embodiment, the accuracy of the obtained rubbed image and the time required for processing were compared between the method using the image processing apparatus of the present invention and another conventional method.

FIG. 15 is a diagram for comparing the method of the present invention with other conventional methods. (A) is an image of a paper book rubbing book using conventional ink and paper. (B) is a two-dimensional image cut out from the three-dimensional shape data recorded by the SfM technique. (C) is a rubbing image by the image processing apparatus of the present invention. (D) is a reprint of the characters written on the stone monument, which is the object. The source of the reprint is "Ama-gun Magazine" compiled by the Ama-gun Magazine Publishing Association and published by the Ama-gun Magazine Publishing Association in 1945.

It took about 30 minutes to prepare the paper book rubbed shown in (A). The total working time required to create the two-dimensional image shown in (B) was about 6 hours 57 minutes 20 seconds. The breakdown of the working time was that the time required for shooting was about 5 minutes and 40 seconds, and the time required for analysis was about 6 hours 51 minutes and 50 seconds. The rubbing image shown in (C) is a rubbing image created in the third embodiment described later, and the total working time required to generate the rubbing image was 4 minutes and 47 seconds.

Based on the transcription of the characters shown in (D), the accuracy of the rubbing image shown in (A) to (C) was confirmed. It was confirmed that the rubbing image by the image processing apparatus of the present invention shown in (C) reproduces the transcribed characters shown in (D) with the clearest and most accurate accuracy. Further, it was confirmed that the rubbing image by the image processing apparatus of the present invention shown in (C) was generated most rapidly.

In the second to fourth embodiments, the time required for processing the rubbing image was verified for various objects.

FIG. 16 is an image according to the second embodiment. (A) is an image of an object, and (B) is a rubbing image by the image processing apparatus of the present invention.

For the object shown in (A), background images and oblique light images are alternately imaged, 35 sets of background image and oblique light image pairs are imaged, a total of 70 images are imaged, and 35 emphasized images are comparatively brightly combined. The Takumoto image shown in (B) was generated. The total work time required from the start of imaging the background image to the acquisition of the rubbing image, including the partial irradiation of the surface of the object with oblique light, was 6 minutes and 19 seconds. After a total of 70 images of the object were captured, the captured images were transferred to an image processing device, and image processing was performed by the image processing device to generate the Takumoto image shown in (B). The time required for image processing was 28 seconds.

FIG. 17 is an image according to the third embodiment. (A) is an image of an object, and (B) is a rubbing image by the image processing apparatus of the present invention. (C) is a black-and-white inverted and enlarged image of the rubbing image of (B).

For the object shown in (A), a total of nine images, one background image and eight oblique light images, were imaged, and the eight emphasized images were comparatively brightly combined to generate the rubbing image shown in (B). .. The total working time was 1 minute. After a total of nine images of the object were captured, the captured images were transferred to an image processing device, and image processing was performed by the image processing device to generate the Takumoto image shown in (B). The time required for image processing was 7 seconds. When character recognition was performed on the black-and-white inverted takumoto image shown in (C) using commercially available software having an OCR function, the character recognition rate was about 88%.

FIG. 18 is an image according to the fourth embodiment. (A) is an image of an object, and (B) is a rubbing image by the image processing apparatus of the present invention. The stone monument shown in (A) as an object is said to be the oldest tsunami monument in Japan that exists in Minami Town, Tokushima Prefecture. This stone monument is said to be a memorial monument due to the tsunami disaster of the Nankai earthquake that occurred on June 24, 1661 (August 3, 1361 AD), and was erected in 1380, 20 years after the tsunami disaster. Because it was made, it is called the monument of the Kang calendar.

For the object shown in (A), background images and oblique light images are alternately imaged, 224 sets of background image and oblique light image pairs are imaged, a total of 448 images are imaged, and 224 emphasized images are comparatively brightly combined. The Takumoto image shown in (B) was generated. The total working time was 4 minutes and 47 seconds. In the fourth embodiment, a Gaussian filter was used for the smoothing treatment. This enables real-time image transfer and real-time image processing with an image processing device.

FIG. 19 is an image according to the fifth embodiment. (A) is a rubbing image generated by comparatively brightly synthesizing a plurality of low-luminance-enhanced images, and (B) is a composite image of a plurality of low-luminance-enhanced images and high-luminance-enhanced images generated by comparatively brightly synthesizing. This is the generated Takumoto image.

A background image and an oblique light image of exactly the same source data were used to generate the rubbing images shown in (A) and (B). Comparing the rubbing images of (A) and (B), the character shape of the character (rubbing character) engraved on the stone monument is clearer in the rubbing image shown in (B) than in the rubbing image shown in (A). It was confirmed that there was.

As described above, through the first to fifth examples, it was confirmed that the image processing apparatus of the present invention can quickly and easily generate a rubbing image of an object.

1 Image processing device 2 External imaging device 3 Light source 4 Optical lens 5 Tripod 8 Concavo-convex 9 Object (stone monument)
11 Imaging unit 12 Data processing unit 13 Auxiliary storage device 14 Input unit 15 Display unit 16 Communication interface unit 21 Background image acquisition unit 22 Oblique image acquisition unit 23 Brightness change image generation unit 24 Smoothing image generation unit 25 Emphasis image generation unit 26 Emphasis Image synthesizer 31 Background image 32 Oblique image 33 Brightness change image 34 Smoothed image 35 Emphasis image P Image processing program

Claims (6)

A background image acquisition unit that acquires a background image including irregularities formed on the surface of an object,
An oblique light image acquisition unit that acquires an oblique light image in which at least a part of the surface is irradiated with oblique light,
A luminance change image generation unit that generates a luminance change image representing a luminance change due to irradiation of the light based on the oblique light image and the background image.
A smoothed image generation unit that generates a smoothed image by applying a smoothing process to the luminance change image,
An enhanced image generation unit that generates an enhanced image in which the unevenness is emphasized based on the smoothed image and the brightness change image.
Equipped with a,
The brightness change image generation unit generates the brightness change image by the difference between the oblique light image and the background image.
The enhanced image generation unit generates the enhanced image based on the difference between the smoothed image and the brightness change image.
The emphasized image generation unit
A low-brightness enhanced image is generated by the difference between the smoothed image and a portion of the brightness-changing image whose brightness is lower than that of the smoothed image.
A high-luminance enhanced image is generated by the difference between the portion of the brightness-changing image having a higher brightness than the smoothed image and the smoothed image.
Wherein by combining the low brightness enhanced image and the high brightness enhanced image, that generates the enhanced image, the image processing apparatus. As the smoothing process, the smoothed image generation unit applies any one or a combination of a smoothing filter, a median filter, and a Gaussian filter to the brightness change image to generate the smoothed image. , The image processing apparatus according to claim 1. The oblique light image acquisition unit acquires a plurality of the oblique light images having different light irradiation regions, and obtains the oblique light images.
The image processing apparatus according to claim 1 or 2 , further comprising a emphasized image synthesizing unit for synthesizing a plurality of the enhanced images created for each of the oblique light images having different irradiation regions. The image processing apparatus according to claim 3 , wherein the enhanced image synthesizing unit comparatively brightly synthesizes a plurality of the enhanced images based on the brightness of the unevenness in the enhanced image. A background image acquisition step for acquiring a background image including irregularities formed on the surface of an object, and
An oblique light image acquisition step of acquiring an oblique light image in which at least a part of the surface is irradiated with oblique light, and
A brightness change image generation step of generating a brightness change image representing a brightness change due to irradiation of the light based on the oblique light image and the background image.
A smoothing image generation step of applying a smoothing process to the brightness change image to generate a smoothed image,
A highlight image generation step of generating a emphasized image in which the unevenness is emphasized based on the smoothed image and the brightness change image, and
Only including,
The brightness change image generation step generates the brightness change image by the difference between the oblique light image and the background image.
In the enhanced image generation step, the enhanced image is generated by the difference between the smoothed image and the brightness change image.
The enhanced image generation step
A low-luminance enhanced image is generated by the difference between the smoothed image and a portion of the brightness-changing image whose brightness is lower than that of the smoothed image.
A high-luminance enhanced image is generated by the difference between the portion of the brightness-changing image having a higher brightness than the smoothed image and the smoothed image.
An image processing method for generating the enhanced image by synthesizing the low-luminance-enhanced image and the high-luminance-enhanced image. A program for operating a computer as each part of the image processing apparatus according to claims 1 to 4.

Images