JP7170023B2 - DAMAGE DATA EDITING DEVICE, DAMAGE DATA EDITING METHOD, PROGRAM, AND SYSTEM - Google Patents

Description

TECHNICAL FIELD The present invention relates to a damage data editing device, a damage data editing method, and a program, and more particularly to a technique for editing damage data generated from a photographed image of a building to be inspected.

Structures such as bridges and tunnels are more likely to be damaged over time. Therefore, it is required to periodically inspect the occurrence of damage and the progress of damage.

An investigator who inspects a building needs to prepare an inspection record in a predetermined format as a form showing the results of the inspection. As one of the inspection records, there is a damage diagram showing the state of damage to the building. Even an expert who is different from the investigator who actually inspected the building can grasp the occurrence and progress of the damage to the building by looking at the damage diagram created in the prescribed format, and formulate a maintenance management plan for the building. It becomes possible to formulate

Further, in recent years, there are cases where inspections of buildings are performed using photographed images. That is, an investigator acquires a photographed image of a building to be inspected, and inspects the building for damage by detecting damage (for example, cracks) from the photographed image.

Patent Literature 1 describes a technique for creating an inspection drawing (corresponding to a damage drawing) using a photographed image of a building. Specifically, it describes a technique for creating damage objects such as lines, planes, and characters from the characteristics of damage in a photographed image of a building to be inspected, and arranging them on drawing data read by a scanner.

Japanese Patent Application Laid-Open No. 2004-318790

Here, when detecting damage from a photographed image by image processing, damage can be detected with higher accuracy by performing damage detection processing before applying processing such as projective transformation to the photographed image. That is, when a photographed image is processed such as projective transformation, the data of the photographed image is affected by the influence of the processing, and damage cannot be accurately detected in some cases.

However, the technique described in Patent Literature 1 does not mention generating a damaged object based on a photographed image before projective transformation.

In addition, when detecting damage from a photographed image by image processing, damage is not always detected with high accuracy. It may be necessary to edit (modify and refine) the results.

However, the technique described in Patent Literature 1 does not refer to an editing function such as combining the damage in the captured image with the object data when generating the object data that is the detection result of the damage. That is, the technique described in Patent Document 1 does not mention a function for the user to check whether the generated object data accurately expresses the damage of the captured image and to edit the data.

SUMMARY OF THE INVENTION The present invention has been made in view of such circumstances, and aims to provide a damage data editing apparatus, a damage data editing method, and a damage data editing method capable of obtaining damage data in which damage captured in a photographed image is expressed with high accuracy. to provide the program.

A damage data editing device, which is one aspect of the present invention for achieving the above object, includes an image acquisition unit that acquires a photographed image of a building to be inspected, and an image analysis of the photographed image to obtain damage data. A damage data generation unit that generates data, a display unit, a display control unit that displays a photographed image on the display unit and overlays the damage data on the photographed image, and an instruction to edit the damage data displayed on the display unit. , and edit the damage data based on the editing command, the drawing data acquisition unit acquires the drawing data of the building to be inspected, and the captured image and the drawing data acquisition unit displayed on the display unit A projection relation reception unit that receives an input of a projection relation with the acquired drawing data, and damage data generated by the damage data generation unit or damage edited by the editing unit based on the projection relation received by the projection relation reception unit. A projective transformation unit for projectively transforming data, and an output unit for outputting damage data projectively transformed by the projective transformation unit.

According to this aspect, the damage data is generated by image analysis of the photographed image in the damage data generation unit, and the generated damage data is projectively transformed based on the projective relationship between the photographed image and the drawing data. . That is, the damage data of this aspect is generated based on the detection of damage in the captured image before projective transformation, and is then projectively transformed. As a result, in this aspect, the captured image for damage detection is not affected by image quality deterioration due to projective transformation, and damage data that accurately reproduces the damage shown in the captured image can be generated.

Further, according to this aspect, the display control unit causes the display unit to display the photographed image, and also displays the damage data overlaid on the photographed image. can be edited. That is, the user can confirm whether or not the generated damage data accurately represents the damage shown in the captured image. For example, if there is a difference between the damage shown in the captured image and the damage data, the user can edit the damage data with reference to the damage shown in the captured image. As a result, this aspect can generate damage data that accurately represents the damage captured in the captured image.

Preferably, the display control unit causes the display unit to display the drawing data, and the projection relation reception unit displays the projection relation between the captured image displayed on the display unit and the drawing data acquired by the drawing data acquisition unit. 4 points of image and 4 points of drawing data are accepted.

According to this aspect, the projective relation receiving unit obtains four points of the photographed image and four points of the drawing data, which indicate the projective relationship between the photographed image displayed on the display unit and the drawing data obtained by the drawing data obtaining unit. Acceptable. As a result, in this aspect, the projective relationship is appropriately accepted, and the accurate projective transformation of the damage data is performed.

Preferably, the projective transformation unit calculates a projective transformation matrix that aligns the four points of the captured image with the four points of the drawing data, and uses the calculated projective transformation matrix to generate damage data generated by the damage data generating unit or edited by the editing unit. Projectively transform the damage data.

According to this aspect, the projective transformation unit calculates the projective transformation matrix that aligns the four points of the captured image with the four points of the drawing data, and the damage data generated by the damage data generating unit or the editing unit is calculated based on the calculated projective transformation matrix. The edited damage data is projectively transformed. Thereby, this aspect can perform the projective transformation of damage data correctly.

Preferably, the damaged data editing device includes: a mirroring relationship reception unit that receives an input of a mirroring relationship between the photographed image displayed on the display unit and the drawing data acquired by the drawing data acquisition unit; a mirroring unit for mirroring the damage data based on the mirroring relationship, the output unit outputting the mirrored and projectively transformed damage data.

According to this aspect, the mirroring relationship reception unit receives the input of the mirroring relationship between the photographed image displayed on the display unit and the drawing data acquired by the drawing data acquisition unit, and the mirroring relationship reception unit receives Damaged data is mirrored based on the accepted mirroring relationship. This allows the present embodiment to obtain damage data that is mirrored and provides damage data that illustrates the damage state as seen from the opposite side of the building to the photographed side.

Preferably, the damage data editing device includes a photographed image synthesizing unit that synthesizes a plurality of photographed images obtained by separately photographing a section of the building, and the damage data generating unit includes, for each of the plurality of photographed images, Generate damage data.

According to this aspect, the photographed image synthesizing unit synthesizes a plurality of photographed images obtained by separately photographing one section of the building. A photographed image can be displayed. Further, according to this aspect, the damage data is generated for each of the plurality of photographed images by the damage data generator. That is, in this aspect, since the damage data is generated for each photographed image before being combined, the damage data generation unit can generate the damage data with high accuracy without being affected by the composition of the photographed images. can.

Preferably, the photographed image synthesizing unit synthesizes a plurality of photographed images photographed corresponding to one section represented by the drawing data for each section.

According to this aspect, the photographed image synthesizing unit synthesizes a plurality of photographed images photographed corresponding to one section represented by the drawing data for each section. Accordingly, in this aspect, a composite image for each section can be obtained, and the composite image for each section can be displayed.

Preferably, the damage data editing device includes a damage data synthesizing unit that synthesizes the damage data based on synthesis information of the plurality of photographed images synthesized by the photographed image synthesizing unit.

According to this aspect, the damage data synthesizing unit synthesizes the damage data based on the synthesis information of the plurality of photographed images synthesized by the photographed image synthesizing unit. Accordingly, this embodiment can obtain synthesized damage data.

Preferably, the damage data synthesizing unit performs synthesizing for each section.

According to this aspect, the damage data synthesizing section performs synthesizing for each section. As a result, this embodiment can obtain damage data synthesized for each section.

Preferably, the damage data editing device includes a damage drawing data creation unit that creates damage drawing data in which the damage data projectively transformed by the projective conversion unit and drawing data corresponding to the damage data are superimposed, and the output unit includes: Output damaged drawing data.

According to this aspect, the damage drawing data creation unit creates the damage drawing data in which the damage data projectively transformed by the projective transformation unit and the drawing data corresponding to the damage data are superimposed, and the output unit outputs the damage drawing data is output. As a result, according to this aspect, it is possible to obtain damage drawing data having damage data representing the damage of the photographed image with high accuracy.

Preferably the structure is a bridge or a tunnel.

A damage data editing method according to another aspect of the present invention includes an image acquisition step of acquiring a photographed image of a structure to be inspected, and a damage data generation of generating damage data by image analysis of the photographed image. a display control step of displaying the photographed image on the display unit and displaying damage data overlaid on the photographed image; receiving an instruction to edit the damage data displayed on the display unit; An editing step of editing data, a drawing data acquisition step of acquiring drawing data of a building to be inspected, and inputting a projection relationship between the photographed image displayed on the display unit and the drawing data acquired in the drawing data acquisition step. a projective relationship receiving step for receiving; a projective transformation step for projectively transforming the damage data generated in the damage data generating step or the damage data edited in the editing step based on the projective relationship received in the projective relationship receiving step; and an output step of outputting the damage data that has been projectively transformed in the transform step.

A program for causing a computer to execute a damage data editing step, which is another aspect of the present invention, includes an image acquisition step of acquiring a photographed image of a building to be inspected, and an image analysis of the photographed image to generate damage data. a display control step of displaying the photographed image on the display unit and overlaying the damage data on the photographed image; receiving an instruction to edit the damage data displayed on the display unit, and editing an editing step of editing damage data based on the command of , a drawing data acquisition step of acquiring drawing data of a building to be inspected, and a photographed image displayed on the display unit and the drawing data acquired in the drawing data acquisition step projective transformation of the damage data generated in the damage data generating step or the damage data edited in the editing step based on the projective relation receiving step of receiving the input of the projective relation with and the projective relation received in the projective relation receiving step a projective transformation step; and an output step of outputting the damage data projectively transformed in the projective transformation step.

According to the present invention, the damage data is generated based on the detection of damage in the photographed image before projective transformation, and is then projectively transformed. Therefore, it is possible to generate damage data accurately reproducing the damage shown in the photographed image. Further, according to the present invention, it is possible to provide the user with a function of confirming whether or not the generated damage data accurately represents the damage shown in the photographed image, and editing the data. It is possible to generate damage data that accurately expresses the damage that has been done.

FIG. 1 is a perspective view showing the structure of a bridge, which is an example of a building. FIG. 2 is a block diagram showing a configuration example of the damaged data editing device. FIG. 3 is a diagram showing an example of damage graphics stored in the storage unit. FIG. 4 is a flow chart showing the operation flow of the damaged data editing device. FIG. 5 is a diagram for explaining a specific example of the projective relationship received by the projective relationship receiving unit. FIG. 6 is a block diagram showing a configuration example of the damaged data editing device. FIG. 7 is a diagram for explaining mirroring. FIG. 8 is a diagram for explaining mirroring. FIG. 9 is a diagram showing an example of damaged drawing data. FIG. 10 is a diagram showing an example of inverted damage drawing data. FIG. 11 is a flow chart showing the operation flow of the damaged data editing device. FIG. 12 is a diagram showing an example of a CAD drawing. FIG. 13 is a diagram for explaining a specific example of inputting the projection relationship and the mirroring relationship. FIG. 14 is a diagram for explaining mirroring. FIG. 15 is a diagram showing damage data obtained by mirroring. FIG. 16 is a diagram for explaining the creation of damage drawing data. FIG. 17 is a diagram showing a display example of an edit screen relating to floor slab coffers. FIG. 18 is a diagram schematically showing an example of a crack progress model in a floor slab. FIG. 19 is a diagram showing a photographed image in which a coffer is photographed. FIG. 20 is a diagram conceptually showing a robot device equipped with an imaging device. FIG. 21 is an explanatory diagram for explaining variations of mirroring.

DESCRIPTION OF THE PREFERRED EMBODIMENTS Preferred embodiments of a damage data editing device, a damage data editing method, and a program according to the present invention will be described below with reference to the accompanying drawings.

FIG. 1 is a perspective view showing the structure of a bridge, which is an example of a building to which the present invention is applied, and is a perspective view when the bridge 1 is viewed from below. A bridge 1 in this figure has a main girder 2 , a cross girder 3 , a tilting structure 4 and a horizontal structure 5 . Above the main girder 2, a floor slab 6, which is a member made of concrete, is provided. The main girder 2 is a member that spans between the bridge piers 7 (or abutments) and supports the load on the floor slab 6 . The horizontal girder 3 is a member that connects the main girder 2 so that the load is supported by the main girder 2 . The counter leaning structure 4 and the lateral structure 5 are members that connect the main girder 2 in order to particularly resist lateral loads. The bridge piers 7 are columns that support the superstructure of the main girder 2 , transverse girder 3 , opposite tilting structure 4 , horizontal structure 5 and floor slab 6 of the bridge 1 , and are members corresponding to the legs of the bridge 1 .

It should be noted that the bridge 1 shown in FIG. 1 is an example of a building introduced for use in explaining the present invention, and the building in the present invention is not limited to such a bridge 1. Any object that can detect damage from a photographed image and that can be inspected corresponds to the building of the present invention. For example, a building according to the invention may be a tunnel.

<First embodiment>
FIG. 2 is a block diagram showing a configuration example of the damage data editing device 20 according to the first embodiment. The damage data editing device 20 acquires the photographed image of the building to be inspected photographed by the photographing device 10 .

The photographing device 10 is configured by a photographing device for photographing a building to be inspected. Examples of photographing devices include digital cameras, cameras built into smartphones, and cameras built into tablet terminals. A photographed image acquired by the photographing device 10 is input to the damage data editing device 20 .

The damage data editing device 20 of this embodiment is provided in, for example, a computer. The damage data editing device 20 mainly includes an image acquisition section 22 , a drawing data acquisition section 24 , a display section 26 , an operation section 28 , a CPU 30 and a storage section 50 .

The image acquisition unit 22 acquires a photographed image of a building to be inspected. Further, the image acquisition unit 22 may acquire a plurality of photographed images obtained by dividing and photographing one section of the building. Here, one section of the building is, for example, one section of the building which is delimited for convenience of inspection, and includes a coffer in the floor slab 6 (see FIGS. 1 and 19).

The drawing data acquisition unit 24 acquires drawing data of a building to be inspected. For example, the drawing data acquisition unit 24 acquires a CAD (computer aided design) drawing showing a building to be inspected. The drawing data acquisition unit 24 may also acquire drawing data read by a scanner or the like.

The image acquisition unit 22 and the drawing data acquisition unit 24 can be configured by an input device for inputting digital signals. As such an input device, for example, a communication device that performs wireless communication or wired communication may be used. An interface device with a storage medium such as a memory card may be used as an input device.

The display unit 26 is configured by a display device such as a liquid crystal display device. For example, the display unit 26 is configured by a monitor of a computer.

The operation unit 28 receives user operations. Can be configured with keyboard and mouse. The operation unit 28 may be configured by an operation device other than these, for example, a touch panel provided on the display unit 26 .

The CPU 30 is a CPU (central processing unit) 30 that executes programs, and is composed of a single CPU or a plurality of CPUs.

The storage unit 50 stores programs and information necessary for executing the programs. The storage unit 50 is configured by a memory device.

The CPU 30 includes a damage data generation unit 32 , a projection relationship reception unit 34 , a projective transformation unit 36 , a display control unit 38 , an editing unit 40 and an output unit 44 .

The damage data generator 32 generates damage data by image analysis of the captured image. That is, the damage data generation unit 32 detects damage by performing image analysis on the photographed image acquired by the image acquisition unit 22, and generates damage data expressing the damage. Here, the damage that appears in the photographed image includes cracks, peeling, exposure of reinforcing bars, free lime, and water leakage. The damage data is data representing the damage shown in the photographed image on the display unit 26 . For example, the damage data may be image data obtained by tracing the damage appearing in the photographed image, or damage graphics (see FIG. 3) stored in advance in the storage unit 50 for each type of damage. Known techniques are used for the image analysis for detecting the damage image in the photographed image performed by the damage data generator 32 and the technique for generating the damage data.

FIG. 3 is a diagram showing an example of damage graphics stored in the storage unit 50. As shown in FIG. As shown in FIG. 3, a damage figure corresponding to the type of damage is stored in the storage part 50, and the stored damage figure is stored in the damage data generation part 32 according to the detected damage. May be used as damage data. FIG. 3 illustrates damage diagrams of cracks, spalling, exposed rebar, free lime, and water leakage.

The projection relationship reception unit 34 receives an input of the projection relationship between the captured image displayed on the display unit 26 and the drawing data acquired by the drawing data acquisition unit 24 . Projection relations received by the projection relation receiving unit 34 are received in various forms. For example, the projection relation receiving unit 34 may receive the projection relation by receiving designation of corresponding four points in the photographed image and the drawing data. Further, for example, the projection relation receiving unit 34 detects a portion of the captured image that should be a rectangle by a known image recognition technique, and designates a portion of the drawing data that corresponds to the detected portion that should be a rectangle, thereby obtaining a projection relation. may be accepted.

The projective transformation unit 36 projectively transforms the damage data detected by the damage data generation unit 32 or the damage data edited by the editing unit 40 based on the projective relation received by the projective relation receiving unit 34 . The projective transformation unit 36 projectively transforms the damage data or the damage data edited by the editing unit 40 using a known technique. For example, the projective transformation unit 36 calculates a projective transformation matrix that aligns the four points of the captured image with the four points of the drawing data, and uses the calculated projective transformation matrix to generate damage data generated by the damage data generating unit 32 or edited by the editing unit 40. Projectively transform the damage data.

The display control unit 38 causes the display unit 26 to display the photographed image, and also causes the damage data to be overlaid on the photographed image. That is, the display control unit 38 causes the display unit 26 to display the photographed image, and superimposes the damage data so as to correspond to the damage in the photographed image. The display control unit 38 has a function of causing the display unit 26 to appropriately display the photographed image, the damage data, the drawing data, and the damage drawing data.

The editing unit 40 receives an instruction to edit the damage data displayed on the display unit 26, and edits the damage data based on the editing instruction. For example, the user compares the captured image displayed on the display unit 26 with the damage data automatically generated by the damage data generation unit 32, and if the damage data does not accurately represent the damage in the captured image, Edit (process) damage data. In this case, the user uses a mouse (manipulation unit 28) attached to the computer to enter commands for editing the damage data. Then, the editing unit 40 edits the damage data based on the editing command. The editing unit 40 also accepts various editing operations such as addition, deletion, and correction.

The output unit 44 outputs the damage data projectively transformed by the projective transformation unit 36 . The output form of the output unit 44 is not particularly limited as long as the damage data or damage drawing data (FIGS. 9 and 10) can be effectively used. For example, the output unit 44 may output in a DXF (Drawing Exchange Format) file format, or in another image data format.

Next, the operational flow of the damaged data editing device 20 will be described. FIG. 4 is a flow chart showing the operation flow of the damaged data editing device 20 of the first embodiment.

First, the image acquisition unit 22 acquires a photographed image photographed by the photographing device 10 (step S10: image acquisition step). Then, the damage data generation unit 32 performs image analysis on the acquired photographed image, detects damage appearing in the photographed image, and generates damage data based on the detection result (Step S11: Damage data generation step). Thereafter, the display control unit 38 overlays the captured image and the detected damage data on the display unit 26 (step S12: display control step).

While confirming the photographed image and the damage data displayed on the display unit 26, the user inputs a command to edit the damage data displayed on the display unit 26 using a mouse connected to the personal computer, and the editing unit 40 The damage data is edited based on the input command (step S13: editing step).

Next, the drawing data is obtained by the drawing data obtaining unit 24 (step S14: drawing data obtaining step). Note that the acquisition of the drawing data may be performed in advance. For example, the drawing data acquisition unit 24 may acquire drawing data of a building to be inspected before inspection work, and the acquired drawing data may be stored in the storage unit 50 .

After that, the projection relation receiving unit 34 receives the input of the projection relation (step S15: projection relation receiving step).

FIG. 5 is a diagram for explaining a specific example of the projection relationship accepted by the projection relationship acceptance unit 34. As shown in FIG.

A photographed image 100 of the bridge pier 7 to be inspected is displayed on the display unit 26 by the display control unit 38, and a CAD drawing 70 (drawing data) of the building is displayed. Specifically, a captured image 100 and a CAD drawing 70 are displayed on the monitor of the computer.

The damage data DD1 and DD2 generated by the damage data generator 32 are overlaid and displayed so as to correspond to the damage in the captured image.

The user uses the pointing device of the computer to input points corresponding to the four points of the rectangle in the captured image 100 and the CAD drawing 70 of the bridge pier 7 displayed on the display unit 26 . That is, four points A1, B1, C1, and D1 of the captured image 100 are specified and input, and corresponding four points A2, B2, C2, and D2 are specified and input in the CAD drawing 70. FIG. Then, the projective relation receiving unit 34 receives each of the four points (A1, B1, C1, and D1 and A2, B2, C2, and D2 of the captured image 100) input by the user as a projective relation.

Returning to FIG. 4, the projective transformation unit 36 projectively transforms the damage data DD1 and DD2 of the captured image 100 based on the projective relationship received by the projective relationship receiving unit 34 (step S16: projective transformation step). Specifically, the projective transformation unit 36 calculates a projective transformation matrix that matches the four points A1, B1, C1, and D1 of the captured image 100 with the four points A2, B2, C2, and D2 of the drawing data, The damage data detected by the damage data generating unit 32 or the damage data edited by the editing unit 40 is projectively transformed using the calculated projective transformation matrix.

Thereafter, the damage data subjected to the projective transformation is output by the output unit 44 (step S17: output step).

In the above embodiment, the hardware structure of the processing unit that executes various processes is various processors as shown below. Various processors include a CPU, which is a general-purpose processor that executes software (programs) and functions as various processing units, and a programmable processor, such as a FPGA (Field Programmable Gate Array), whose circuit configuration can be changed after manufacturing. A logic device (Programmable Logic Device: PLD), an ASIC (Application Specific Integrated Circuit), and a dedicated electric circuit that is a processor having a circuit configuration specially designed to execute specific processing are included.

One processing unit may be composed of one of these various processors, or may be composed of two or more processors of the same type or different types (eg, multiple FPGAs, or combinations of CPUs and FPGAs). may Also, a plurality of processing units may be configured by one processor. As an example of configuring a plurality of processing units in a single processor, first, as represented by a computer such as a client or server, a single processor is configured by combining one or more CPUs and software. There is a form in which a processor functions as multiple processing units. Secondly, as typified by System On Chip (SoC), etc., there is a form of using a processor that realizes the function of the entire system including a plurality of processing units with a single IC (Integrated Circuit) chip. be. In this way, the various processing units are configured using one or more of the above various processors as a hardware structure.

Further, the hardware structure of these various processors is, more specifically, an electrical circuit that combines circuit elements such as semiconductor elements.

Each of the configurations and functions described above can be appropriately realized by arbitrary hardware, software, or a combination of both. For example, a program that causes a computer to execute the above processing steps (procedures), a computer-readable recording medium (non-temporary recording medium) recording such a program, or a computer that can install such a program However, it is possible to apply the present invention.

<Second embodiment>
Next, a second embodiment of the invention will be described.

FIG. 6 is a block diagram showing a configuration example of the damaged data editing device 20 according to the second embodiment. In addition, the same code|symbol is attached|subjected to the location which already demonstrated in FIG. 2, and description is abbreviate|omitted. The damage data editing device 20 mainly includes an image acquisition section 22 , a drawing data acquisition section 24 , a display section 26 , an operation section 28 , a CPU 30 and a storage section 50 .

The CPU 30 includes a damage data generator 32 , a captured image synthesizing unit 46 , a damage data synthesizing unit 48 , a mirroring relationship reception unit 52 , a projection relationship reception unit 34 , a projective transformation unit 36 , a mirroring unit 54 , a display control unit 38 , and an editing unit 40 . , a damage drawing data creation unit 42 and an output unit 44 .

The photographed image synthesizing unit 46 synthesizes a plurality of photographed images obtained by separately photographing one section of the building. That is, when a plurality of photographed images are input from the image acquisition unit 22, the photographed image synthesizing unit 46 synthesizes the plurality of input photographed images. The image synthesis of the photographed image synthesizing unit 46 is performed by using a known technique. For example, the photographed images are synthesized by detecting feature points of each photographed image. In addition, the photographed image synthesizing unit 46 may synthesize a plurality of photographed images photographed corresponding to one section represented by the drawing data for each section. For example, when a plurality of captured images in which a coffer is adopted as one section is input, the captured image synthesizing unit 46 may synthesize the captured images for each coffer.

Even when a plurality of photographed images are acquired by the image acquisition unit 22, the damage data generation unit 32 detects damage and generates damage data for each of the plurality of photographed images. That is, even when a plurality of images are acquired by the image acquisition unit 22, the damage data generation unit 32 performs image processing on each of the captured images acquired before being combined to generate damage data. do. As a result, it is possible to suppress deterioration in damage detection accuracy due to the influence of image synthesis.

The damage data synthesizing unit 48 synthesizes damage data based on the synthesis information of the plurality of photographed images synthesized by the photographed image synthesizing unit 46 . That is, the damage data synthesizing unit 48 generates in each shot image based on synthesis information (for example, overlapping of each shot image when synthesizing, coordinates of feature points, etc.) in the case of synthesis in the shot image synthesizing unit 46. Synthesize damage data. Also, the damage data synthesizing unit 48 may synthesize a plurality of photographed images for each section. For example, the damage data synthesizing unit 48 synthesizes damage data for each partition when a partition is adopted as one partition.

The mirroring relationship reception unit 52 receives an input of the mirroring relationship between the photographed image displayed on the display unit 26 and the drawing data acquired by the drawing data acquisition unit 24 . The input of the mirroring relationship is performed, for example, by the user specifying four points corresponding to the photographed image and the drawing data displayed on the display unit 26 with a pointing device.

The mirroring unit 54 mirrors the damaged data based on the mirroring relationship accepted by the mirroring relationship accepting unit 52 . Note that mirroring is also called mirror inversion. The direction of the axis that is the center of mirroring (hereinafter referred to as "mirroring axis") is not limited to the vertical direction of the image. That is, mirroring is not limited to so-called left-right inversion. Mirroring includes reversal other than horizontal reversal (for example, vertical reversal). Mirroring in the present invention can be said to be image processing that inverts damage data about a mirroring axis. The direction of the mirroring axis can be determined by image analysis of the input photographed image by the mirroring section 54 . Further, the mirroring of the present invention includes a mode in which the direction of the mirroring axis is fixed and a mode in which the direction of the mirroring axis is variable. The latter directional aspect will be described in detail later in this specification.

Such mirroring will be described with reference to FIGS. 7 and 8. FIG. As shown in FIG. 7, if the character "F" drawn on the upper surface of the floor slab 6 is seen through from below the floor slab 6, the letter "F" actually appears to be upside down. This is because "F" has an asymmetric shape. Similarly, general damage (for example, cracks) has an asymmetrical shape. It can be seen that the shape of the damage when the floor slab 6 is seen through from the top and the shape of the damage are in a mirroring relationship with each other. Therefore, as shown in FIG. 8, a photographed image IMG1 obtained by photographing from the lower side of the bridge 1 is converted into a mirrored image IMG2 corresponding to an image viewed from the upper side of the bridge 1 by mirroring. Then, the shape of the damage changes from the shape when viewed from the bottom of the bridge 1 (the shape of "F" in FIG. 8 is inverted upside down) to the shape when viewed from the top of the bridge 1 (the shape of "F" in FIG. ). It can be seen that the position of the damage in the image can also be converted by mirroring from the position viewed from the imaging side to the position viewed from the side opposite to the imaging side.

In the example shown in FIG. 8, since the direction of the coordinate axis Gx of the global coordinate system with the bridge 1 as a reference matches the horizontal direction (x direction) of the captured image IMG1, the captured image IMG1 is vertically inverted as mirroring. However, for example, when the direction of the coordinate axis Gx of the global coordinate system and the vertical direction (y direction) of the captured image IMG1 match, the captured image IMG1 is horizontally reversed as mirroring.

The damage drawing data creation unit 42 creates damage drawing data (damage drawing) in which the damage data projectively transformed by the projective transformation unit 36 and the drawing data corresponding to the damage data are superimposed. In addition, the damage drawing data creation unit 42 can add various types of information input from the operation unit 28 to the damage drawing data.

The damage drawing data creating unit 42 adds the input inspection result information to the damage drawing data. For example, when the type of damage and the evaluation classification of the degree of damage (also referred to as "rank information") are input via the operation unit 28, the damage drawing data creating unit 42 adds them to the damage drawing data. The "input" of the inspection result information may be a "selective input" that is an input in which a selection is made from predetermined candidates.

Further, the damage drawing data creation unit 42 has a feature amount measuring function for measuring the feature amount of damage based on the damage data or the photographed image. The damage drawing data creation unit 42 adds information indicating the measured feature amount to the damage drawing data. For example, the width, length, and spacing of crack images in the damage data of the floor slab 6 of the bridge 1 are detected as the dimensions of the local coordinate system, and the dimensions of the local coordinate system are converted to the dimensions of the global coordinate system. The display control unit 38 causes the display unit 26 to display the measured feature amount of the damage and the mirror image.

The output unit 44 outputs the mirrored and projectively transformed damage data. Further, when the damage drawing data is created by the damage drawing data creation unit 42, the output unit 44 outputs the damage drawing data.

The damage drawing data output by the output unit 44 when the damage drawing data is generated by the damage drawing data generation unit 42 will be described below. The damage drawing data is data representing the damage drawing.

FIG. 9 is a diagram showing an example of damage drawing data. This damage drawing data includes damage data indicating damage such as cracks, water leakage, and free lime for each coffer, and the member name (in this example, "floor version"), element numbers (Ds0101 to Ds0104, Ds0201 to Ds0204), types of damage ("cracks", "leakage", "free lime", etc.), and evaluation categories for the degree of damage (also called "rank information" ) and characteristic quantities of the damage (for example, width and interval of cracks). In this figure, the degree of damage is classified into five grades from "a" to "e".

Furthermore, FIG. 10 is a diagram showing an example of inverted damage drawing data obtained by inverting the damage drawing data shown in FIG. The inverted damage drawing data shown in FIG. 10 is composed of damage data mirrored by the mirroring unit 54 . Drawing data is also mirrored by the mirroring unit 54 . In the present invention, it is possible to edit (process) the damage data that constitutes the damage drawing data from the photographed side (Fig. 9) and the inverted damage drawing data (Fig. 10) viewed from the opposite side of the photographed side. is.

In the inverted damage drawing data shown in FIG. 10, non-text components such as CAD drawings and damage data are mirrored by the mirroring unit 54 . In addition, the position in the drawing is changed by the damage drawing data creation unit 42 without mirroring the text components such as the member name, element number, damage type, rank information, and damage feature amount.

FIG. 11 is a flow chart showing the operation flow of the damaged data editing device 20 of the second embodiment.

First, the image acquisition unit 22 acquires a plurality of captured images captured by the imaging device 10 (step S20). After that, the damage data generation unit 32 detects damage appearing in the captured image by performing image processing on each captured image, and generates damage data based on the detection (step S21).

Next, a plurality of photographed images acquired by the photographed image synthesizing unit 46 are synthesized (step S22). Damage data is synthesized by the damage data synthesizing unit 48 based on the result of synthesizing the photographed images by the photographed image synthesizing unit 46 (step S23).

Thereafter, the display control unit 38 overlays the captured image and the detected damage data on the display unit 26 (step S24).

While confirming the photographed image and the damage data displayed on the display unit 26, the user inputs an instruction to edit the damage data displayed on the display unit 26 using, for example, a mouse connected to a personal computer. edits the damage data based on the input edit command (step S25).

Next, the drawing data is obtained by the drawing data obtaining unit 24 (step S26). For example, the drawing data acquisition unit 24 inputs a CAD drawing showing the bridge 1 as drawing data. FIG. 12 is a diagram showing an example of a CAD drawing. “DsXXXX” (Ds0101 to Ds0105 and Ds0201 to DS0205) in the figure is an element number identifying a coffer which is an element of the floor slab 6 of the bridge 1 . Here, "Ds" is a symbol representing "floor slab", and "XXXX" is the grid arrangement position of the coffers, which is the range surrounded by the main girders 2 and cross girders 3 of the floor slab 6. is a number indicating In other words, the floor slab 6 of this example is composed of a plurality of coffers arranged in a grid pattern, and an element number is assigned to each coffer. The CAD drawing 80 of this example contains the rectangular frame information of the coffers. Element numbers DsXXXX in the drawing may be omitted from the display on the display unit 26 .

Returning to FIG. 11, the input of the projection relation is received by the projection relation receiving unit 34, and the input of the mirroring relation is received by the mirroring relation receiving unit 52 (step S27).

FIG. 13 is a diagram for explaining a specific example of input of the projection relationship and the mirroring relationship accepted by the projection relationship acceptance unit 34 and the mirroring relationship acceptance unit 52. As shown in FIG. Note that the case shown in FIG. 13 is an example in which the projection relationship and the mirroring relationship are input at the same time.

A photographed image 102 of the bridge pier 7 to be inspected is displayed on the display section 26 by the display control section 38, and a CAD drawing 80 of the building is displayed. In the photographed image 102, a coffer of the floor slab 6 is photographed, and the floor slab 6 is damaged (cracked). Also, the damage data DD3 generated by the damage data generator 32 by image analysis of the photographed image 102 to detect cracks is displayed in a superimposed manner. That is, the damage data DD3 is displayed overlaid on the damage in the photographed image 102 .

For example, the user uses the pointing device of the computer to input points corresponding to the four points of the rectangle in the captured image 100 and the CAD drawing of the bridge pier 7 displayed on the display unit 26 . That is, four points A1, B1, C1, and D1 of the photographed image 100 are specified and input, and corresponding four points A2, B2, C2, and D2 are specified and input on the CAD drawing. This input is an input of a projection relation and an input of a mirroring relation at the same time.

Next, the damage data DD3 of the captured image 100 is projectively transformed and mirrored by the projective transformation unit 36 based on the projective relationship received by the projective relationship receiving unit 34 (step S28). For example, if damage data DD3 is projectively transformed, it becomes damage data DD4.

FIG. 14 is a diagram for explaining the mirroring of each damaged data performed by the mirroring unit 54. As shown in FIG. FIG. 14 shows the coffer damage data DD5 before being mirrored. This damage data DD5 is an image showing the damage state of the bridge 1 viewed from below, and is an image showing the damage state of one coffer of the floor slab 6. FIG. FIG. 15 shows damage data DD6 obtained by mirroring the damage data DD5 of FIG. In this example, since the coordinate axis Gx of the global coordinate system with the bridge 1 as a reference and the horizontal direction (x direction) of the damage data DD5 are matched, by vertically inverting the damage data DD5 as mirroring, the damage data DD6 is obtained. That is, the direction of the mirroring axis MAx in this example is the horizontal direction (x direction) of the damage data DD5.

Returning to FIG. 11, after that, the output unit 44 outputs the damage drawing data composed of the projectively transformed damage data (step S29).

Here, a specific example of creation of damage drawing data performed by the damage drawing data creation unit 42 will be described.

FIG. 16 is a diagram for explaining the creation of damage drawing data performed by the damage drawing data creating unit 42. As shown in FIG. The display unit 26 displays the damage drawing data in which the damage data DD6 generated by the damage drawing data creation unit 42 is superimposed on the CAD drawing 80 and displayed. FIG. 16 shows an example in which the damage data DD6 shown in FIG. 15 is superimposed on the CAD drawing 80 shown in FIG. Note that alignment between the CAD drawing 80 and the damage data DD6 is performed by a known technique. For example, the CAD drawing 80 and the damage data DD6 are aligned based on the coordinate data of the CAD drawing 80 and the coordinate data of the damage data DD6.

Next, the damage drawing data creation unit 42 measures the feature amount of the damage based on the damage data DD6. For example, the width and spacing of cracks are measured as feature quantities.

Next, the damaged drawing data creating unit 42 edits each element of the damaged drawing data. FIG. 17 is a diagram showing a display example of an edit screen relating to a coffer to which the element number Ds0202 of the floor slab 6 is assigned. In this example, the display unit 26 is caused to display the CAD drawing 80 and the damage data DD6 as well as the damage list 82 . In this example, since a crack is detected based on the damage data DD6 in the crate of the element number Ds0202, the "type" (type of damage) of "crack" is added to the damage list by the damage drawing data creation unit 42. 82 is automatically filled in. In addition, since the width and interval of the "crack" are measured by the damage drawing data creation unit 42 as the characteristic amount of the "crack" in the crate of the element number Ds0202, the damage drawing data creation unit 42 " "Dimensions" column will be filled in automatically. Items that have not been automatically entered by the damage drawing data creation unit 42 can be entered by the user through the operation unit 28 . For example, when water leakage and free lime can be visually recognized in the damage data DD6, the user can input the type of "water leakage + free lime" via the operation unit 28 .

Further, when an automatically undetected crack can be visually recognized in the damage data DD6, the user can input the type and dimensions of the crack via the operation unit . Although not shown in FIG. 17, a scale image for measuring the crack dimension can be displayed on the display unit 26, and the scale image can be used to measure the dimension of an undetected crack. is. In order to reflect undetected cracks in the damage drawing data, the operation unit 28 can perform a tracing operation on the damage image in the damage data DD6. For example, if the operation unit 28 is composed of a touch panel, if the crack image in the damage data DD6 is traced with a finger or a pen, the locus of the trace is added to the damage drawing data as damage data. Here, the shape and position of the crack image in the damage data DD6 in the CAD drawing 80 correspond to the actual shape and position of the crack when the bridge 1 is viewed from above (the side opposite to the photographing side). can be entered easily and accurately.

"Category" of the damage list 82 is a column for inputting an evaluation category (also referred to as "rank information") of the degree of damage. FIG. 18 is a diagram schematically showing an example of a crack progress model in the floor slab 6. As shown in FIG. In this progression model, the evaluation category RANK is expressed in five stages from "a" to "e". Correspondence between each evaluation category RANK and crack state STATE is as follows.

a: No damage.

b: A state in which a plurality of cracks are generated in parallel along the lateral direction (transverse direction perpendicular to the vehicle traffic direction). This is the stage where multiple cracks due to drying shrinkage form parallel beams.

c: Cracks in the longitudinal direction (longitudinal direction parallel to the direction of travel of the vehicle) and transverse cracks intersect each other. This is the stage where multiple cracks form a grid due to the live load and the crack density in the grid region increases. In the latter half of the period, the crack penetrates the floor slab 6 in the vertical direction (perpendicular to the lower surface of the floor slab).

d: A state in which the crack density in the lattice-like region exceeds a specified value, and the fracture surfaces of a plurality of penetrating cracks are smoothed. This is the stage where the floor slab 6 loses shear resistance due to the polishing action.

e: A state in which omission has occurred. Wheel loads exceeding the reduced punch shear strength cause pull-out.

In this manner, the damaged drawing data generation unit 42 edits the damaged drawing data as a whole. Further, for example, the damage drawing data creation unit 42 may input the soundness determination classification for the entire floor slab 6 to the damage drawing data via the operation unit 28 .

Next, the output unit 44 outputs the damaged drawing data edited by the damaged drawing data creating unit 42 . The damaged drawing data can be displayed on the display unit 26 by the display control unit 38, and can also be printed out on a printer (not shown) via the network. For example, the damaged drawing data shown in FIG. 9 is printed out by a printer. Also, the damaged drawing data can be uploaded to a database (not shown) via a network.

<Mirroring variations>
In the above-described second embodiment, the case in which the direction of the mirroring axis is fixed has been described as an example in order to facilitate understanding of the present invention. is not limited to The present invention may have a configuration in which the direction of the mirroring axis is variable.

The mirroring section 54 has a function of determining the direction of the mirroring axis based on various information. The mirroring unit 54 performs mirroring around the mirroring axis in the determined direction.

Firstly, there is a mode in which an image (which is an image before mirroring) input by the image acquisition unit 22 is image-analyzed, and the direction of the mirroring axis is determined based on the result of the image analysis.

FIG. 19 is a diagram showing a photographed image in which a coffer is photographed. For example, assume that a photographed image IMG21 of a coffer of the floor slab 6 shown in FIG. 19 is obtained. This photographed image IMG21 includes an image EG of the edge of the coffer (hereinafter referred to as "edge image"). In real space, the shape of the edge of a coffer is a rectangle, and the lengths of two mutually orthogonal sides forming the rectangle are generally different. That is, the edge rectangle has long sides and short sides that are perpendicular to each other. In the real space, the two sides of the edge rectangle that are orthogonal to each other are oriented in the same direction as the two coordinate axes Gx and Gy in the global coordinate system with the bridge 1 as a reference. Therefore, the mirroring unit 54 detects an edge image from the captured image IMG21, performs projective transformation of the edge image into a rectangle as necessary, and then converts one of the mutually orthogonal long and short sides of the rectangle to The direction of the mirroring axis can be determined based on the direction of . Of course, the object to be photographed (the object to be inspected) of the present invention is not limited to the crate of the floor slab 6, and the direction of the mirroring axis may be determined based on the geometric features of other objects to be photographed. stomach.

Secondly, the coordinate axes of the global coordinate system (“first coordinate system”) based on the building and the local coordinate system (“second coordinate system”) based on the imaging device 10 that captured the image ), the direction of the mirroring axis is determined based on the angle formed by the coordinate axes.

For example, as shown in FIG. 20, an image captured by the imaging device 10 mounted on the robot device 90 is input by the image acquiring unit 22, and the imaging device control information according to the imaging device control of the robot device 90. is input by the image acquisition unit 22 . As shown in FIG. 21, this imaging device control information includes coordinate axes Gx, Gy, and Gz of a global coordinate system with the bridge 1 as a reference, and coordinate axes Lx, Ly, and Lz of a local coordinate system with the imaging device 10 as a reference. It contains angle information indicating the angle between the two. In the local coordinate system of this example, one coordinate axis Lz of the three mutually orthogonal coordinate axes Lx, Ly, and Lz is the photographing direction (also referred to as the optical axis direction) of the photographing device 10, and is the floor slab 6 of the bridge 1. (which is the surface to be photographed). That is, in this example, the direction of one coordinate axis Gz of the global coordinate system and the direction of one coordinate axis Lz of the local coordinate system match.

The robot device 90 of this example is a suspended robot suspended from the bridge 1, and FIG. The suspension type robot may be a robot that suspends from a running body (vehicle) on the bridge instead of suspending from the bridge 1 . Also, the present invention can be applied to a case where an unmanned flying robot such as a drone (unmanned flying object) is used instead of the suspended robot.

The robot device 90 of this example includes an imaging device control mechanism 92 that controls movement and rotation of the imaging device 10 . The photographing device control mechanism 92 includes an XYZ movement mechanism capable of moving the photographing device 10 in each of the Gx, Gy, and Gz directions orthogonal to each other, and rotating the photographing device 10 in each of the pan direction P and tilt direction T. It also serves as a possible pan-tilt mechanism.

The mirroring unit 54 can determine the direction of the mirroring axis based on the angle information indicating the angle between the coordinate axis Gx (or Gy) of the global coordinate system and the coordinate axis Lx (or Ly) of the local coordinate system. .

Further, when the photographing device 10 is rotated in the panning direction P, that is, when the azimuth angle of the photographing device 10 is variable with respect to the global coordinate system with the bridge 1 as a reference, the mirroring unit 54 moves the coordinate axes of the global coordinate system. Based on the azimuth angle of imager 10 with respect to Gx or Gy, it is possible to determine the direction of the mirroring axis.

Note that there are various modes of setting the coordinate axes Lx, Ly, and Lz of the local coordinate system with respect to the coordinate axes Gx, Gy, and Gz of the global coordinate system, and pan-tilt modes (angle control modes) of the photographing device 10. It is only necessary to obtain angle information according to the setting mode of the coordinate axes of the system and the angle control mode of the imaging device 10, and determine the direction of the mirroring axis based on the angle information.

<Client-server type>
In a client-server system, the damaged data editing device 20 may be configured by at least one of a client device and a server device. In other words, the damaged data editing device 20 of the present invention may be configured by one or more computer devices in a client-server system.

Although the embodiments for carrying out the present invention have been described above, the present invention is not limited to the above-described embodiments and modifications, and various modifications are possible without departing from the gist of the present invention.

1 Bridge 2 Main Girder 3 Cross Girder 4 Tilting Structure 5 Horizontal Structure 6 Floor Slab 7 Pier 10 Imaging Device 20 Damage Data Editing Device 22 Image Acquisition Unit 24 Drawing Data Acquisition Unit 26 Display Unit 28 Operation Unit 30 CPU
32 Damage data generation unit 34 Projection relationship reception unit 36 Projection transformation unit 38 Display control unit 40 Editing unit 42 Damage drawing data generation unit 44 Output unit 46 Photographed image synthesis unit 48 Damage data synthesis unit 50 Storage unit 52 Mirroring relationship reception unit 54 Mirroring Part 82 Damage list 90 Robot device 92 Imaging device control mechanism Steps S10 to S17 Damage data editing process of the first embodiment Steps S20 to S29 Damage data editing process of the second embodiment

Claims (10)

A damage data editing device comprising a processor and a storage unit for storing drawing data of a building to be inspected,
The processor
Acquiring a composite image synthesized from a plurality of photographed images obtained by separately photographing the building to be inspected,
Damage data expressing damage detected by image analysis of the plurality of captured images before the plurality of captured images are combined, the damage combined based on the combined information of the combined image. get the data,
Acquiring the drawing data of the building to be inspected from the storage unit;
Receiving an input of a projection relationship between the synthesized image and the acquired drawing data;
projectively transforming the synthesized damage data based on the received projective relationship;
outputting the combined damage data that has been projectively transformed;
Damage data editor. further comprising a display,
The processor
displaying the synthesized image on the display unit and displaying the synthesized damage data overlaid on the synthesized image;
receiving a command to edit the synthesized damage data displayed on the display unit, and editing the synthesized damage data based on the command to edit;
projectively transforming the edited synthesized damage data based on the received projective relationship;
The damage data editing device according to claim 1. further comprising a display,
The processor
causing the display unit to display the drawing data;
3. The damage data editing device according to claim 1, wherein four points of said synthesized image and four points of said drawing data, which indicate said projection relationship between said synthesized image displayed on said display unit and said drawing data, are received. The processor
4. Damage data editing according to claim 3 , wherein a projective transformation matrix is calculated for matching the four points of the synthesized image with the four points of the drawing data, and the synthesized damage data synthesized by the calculated projective transformation matrix is subject to projective transformation. Device. The processor
receiving an input of a mirroring relationship between the composite image and the drawing data;
mirroring the combined damage data based on the received mirroring relationship;
5. The damage data editing device according to any one of claims 1 to 4, which outputs the combined damage data that has undergone mirroring and projective transformation. The processor
Creating damage drawing data by superimposing the synthesized damage data that has undergone projective transformation and the drawing data corresponding to the synthesized damage data,
6. The damage data editing device according to claim 1, which outputs the damage drawing data. 7. The damage data editing device according to any one of claims 1 to 6, wherein the structure to be inspected is a bridge or a tunnel. A system comprising the damage data editing device according to any one of claims 1 to 7 and an image processing device,
The processor of the image processing device,
Acquiring a plurality of captured images obtained by dividing and capturing the building,
performing image synthesis processing on the plurality of captured images;
Before combining the plurality of captured images, damage is detected by image analysis of the plurality of captured images, and damage data representing the damage is generated;
synthesizing the damage data based on the synthesis information of the photographed images obtained in the image synthesizing process;
outputting the synthesized damage data and a synthesized image, which is a photographed image subjected to the image synthesis processing, to the damage data editing device;
The processor of the damaged data compilation device comprises:
obtaining the composite image;
obtaining the combined damage data;
acquiring the drawing data of the building from the storage unit;
Receiving an input of a projection relationship between the synthesized image and the acquired drawing data;
projectively transforming the synthesized damage data based on the received projective relationship;
outputting the combined damage data that has been projectively transformed;
system. A damage data editing method for a damage data editing device comprising a processor and a storage unit for storing drawing data of a building to be inspected,
The processor
acquiring a composite image synthesized from a plurality of photographed images obtained by separately photographing the building to be inspected;
Damage data representing damage detected by image analysis of the plurality of captured images before the plurality of captured images are combined, the damage combined based on the combined information of the combined image. obtaining data;
a step of acquiring drawing data of the building to be inspected from the storage unit;
receiving an input of a projection relationship between the composite image and the acquired drawing data;
projectively transforming the combined damage data based on the received projective relationship;
outputting the composite damage data that has been projectively transformed;
damage data compilation method. A program for a damage data editing device comprising a processor and a storage unit for storing drawing data of a building to be inspected,
to the processor;
acquiring a composite image synthesized from a plurality of photographed images obtained by separately photographing the building to be inspected;
Damage data expressing damage detected by image analysis of the plurality of captured images before the plurality of captured images are combined, the damage combined based on the combined information of the combined image. obtaining data;
a step of acquiring drawing data of the building to be inspected from the storage unit;
receiving an input of a projection relationship between the composite image and the acquired drawing data;
projectively transforming the combined damage data based on the received projective relationship;
outputting the composite damage data that has been projectively transformed;
A program that allows you to do

Images