JP2023139821A - Image management system and method - Google Patents

Abstract

To provide an image management technique capable of easily managing each image in a case of imaging the same subject with a plurality of cameras having different field angles.SOLUTION: An image management system 1 comprises a management computer 3 which manages each image captured by a first camera 4 and a second camera 5 that can perform imaging in a wider field angle than the first camera 4. The management computer 3 acquires a first image 13 captured by the first camera 4 and a second image 14 captured by the second camera 5, records the first image 13 in association with a coordinate position being the coordinate position in the second image 14 and matching with the imaging direction of the first camera 4 when the imaging direction of the first camera 4 is included in an imaging range of the second camera 5, and displays the first image 13 recorded in association with the selected coordinate position when an arbitrary coordinate position of the second image 14 is selected.SELECTED DRAWING: Figure 2

Description

Embodiments of the present invention relate to image management technology.

At power plants and other facilities, regular patrol inspections are conducted to manage construction progress and detect abnormalities. During patrol inspections, images are taken with a camera to record the inspection points. When using a camera with a fisheye lens for photographing, a wide range of images can be acquired, and even if there are multiple inspection points, they can be captured in one image. Furthermore, it is also possible to use a camera that can take pictures in all directions by arranging a plurality of cameras each having a fisheye lens. On the other hand, when a detailed image of an inspection area is taken, the image is taken with the camera close to the inspection area, or by using the zoom function of the camera. The photographed inspection points are managed by recording them on recording paper or electronic media using a tablet PC or the like. Inspection workers analyze the captured images and detect abnormalities such as damage, deformation, cracks, discoloration, and leakage of internal liquid in the inspection area.

Japanese Patent Application Publication No. 2005-192057

When detecting an abnormality at an inspection location from a captured image, it is desirable to zoom in on the inspection location to obtain a more detailed image. However, when a plurality of inspection points have similar shapes, it may be difficult to identify which inspection point appears in a photographed image. In particular, when the same subject is photographed using a wide-angle image taken with a camera with a wide-angle lens or a fisheye lens and a detailed image taken with a camera with a zoom function, it is difficult to manage each image. There is a desire to do this more easily.

The embodiments of the present invention have been made in consideration of such circumstances, and provide images that allow easy management of each image when the same subject is photographed using multiple cameras with different angles of view. The purpose is to provide management technology.

An image management system according to an embodiment of the present invention includes a management computer that manages images taken by a first camera and a second camera capable of taking images at a wider angle of view than the first camera, and The computer acquires a first image taken by the first camera and a second image taken by the second camera, and determines whether the photographing direction of the first camera is included in the photographing range of the second camera. , the first image is recorded in association with the coordinate position in the second image that coincides with the photographing direction of the first camera, and any of the coordinates of the second image is recorded. When a position is selected, the first image recorded in association with the selected coordinate position is displayed.

Embodiments of the present invention provide an image management technique that allows easy management of each image when the same subject is photographed using multiple cameras with different angles of view.

FIG. 1 is a block diagram showing the overall configuration of an image management system. FIG. 3 is a block diagram showing a management computer. FIG. 2 is a side view showing a patrol inspection robot. An explanatory diagram showing the positional relationship of detailed images in an omnidirectional image. FIG. 3 is an explanatory diagram showing a mode of setting corresponding points in an omnidirectional image. FIG. 3 is an explanatory diagram showing the relationship between the pan-tilt angle of a camera and the coordinates of an omnidirectional image. FIG. 3 is a screen diagram showing how images are displayed in the image management system.

Embodiments of an image management system and an image management method will be described in detail below with reference to the drawings.

Reference numeral 1 in FIG. 1 is an image management system of this embodiment. This image management system 1 is used, for example, to photograph objects such as equipment or structures installed at a predetermined inspection site, and to perform various inspections based on the photographed images. The image management system 1 mainly takes images at an inspection site and manages a large number of images taken. Inspection sites include power plants, chemical plants, and factories. These plants have a large number of equipment or structures that are subject to inspection.

The image management system 1 includes a patrol inspection robot 2 and a management computer 3. The patrol inspection robot 2 includes a pan-tilt-zoom camera 4 as a first camera, an omnidirectional camera 5 as a second camera, a photographing direction detection section 6, a distance measurement section 7, a position detection section 8, and a traveling device 9. and a control computer 10 that controls these in an integrated manner. Note that the control computer 10 is connected to the management computer 3.

The control computer 10 and the management computer 3 have hardware resources such as a CPU, ROM, RAM, and HDD, and when the CPU executes various programs, information processing by software is realized using the hardware resources. It is. Furthermore, the image management method of this embodiment is realized by causing a computer to execute various programs.

The patrol inspection robot 2 travels within the inspection site according to a preset route and photographs the inspection site. The images taken by this patrol inspection robot 2 are sent to the management computer 3. An inspection worker (user) can inspect the equipment or structure at the inspection site by checking the images stored in the management computer 3.

An example of an image managed by the management computer 3 is a still image. Note that the images may include moving images, or may include both still images and moving images.

This patrol inspection robot 2 will be explained using FIG. 3. Note that the left side of the paper in FIG. 3 will be described as the front side (front side), which is the traveling direction of the patrol inspection robot 2.

The patrol inspection robot 2 is capable of traveling on the ground using a traveling device 9 having wheels 11. The patrol inspection robot 2 runs under autonomous control, for example. Note that the patrol inspection robot 2 may be driven by remote control by an inspection worker.

A pan-tilt-zoom camera 4 as a first camera is provided on the top of the patrol inspection robot 2. This pan-tilt-zoom camera 4 has a pan function that allows swinging in the horizontal direction, a tilt function that allows swinging in the vertical direction, and a zoom function that allows zooming in (telephoto) and zooming out (wide angle). This pan-tilt-zoom camera 4 can take detailed images of objects such as equipment or structures at the inspection site. The detailed image 13 (FIG. 7) taken by this pan-tilt-zoom camera 4 is the first image of this embodiment.

The photographing direction detection unit 6 (FIG. 1) detects the photographing direction of the pan-tilt-zoom camera 4. The photographing direction is the direction of the pan-tilt-zoom camera 4 and corresponds to the center position of the detailed image 13.

The photographing direction detecting section 6 is composed of, for example, a plurality of sensors provided at movable parts of the pan-tilt-zoom camera 4. Note that the photographing direction detection section 6 may be configured with a plurality of motors that move the pan-tilt-zoom camera 4. For example, the direction of the pan-tilt-zoom camera 4 is specified based on the rotation speed (rotation angle) of the motor. The photographing direction of the pan-tilt-zoom camera 4 detected by the photographing direction detection section 6 is input to the control computer 10.

Photographing direction information 15 (FIG. 2) indicating the photographing direction of the pan-tilt-zoom camera 4 is sent from the control computer 10 to the management computer 3. This photographing direction information 15 may be, for example, a detection signal output from a sensor (photographing direction detection unit 6) that detects the photographing direction, or a control signal output from the control computer 10 that controls the photographing direction. In this way, even if the shooting direction of the pan-tilt-zoom camera 4 changes, the management computer 3 can specify the shooting direction.

An omnidirectional camera 5 as a second camera is provided near the pan-tilt-zoom camera 4 (first camera). Note that the relative positions of the pan-tilt-zoom camera 4 and the omnidirectional camera 5 are fixed at a certain distance. Further, the pan-tilt-zoom camera 4 is provided at a slightly higher position than the omnidirectional camera 5. In this way, the omnidirectional camera 5 is not included in the photographing range of the pan-tilt-zoom camera 4 and does not get in the way.

The omnidirectional camera 5 (second camera) is capable of photographing at a wider angle of view than the pan-tilt-zoom camera 4 (first camera). For example, the angle of view of the omnidirectional camera 5 is wider than that when the pan-tilt-zoom camera 4 is zoomed out.

This omnidirectional camera 5 has a wide-angle lens or a fisheye lens and is capable of photographing in all directions. In this way, the omnidirectional camera 5 can photograph a wide range. Then, the omnidirectional camera 5 can grasp the location where the subject photographed by the pan-tilt-zoom camera 4 exists.

For example, the omnidirectional camera 5 is equipped with two wide-angle cameras 12 each having an image sensor with a fisheye lens. These wide-angle cameras 12 can photograph a wide range around the omnidirectional camera 5.

The two wide-angle cameras 12 are arranged back to back and can simultaneously photograph the scenery in front and behind. That is, using the omnidirectional camera 5, it is possible to simultaneously capture an omnidirectional image 14 (FIG. 7), which is a 360-degree panoramic image that captures all directions, top, bottom, left, and right of the patrol inspection robot 2. The omnidirectional image 14 taken by this omnidirectional camera 5 is the second image of this embodiment. Note that the second image does not need to be the omnidirectional image 14, but may simply be a wide-angle image with a wider angle of view than the detailed image 13.

In this embodiment, two wide-angle cameras 12 are illustrated as the omnidirectional camera 5 (second camera), but the omnidirectional camera 5 may be configured using one camera. For example, the omnidirectional image 14 may be captured by guiding the surrounding scenery to one camera using a convex mirror or the like.
I will explain about it.

If the omnidirectional camera 5 is used, it is possible to take images in all directions with one omnidirectional camera 5 without changing its orientation at the inspection site. Alternatively, the omnidirectional image 14 may be captured by using two or more cameras each having one image sensor and combining the images captured by these cameras.

The omnidirectional image 14 is, for example, a spherical image. Note that the omnidirectional image 14 does not have to be an omnidirectional image, but may be one that captures at least a 360 degree range in the horizontal direction (all directions left and right).

A distance measuring section 7 is provided near the omnidirectional camera 5. This distance measuring section 7 measures the distance from the omnidirectional camera 5 to the subject. Note that if the relative positional relationship between the pan-tilt-zoom camera 4 (first camera) and the omnidirectional camera 5 (second camera) is determined in advance, the pan-tilt zoom camera 4 (first camera) and the omnidirectional camera 5 (second camera) are The distance from the zoom camera 4 to the subject can be estimated. That is, the distance measurement unit 7 measures the distance from at least one of the pan-tilt-zoom camera 4 and the omnidirectional camera 5 to the subject.

The distance measuring section 7 is composed of a laser distance meter or the like that can change the direction in conjunction with the pan/tilt mechanism (pan/tilt function) of the pan/tilt/zoom camera 4. Further, the distance measuring section 7 may measure the distance to the subject based on focus information of a predetermined camera having a pan/tilt mechanism.

Note that the distance measurement unit 7 may measure the distance to the subject by laser scanning the inspection site using a laser sensor such as an infrared sensor or LiDAR. For example, the distance measurement unit 7 can measure the distance to the object by projecting a laser beam onto the object and receiving the reflected light with a light receiving element. Further, the distance measurement unit 7 can measure the distance to surrounding objects using a ToF (Time of Flight) method that converts the delay time of the light reception pulse with respect to the light emission pulse into distance.

Furthermore, a position detection section 8 is provided on the upper part of the patrol inspection robot 2. This position detection unit 8 detects the current position (photographing position) of the patrol inspection robot 2. This position detection unit 8 detects its own position using, for example, a satellite positioning system, radar, or the like.

Note that the position detection unit 8 may detect its own position using a radio wave signal or the like used for transmitting and receiving information. For example, the position detection unit 8 may acquire the position information 16 using wireless communication. Further, the position detection unit 8 may acquire the position information 16 by PDR using an acceleration sensor or the like.

Further, the position detection unit 8 may detect its own position using known techniques such as SLAM (Simultaneous Localization and Mapping) and SfM (Structure from Motion). Furthermore, the position detection unit 8 may detect its own position using a predetermined position marker placed on the ground.

Since the omnidirectional camera 5 (second camera) can capture images over a wide range, it is easy to grasp the location of the inspection target (object) such as equipment or structures. There is also the drawback that you cannot take pictures. On the other hand, when the pan-tilt-zoom camera 4 (first camera) faces the direction of the inspection target, it can photograph only the inspection target using a zoom function or the like. As a result, high-definition images can be obtained, but on the other hand, since the images obtained are enlarged, it becomes difficult to grasp the location of the inspection target at the inspection site (for example, inside a building), making it difficult to manage the images. This creates a burden on the inspection worker (user). In particular, it is desired that the positional relationship of the inspection target shown in the omnidirectional image 14 (second image) and the detailed image 13 (first image) be managed in association with each other, so that the management of the inspection target can be realized more easily. There is. This embodiment can solve such problems.

As shown in FIG. 4, the management computer 3 controls the omnidirectional image 14 ( The detailed image 13 (first image) is recorded (registered) in association with the coordinate position in the second image) that coincides with the photographing direction of the pan-tilt-zoom camera 4. Then, when an arbitrary coordinate position of the omnidirectional image 14 is selected, the management computer 3 displays the detailed image 13 recorded in association with the selected coordinate position.

For example, as shown in FIG. 7, it is assumed that a large number of meters 28 and valves 29 are present at the inspection site. Here, the omnidirectional image 14 is a wide range in which a large number of meters 28 and bulbs 29 are photographed. On the other hand, the detailed image 13 is a narrow range in which the meter 28 and the bulb 29 are individually photographed.

When displaying the omnidirectional image 14 on the display, the management computer 3 causes a predetermined mark 27 to be displayed near the meter 28 and valve 29 to be inspected. The inspection worker (user) selects any mark 27 using the mouse cursor 30. The management computer 3 displays the detailed image 13 corresponding to the selected mark 27 on the display. For example, in the omnidirectional image 14, even if the numerical value of the meter 28 cannot be read, the inspection worker can display the detailed image 13 in which the meter 28 is enlarged by selecting the mark 27 corresponding to the meter 28. I can do it. In this way, the numerical value on the meter 28 can be read.

Next, the system configuration of the management computer 3 will be explained with reference to the block diagram shown in FIG. Note that the arrows in FIG. 2 are an example of a flow of processing, and there may be a flow of processing other than the arrows. Further, the context of each process is not necessarily fixed, and the context of some processes may be switched. Further, some processes may be executed in parallel with other processes. Furthermore, the management computer 3 may include components other than the configuration shown in FIG. 2, or some of the configuration shown in FIG. 2 may be omitted.

The management computer 3 receives a detailed image 13 (first image), an omnidirectional image 14 (second image), as well as shooting direction information 15, position information 16, and distance information 17.

The photographing direction information 15 is information indicating the photographing direction of the pan-tilt-zoom camera 4 (first camera) detected by the photographing direction detection unit 6 (FIG. 1). For example, the photographing direction information 15 includes angle information indicating the orientation of the pan-tilt-zoom camera 4, such as a pan angle or a tilt angle.

Note that the photographing direction information 15 does not necessarily have to be input from the control computer 10 to the management computer 3. For example, the management computer 3 may estimate the pan angle or tilt angle based on the operation of the pan-tilt mechanism of the pan-tilt-zoom camera 4 that is reflected in the omni-directional image 14 taken by the omni-directional camera 5. The photographing direction information 15 may be acquired in this manner.

The position information 16 (FIG. 2) is the position of the patrol inspection robot 2 detected by the position detection unit 8, and is information indicating the photographing position. The distance information 17 (FIG. 2) is the distance to the subject measured by the distance measuring unit 7, and is information indicating the distance from at least one of the pan-tilt-zoom camera 4 and the omnidirectional camera 5 to the subject.

The management computer 3 includes a first image acquisition section 18 , a second image acquisition section 19 , a distortion correction section 20 , a photographing position estimation section 21 , and a display control section 22 . These are realized by the CPU executing programs stored in the memory or HDD.

Furthermore, the management computer 3 includes an image database 23. This image database 23 is a collection of information stored in a memory, HDD, or cloud and organized so that it can be searched or stored.

Each component of the management computer 3 does not necessarily need to be provided in one computer. For example, one management computer 3 may be realized by a plurality of computers connected to each other via a network. For example, the image database 23 may be installed in another computer.

Although not particularly illustrated, the management computer 3 includes an input section and an output section. Predetermined information is input into the input unit according to a user's operation. This input unit includes an input device such as a mouse or a keyboard. That is, predetermined information is input to the input unit in response to operations on these input devices.

The output unit outputs predetermined information. The management computer 3 includes a device such as a display that displays images. Note that the display may be separate from the computer main body, or may be integrated with the computer main body. Additionally or alternatively, the management computer 3 may control images displayed on displays of other computers connected via the network.

The first image acquisition unit 18 acquires a detailed image 13 (first image) taken by the pan-tilt-zoom camera 4 (first camera) from the control computer 10 (FIG. 1), and also Photographing direction information 15 indicating the photographing direction of the pan-tilt-zoom camera 4 at the time is acquired. In this way, the photographing direction of the pan-tilt-zoom camera 4 can be acquired, and based on this, the corresponding coordinate position in the omnidirectional image 14 can be specified.

Note that if the shooting direction of the pan-tilt-zoom camera 4 can be estimated based on the subject shown in the detailed image 13 or the like, the management computer 3 does not need to acquire the shooting direction information 15 from the control computer 10. For example, the management computer 3 may generate the photographing direction information 15 based on the detailed image 13.

The second image acquisition unit 19 acquires an omnidirectional image 14 (second image) taken by the omnidirectional camera 5 (second camera) from the control computer 10 (FIG. 1).

The distortion correction unit 20 performs image processing to correct distortion of the omnidirectional image 14 acquired by the second image acquisition unit 19. The distortion correction unit 20 performs processing to correct an image distorted by, for example, a fisheye lens into an image without distortion similar to the detailed image 13.

The photographing position estimating unit 21 acquires a detailed image 13, an omnidirectional image 14, position information 16, and distance information 17.

This photographing position estimating unit 21 determines whether a detailed image 13 (first image) when a predetermined direction is photographed by the pan-tilt-zoom camera 4 (first camera) is photographed by the omnidirectional camera 5 (second camera). A process is performed to estimate which coordinate position in the omnidirectional image 14 (second image) corresponds. That is, when the shooting direction of the pan-tilt-zoom camera 4 (for example, the center position of the shot image) is included in the shooting range of the omnidirectional camera 5, the shooting position estimating unit 21 calculates the coordinates in the omnidirectional image 14. Processing is performed to associate the detailed image 13 with a coordinate position that coincides with the photographing direction of the pan-tilt-zoom camera 4.

For example, using the pan-tilt-zoom camera 4, images are taken a plurality of times at different pan angles and tilt angles. Then, the detailed images 13 acquired by these photographs are associated with the coordinate positions in the omnidirectional image 14 photographed by the omnidirectional camera 5. Note that this association process may be performed automatically by the management computer 3, or may be performed manually by the user.

A mode in which settings are performed manually by the user will be described using FIG. 5. First, the user selects, for example, a partial range 24 that is included in the photographing range of the omnidirectional image 14 and is estimated to include the photographing direction of the pan-tilt-zoom camera 4. . The user manually sets corresponding points 25 on the omnidirectional image 14 while checking the center position (photographing direction) of the detailed image 13 photographed by the pan-tilt-zoom camera 4. These corresponding points 25 are set, for example, at positions where the pan angle and tilt angle of the pan-tilt-zoom camera 4 are each shifted by one degree. Furthermore, grid lines 26 are set that connect the respective corresponding points 25 and complement the coordinates between the corresponding points 25. Based on these corresponding points 25 and grid lines 26, coordinates can be converted.

By performing such processing in advance, the coordinate position of the omnidirectional image 14 can be converted from the pan angle and tilt angle of the pan-tilt-zoom camera 4. Furthermore, the pan angle and tilt angle of the pan-tilt-zoom camera 4 can be converted from the coordinate position of the omnidirectional image 14. A conversion table or conversion formula that enables these conversions is generated.

For example, as shown in FIG. 6, there is a first coordinate (coordinates on the left side of the paper in FIG. 6) where the pan angle of the pan-tilt-zoom camera 4 is the horizontal axis (p-axis) and the tilt angle is the vertical axis (t-axis). It is assumed that there are second coordinates (coordinates on the right side of the paper in FIG. 6) in which the horizontal direction of the omnidirectional image 14 is the horizontal axis (x-axis) and the vertical direction is the vertical axis (y-axis).

Here, it is assumed that there are four known corresponding points d corresponding to corners of arbitrary squares at the first coordinates. Four known corresponding points b corresponding to these known corresponding points d also exist in the second coordinates. The respective known corresponding points b and d are associated in advance. Once an arbitrary corresponding point d is determined at the first coordinates, one corresponding point b is specified at the second coordinates. Furthermore, once an arbitrary corresponding point b is determined at the second coordinates, one corresponding point d is specified at the first coordinates. A conversion table or conversion formula that allows such coordinate conversion is generated in advance.

The photographing position estimation unit 21 performs a process of converting photographing direction information 15 indicating the photographing direction of the pan-tilt-zoom camera 4 into a coordinate position of the omnidirectional image 14 based on at least one of a conversion table and a conversion formula. In this way, the process of converting the photographing direction of the pan-tilt-zoom camera 4 into the coordinate position of the omnidirectional image 14 can be easily performed.

Note that if the distances from the pan-tilt-zoom camera 4 (first camera) and the omnidirectional camera 5 (second camera) to the subject are different, the spatial coordinates centered on the pan-tilt-zoom camera 4 and the omnidirectional camera The spatial coordinates centered on 5 are different. Therefore, the conversion table or conversion formula is configured so that the conversion can be performed even under conditions where the distance to the subject is different.

The image database 23 records (stores) the detailed image 13 (first image) and the omnidirectional image 14 (second image) that have been associated in the photographing position estimation unit 21. Note that the management computer 3 may analyze the detailed images 13 stored in the image database 23 and automatically perform a process of detecting an abnormality in the inspection target based on the analysis result.

When an arbitrary coordinate position of the omnidirectional image 14 (second image) displayed on the display is selected by the user, the display control unit 22 records it in the image database 23 in association with the selected coordinate position. A process is performed to display the detailed image 13 (first image) shown on the display.

For example, the display control unit 22 performs a process of displaying a mark 27 from which the coordinates can be selected in the omnidirectional image 14 captured by the omnidirectional camera 5 based on information on the coordinates estimated by the imaging position estimation unit 21. I do. When the inspection worker (user) selects the mark 27 in the omnidirectional image 14 displayed on the display, the display control unit 22 performs control to display the corresponding detailed image 13.

Note that if the center of the pan-tilt-zoom camera 4 (first camera) (the physical center of the device) and the center of the omnidirectional camera 5 (second camera) are at the same position, the pan-tilt angle of the pan-tilt-zoom camera 4 and The coordinate position of the omnidirectional image 14 can be converted by a simple calculation. However, due to physical constraints when attaching to the patrol inspection robot 2, the pan-tilt-zoom camera 4 and the omnidirectional camera 5 are fixed at a certain distance. Therefore, in this embodiment, a process is performed to obtain the relationship between the pan-tilt angle of the pan-tilt-zoom camera 4 and the coordinate position of the omnidirectional image 14 in advance.

Further, the photographing position estimating unit 21 estimates a coordinate position in the omnidirectional image 14 that corresponds to the photographing direction of the pan-tilt-zoom camera 4 based on the acquired distance information 17. In this way, the accuracy of identifying the correspondence between the detailed image 13 and the omnidirectional image 14 can be improved based on the distance from the pan-tilt-zoom camera 4 or the omnidirectional camera 5 to the subject.

As shown in FIG. 7, the photographing position estimation unit 21 (FIG. 2) estimates the photographing position based on the input of the photographing direction information 15 of the pan-tilt-zoom camera 4. The display control unit 22 (FIG. 2) displays the mark 27 at the corresponding coordinate position in the omnidirectional image 14. When the inspection worker (user) selects the mark 27 with the mouse cursor 30, the display control unit 22 extracts the detailed image 13 corresponding to the selected coordinate position from the image database 23 and displays it on the display. Display. In this way, it becomes possible to link the detailed image 13 taken by the pan-tilt-zoom camera 4 and the omnidirectional image 14 taken by the omnidirectional camera 5, making it easy to manage images showing inspection targets. can be done.

Note that when a coordinate position other than the mark 27 is selected with the mouse cursor 30, the detailed image 13 corresponding to this coordinate position may be displayed on the display. Furthermore, the mark 27 does not need to be displayed on the omnidirectional image 14, as long as any coordinate position can be selected. Further, the coordinate position may be selected using a method other than the mouse cursor 30. For example, if the display is a touch panel, the coordinate position when the display is touched may be selected.

In this embodiment, when an arbitrary coordinate position is selected while the omnidirectional still image 14 is displayed on the display, the detailed still image 13 is displayed. Note that this detailed image 13 may be displayed as a moving image. Further, the omnidirectional image 14 displayed on the display may be a moving image.

Additionally or alternatively, when acquiring the detailed image 13, the photographing position estimating unit 21 compares the detailed image 13 with the omnidirectional image 14 whose distortion has been corrected. Next, the photographing position estimation unit 21 extracts a similar range similar to the detailed image 13 in the distortion-corrected omnidirectional image 14 . Next, the photographing position estimating unit 21 specifies the extracted similar range as including a coordinate position that coincides with the photographing direction of the pan-tilt-zoom camera 4. In this way, it is possible to extract the similar range corresponding to the detailed image 13 from the omnidirectional image 14 and improve the accuracy of identifying the correspondence between the detailed image 13 and the omnidirectional image 14.

Note that the photographing position estimating unit 21 may use an analysis technique based on learning of artificial intelligence when extracting a similar range similar to the detailed image 13 in the omnidirectional image 14. For example, the photographing position estimation unit 21 may be equipped with artificial intelligence (AI) that performs machine learning.

The system of this embodiment includes a control device with a highly integrated processor such as a dedicated chip, an FPGA (Field Programmable Gate Array), a GPU (Graphics Processing Unit), or a CPU (Central Processing Unit), and a ROM (Read Only memory) or RAM (Random Access Memory), external storage devices such as HDD (Hard Disk Drive) or SSD (Solid State Drive), display devices such as displays, and input devices such as mouse or keyboard. , and a communication interface. This system can be realized with a hardware configuration using a normal computer.

Note that the program executed by the system of this embodiment is provided by being pre-installed in a ROM or the like. Alternatively, this program may be installed as a file in installable or executable format on a computer-readable non-transitory storage medium such as a CD-ROM, CD-R, memory card, DVD, or floppy disk (FD). It may be stored and provided.

Further, the program executed by this system may be stored on a computer connected to a network such as the Internet, and may be downloaded and provided via the network. Furthermore, this system can also be configured by combining separate modules that independently perform the functions of the constituent elements by interconnecting them via a network or dedicated line.

In addition, in this embodiment, the pan-tilt-zoom camera 4 (first camera) itself has a pan function, but other aspects may be used. For example, the panning function of the pan-tilt-zoom camera 4 may be realized by the rotation of the patrol inspection robot 2. In other words, the functions of the patrol inspection robot 2 may be included as the functions of the pan-tilt-zoom camera 4. Furthermore, the photographing direction detection section 6 may detect the direction in which the patrol inspection robot 2 is facing (turning direction).

In this embodiment, the pan-tilt-zoom camera 4 (first camera) and the omnidirectional camera 5 (second camera) are mounted on the patrol inspection robot 2, but other embodiments may be used. For example, an inspection worker may carry the pan-tilt-zoom camera 4 and the omnidirectional camera 5, or use a fixed device such as a tripod to fix the pan-tilt-zoom camera 4 and the omnidirectional camera 5 at the inspection site. It is also possible to arrange and photograph.

According to the embodiment described above, when the photographing direction of the first camera is included in the photographing range of the second camera, the coordinate position in the second image that coincides with the photographing direction of the first camera By recording the first image in association with the position, when the same subject is photographed with a plurality of cameras having different angles of view, each image can be easily managed.

Although several embodiments of the invention have been described, these embodiments are presented by way of example and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, changes, and combinations can be made without departing from the gist of the invention. These embodiments or modifications thereof are within the scope and gist of the invention, as well as within the scope of the invention described in the claims and its equivalents.

DESCRIPTION OF SYMBOLS 1... Image management system, 2... Patrol inspection robot, 3... Management computer, 4... Pan-tilt-zoom camera, 5... Omnidirectional camera, 6... Shooting direction detection section, 7... Distance measurement section, 8... Position detection section, 9... Traveling device, 10... Control computer, 11... Wheels, 12... Wide-angle camera, 13... Detailed image, 14... Omnidirectional image, 15... Shooting direction information, 16... Position information, 17... Distance information, 18... First image acquisition Part, 19... Second image acquisition unit, 20... Distortion correction unit, 21... Shooting position estimation unit, 22... Display control unit, 23... Image database, 24... Partial range, 25... Corresponding points, 26... Grid line , 27...mark, 28...meter, 29...valve, 30...mouse cursor.

Claims (8)

comprising a management computer that manages images taken by a first camera and a second camera capable of taking images at a wider angle of view than the first camera;
The management computer is
obtaining a first image taken by the first camera and a second image taken by the second camera;
When the photographing direction of the first camera is included in the photographing range of the second camera, the coordinate position in the second image coincides with the photographing direction of the first camera. Record the first image in association with each other,
when any of the coordinate positions of the second image is selected, displaying the first image recorded in association with the selected coordinate position;
It is configured as follows.
Image management system. The second camera is capable of photographing in all directions.
The image management system according to claim 1. The management computer is configured to acquire the first image and information indicating the photographing direction of the first camera.
The image management system according to claim 1 or claim 2. The first camera has at least a pan/tilt function,
The information indicating the photographing direction of the first camera is at least one of a control signal output from a control computer that controls the photographing direction and a detection signal output from a sensor detecting the photographing direction.
The image management system according to claim 3. The management computer is configured to convert information indicating the photographing direction of the first camera into the coordinate position of the second image based on at least one of a conversion table and a conversion formula.
The image management system according to claim 3 or 4. The management computer is
performing image processing to correct distortion of the second image;
when acquiring the first image, comparing the first image and the second image in which the distortion has been corrected;
extracting a similar range similar to the first image in the second image;
identifying the similar range as including the coordinate position that coincides with the photographing direction of the first camera;
It is configured as follows.
The image management system according to any one of claims 1 to 5. The management computer is
acquiring information indicating the distance from a distance measurement unit that measures the distance from at least one of the first camera and the second camera to the subject;
estimating the coordinate position corresponding to the photographing direction of the first camera in the second image based on the distance;
It is configured as follows.
The image management system according to any one of claims 1 to 6. This method is carried out using a management computer that manages each image taken by a first camera and a second camera that can take pictures at a wider angle of view than the first camera,
The management computer acquires a first image taken by the first camera and a second image taken by the second camera,
When the photographing direction of the first camera is included in the photographing range of the second camera, the management computer determines that the coordinate position in the second image coincides with the photographing direction of the first camera. recording the first image in association with the coordinate position;
When any coordinate position of the second image is selected, the management computer displays the first image recorded in association with the selected coordinate position.
Image management method.

Images