JP2022147686A - Imaging system and imaging method - Google Patents

Abstract

To provide a system capable of monitoring a place that is difficult to undergo unmanned monitoring by using a transmission member that transmits an infrared region and a visible light region, a thermocamera, and a visible light camera.SOLUTION: In an imaging system, an imaging device 1 includes a transmission member that transmits visible light and infrared light, a first imaging part for imaging visible light of monitoring objects through the transmission member about one or more monitoring objects, and a second imaging part for imaging infrared light of the monitoring objects through the transmission member about the monitoring objects. An information processing device 2 includes a detection part for detecting temperature of the monitoring objects on the basis of first imaging data imaged by the first imaging part and second imaging data imaged by the second imaging part.SELECTED DRAWING: Figure 4

Description

The present invention relates to an imaging system and an imaging method.

Conventionally, thermographic cameras and visible light cameras have been used for automatic monitoring of facilities such as factories. A system has been proposed in which facility monitoring is operated unmanned, and the above-described camera can, for example, image the temperature and state of a switchboard, etc., display the imaged content, record it, and issue a report (for example, Patent document 1).

In the above-mentioned switchboards, loose screws due to poor construction and aged deterioration of parts may cause overheating of terminals, etc., which may cause a fire. there is

However, in the conventional system, since the terminal box of the switchboard is provided with a protective lid, an operator must open the lid and visually inspect the condition inside the terminal box itself, which is unmanned. was difficult to monitor automatically.

The present invention is made in view of the above, and it is difficult to automatically monitor unmanned by using a transparent member that transmits both the infrared region and the visible light region, a thermo camera and a visible light camera. It is to provide a system that can monitor the location.

In order to solve the above-described problems and achieve the object, the present invention provides a transmission member that transmits visible light and infrared light, and one or more monitoring objects, which are transmitted through the transmission member. a first imaging unit for imaging the visible light of the monitoring object, a second imaging unit for imaging the monitoring object with infrared light through the transmission member, and the first imaging unit and a detection unit that detects the temperature of the object to be monitored based on first image data captured by and second image data captured by the second image capture unit.

According to the present invention, the imaging system can grasp the state of the monitored object without the operator directly viewing the monitored object, and monitors the monitored object which has been difficult to monitor unmanned. It has the effect of being able to

FIG. 1 is a system configuration diagram showing an example of an imaging system according to the first embodiment. FIG. 2 is a diagram showing an example of a hardware configuration relating to the imaging apparatus according to the first embodiment. FIG. 3 is a diagram showing an example of the hardware configuration of the information processing apparatus according to the first embodiment. FIG. 4 is a diagram showing an example of the functions of the information processing apparatus and imaging apparatus according to the first embodiment. FIG. 5 is a top perspective view of the imaging device of the first embodiment. FIG. 6 is a flow chart showing an example of the operation of the imaging system according to the first embodiment. FIG. 7 is a system configuration diagram showing an example of an imaging system according to the second embodiment. FIG. 8 is a diagram showing an example of the hardware configuration related to the moving object processing apparatus according to the second embodiment. FIG. 9 is a diagram showing an example of functions of an information processing device, an imaging device, and a moving body processing device according to the second embodiment. FIG. 10 is a top perspective view of the imaging device of the second embodiment. FIG. 11 is a flow chart showing an example of the operation of the imaging system according to the second embodiment.

Embodiments of an imaging system and an imaging method will be described in detail below with reference to the accompanying drawings.

(First embodiment)
FIG. 1 is a diagram showing a schematic configuration of an imaging system according to this embodiment. As shown in FIG. 1 , the imaging system 100 includes an imaging device 1 , an information processing device 2 , a display device 3 , an external device 4 and a transmissive member 5 .

The imaging device 1 captures an image of one or more monitored objects via the transparent member 5 . Details will be described later. The information processing device 2 detects the temperature of the monitored object based on the imaging data of the monitored object imaged by the imaging device 1 . Details will be described later.

The display device 3 is connected to the information processing device 2 and displays the content processed by the information processing device 2 on the display section of the display device 3 . Although the display device 3 is provided outside the information processing device 2 in the example of FIG. 1 , the information processing device 2 may have the display device 3 . The external device 4 is connected to the information processing device 2 and is, for example, a cloud server. Note that the external device 4 is not limited to a cloud server, and may be, for example, a storage medium.

The transmissive member 5 is a member that transmits visible light and infrared light. Specifically, the transmissive member 5 is made of a material that can transmit the wavelengths of visible light and infrared light. The transparent member 5 is used when the imaging device 1 captures images of one or more monitored objects, and is provided in front of the imaging area of the monitored objects. Note that the transparent member 5 may be provided in front of the lens of the imaging device 1 . FIG. 1 shows, as an example, an imaging system 100 including an imaging device 1, an information processing device 2, a display device 3, an external device 4, and a transmissive member 5. As shown in FIG. However, the numbers and devices of the imaging device 1, the information processing device 2, the display device 3, the external device 4, and the transmissive member 5 included in the imaging system 100 are not limited to these.

FIG. 2 is a diagram showing an example of the hardware configuration of the imaging device 1. As shown in FIG. The imaging apparatus 1 includes a CPU (Central Processing Unit) 11, a memory 12, a network I/F (Interface) 13, an RGB camera 14, a thermo camera 15, an external I/F 16, and a bus 17 for transmitting data, control signals, and the like. have.

The CPU 11 is an arithmetic device that implements each function of the imaging device 1 by executing a predetermined program stored in the memory 12 . The memory 12 is, for example, a storage device such as a RAM (Random Access Memory), a ROM (Read Only Memory), or a flash ROM. A network I/F 13 is a communication interface for connecting the imaging device 1 to a network.

The RGB camera 14 is imaging means for imaging the light emitted by the monitored object as visible light. The thermo camera 15 is imaging means for imaging the heat generated by the object to be monitored as infrared light. The external I/F 16 is an interface for connecting an external device to the imaging device 1 .

FIG. 3 is a diagram showing an example of the hardware configuration of the information processing device 2. As shown in FIG. The information processing device 2 has a CPU 21, a RAM 22, a ROM 23, an auxiliary storage device 24, a network I/F 25, an input device 26, a display device 27, an external I/F 28, and a bus 29 for transmitting data, control signals and the like.

The CPU 21 is an arithmetic device that implements each function of the information processing device 2 by reading out programs and data stored in the ROM 23, auxiliary storage device 24, etc. onto the RAM 22 and executing processing. The RAM 22 is a volatile memory used as a work area for the CPU 21 and the like. ROM 23 is a non-volatile memory. The auxiliary storage device 24 is, for example, a storage device such as a HDD (Hard Disk Drive) or an SSD (Solid State Drive), and stores, for example, an OS (Operation System), application programs, various data, and the like.

Network I/F 25 is a communication interface for connecting information processing device 2 to a network. The input device 26 is an input device such as a mouse and keyboard, and is used to input various operations to the information processing device 2 . The display device 27 is a display device such as a display, and is used to display processing results and the like by the information processing device 2 . The external I/F 28 is an interface for connecting the external device 4 to the information processing device 2 .

FIG. 4 is a diagram showing an example of functional configurations of the imaging device 1 and the information processing device 2. As shown in FIG. The imaging device 1 includes a first imaging unit 111 , a second imaging unit 112 , an acquisition unit 113 and a transmission unit 114 . The imaging apparatus 1 implements the functions of the first imaging unit 111, the second imaging unit 112, the acquisition unit 113, the transmission unit 114, and the like by executing a predetermined program.

Here, with reference to FIG. 5, a monitoring target imaged by the first imaging unit 111 and the second imaging unit 112 included in the imaging device 1 will be described. FIG. 5 is a top perspective view of the imaging device 1 including the transmissive member 5 and the monitored object 8. As shown in FIG.

The first imaging unit 111 captures the visible light of one or more monitored objects 8 via the transparent member 5 . Specifically, the first image capturing unit 111 captures the visible light of the monitored object 8 in cooperation with the RGB camera 14 . For example, the first imaging unit 111 captures an image of the monitored object 8, and RGB image data (hereinafter referred to as RGB image data) representing the color of the object and space of the captured monitored object 8 with RGB (three colors) intensity values. , also referred to as first imaging data) and outputs it. The RGB imaging data is composed of a plurality of images (pixels) of M×N (for example, 60×80). The plurality of pixels includes a pixel group corresponding to the monitored object within the monitored area and a pixel group corresponding to the surrounding area of the monitored object.

The second imaging unit 112 captures an infrared light image of one or more monitored objects 8 via the transparent member 5 . Specifically, the second imaging unit 112 captures the visible light of the monitored object 8 in cooperation with the thermo camera 15 . For example, the second imaging unit 112 captures an image of the monitored object 8 and converts it into the temperature of the imaged object or space of the monitored object 8 to obtain temperature imaging data (hereinafter also referred to as second imaging data). ) is generated and output. The temperature imaging data is composed of a plurality of M×N pixels (pixels), similar to the RGB imaging data.

Return to FIG. The obtaining unit 113 obtains first image data captured by the first image capturing unit 111 and second image data captured by the second image capturing unit 112 . Specifically, the acquisition unit 113 acquires RGB image data captured by the RGB camera 14 and temperature image data captured by the thermo camera 15 . The transmission unit 114 transmits the first image data and the second image data acquired by the acquisition unit 113 to the information processing device 2 .

The information processing device 2 includes a receiver 211 , a detector 212 , an output controller 213 , a color pixel output section 214 , a detection frame output section 215 and a temperature output section 216 . The information processing device 2 implements the functions of a receiving unit 211, a detecting unit 212, an output control unit 213, a color pixel output unit 214, a detection frame output unit 215, a temperature output unit 216, and the like by executing a predetermined program. do.

The receiving unit 211 receives the first imaging data and the second imaging data transmitted by the transmission unit 114 of the imaging device 1 and stores them in the auxiliary storage device 24 . The detection unit 212 detects the temperature of the monitored object 8 based on the first imaging data captured by the first imaging unit 111 and the second imaging data captured by the second imaging unit 112. detect. Further, the detection unit 212 detects the object to be monitored 8 based on the first imaging data captured by the first imaging unit 111 and the second imaging data captured by the second imaging unit 112 . Generate a detection frame for detecting temperature.

The detection unit 212 detects temperature changes in each region within the detection frame. The detection unit 212 also compares the temperature of the color image with a predetermined threshold value (set value for abnormal temperature determination) in a plurality of cells forming the detection frame.

The output control unit 213 performs output control for controlling the output of information regarding the temperature detected by the detection unit 212 . Specifically, when the result of the detection unit 212 indicates that the temperature exceeds the threshold value, the output control unit 213 performs alert processing, that is, notification processing. Alert processing includes, for example, sending an e-mail to the display device 3, the external device 4, etc., changing the color display of the signal light provided in the external device 4, etc. from a normal state to a warning state, This is processing such as generating an alarm sound from the

Further, when the temperature threshold is exceeded based on the result of the detection unit 212, the output control unit 213 performs a process of notifying the position information of the position where the temperature threshold is exceeded. Furthermore, the output control unit 213 controls output by the display device 3 or controls output by the external device 4 .

The output control section 213 includes a color pixel output section 214 , a detection frame output section 215 and a temperature output section 216 . The color pixel output unit 214 generates color pixels of the monitored object based on the imaging data captured by the first imaging unit 111 that captures an image of the monitored area including the monitored object and the surroundings of the monitored object. The resulting color pixels are output to the display device 3 . The detection frame output unit 215 detects the temperature of the monitoring area based on the second imaging data captured by the second imaging unit 112 that captures an image of the monitoring area including the monitoring target and the surroundings of the monitoring target. A detection frame composed of a plurality of cells is generated, and the generated detection frame is superimposed on the color image and output to the display device 3 .

The temperature output unit 216 outputs at least temperatures to the plurality of cells forming the detection frame. In other words, the temperature output unit 216 outputs temperature information indicating at least the temperature for each of the plurality of regions forming the detection frame. The temperature display may be, for example, a temperature value (25 degrees, 40 degrees, etc.) and a color corresponding to the temperature value (25 degrees blue, 40 degrees red, etc.).

Next, operation of the imaging system 100 will be described with reference to FIG. FIG. 6 is a flowchart for explaining an example of the operation of the imaging system 100. FIG.

First, the first imaging unit 111 captures an image of visible light of one or more monitored objects 8 through a transparent member. Also, the second imaging unit 112 captures an image of the visible light of one or more monitored objects 8 via a transparent member (step S1). After the imaging process is performed, the process proceeds to step S2.

The acquiring unit 113 acquires first image data captured by the first image capturing unit 111 and second image data captured by the second image capturing unit 112 (step S2). After the acquisition process is performed, the process proceeds to step S3.

The transmission unit 114 transmits the first image data and the second image data acquired by the acquisition unit 113 (step S3). After the transmission process is performed, the process proceeds to step S4.

The receiving unit 211 receives the first imaging data and the second imaging data transmitted by the transmission unit 114 (step S4). After the reception process is performed, the process proceeds to step S5.

The detection unit 212 detects the temperature of the monitored object 8 based on the first image data and the second image data received by the reception unit 211 (step S5). After the detection process is performed, the process proceeds to step S6.

The output control unit 213 controls the output of the information regarding the temperature detected by the detection unit 212 (step S6). After the output control process is performed, the process ends.

As described above, in the present embodiment, the first imaging unit 111 captures one or more monitoring targets through a transparent member that transmits visible light and infrared light, and transmits visible light to the monitoring target. is imaged. In addition, the second imaging unit 112 captures an infrared light image of one or more monitored objects via a transparent member. The detection unit 212 detects the temperature of the monitored object based on the first imaging data and the second imaging data.

According to the configuration of the present embodiment as described above, the object to be monitored can be imaged by the imaging device through the transparent member, so that the temperature state of the object to be monitored can be grasped. Knowing the temperature state of the object to be monitored can reduce the burden of directly viewing the object to be monitored by the operator. Since the visual burden on the worker can be reduced, it is possible to monitor objects that have been difficult to monitor unattended.

(Second embodiment)
Next, a second embodiment will be described. The description of the parts common to the above-described first embodiment will be appropriately omitted. In the first embodiment described above, the imaging device detects the temperature of the monitored object based on the first imaging data and the second imaging data obtained by imaging the monitored object through the transmissive member. On the other hand, the present embodiment differs from the above-described first embodiment in that the imaging device is provided in a moving body and includes a monitoring unit that monitors the operating state of the moving body.

FIG. 7 is a diagram showing a schematic configuration of an imaging system according to the second embodiment. As shown in FIG. 7, the imaging system 100 includes an imaging device 1, an information processing device 2, a display device 3, an external device 4, a transmissive member 5, a moving body 6, and a moving body processing device .

The moving body 6 is provided in the imaging device 1 and runs unmanned under the control of the moving body processing device 7 . The moving body processing device 7 controls the moving body 6 and moves the moving body 6 to the object to be monitored. Details will be described later.

FIG. 8 is a diagram showing an example of the hardware configuration of the mobile object processing device 7. As shown in FIG. The mobile processing device 7 has a CPU 71, a RAM 72, a ROM 73, an auxiliary storage device 74, a network I/F 75, an external I/F 76, and a bus 77 for transmitting data, control signals and the like.

The CPU 71 is an arithmetic unit that implements each function of the mobile object processing device 7 by reading programs and data stored in the ROM 73 and the auxiliary storage device 74 onto the RAM 72 and executing processing. The RAM 72 is a volatile memory used as a work area for the CPU 71 and the like. ROM 73 is a non-volatile memory. The auxiliary storage device 74 is, for example, a storage device such as a HDD (Hard Disk Drive) or an SSD (Solid State Drive), and stores, for example, an OS (Operation System), application programs, various data, and the like.

A network I/F 75 is a communication interface for connecting the mobile processing device 7 to a network. The external I/F 76 is an interface for connecting the mobile object 6 to the mobile object processing device 7 .

FIG. 9 is a diagram showing an example of functional configurations of the imaging device 1, the information processing device 2, and the moving body processing device 7. As shown in FIG. The moving object processing device 7 includes a movement control unit 711 . The moving object processing device 7 implements the functions of the movement control unit 711 and the like by executing a predetermined program.

Here, with reference to FIG. 10, the movement of the imaging device 1 provided with the moving body 6 to the object to be monitored will be described. FIG. 10 is a top perspective view of the imaging device 1 provided with the transmission member 5, the monitored object 8, the travel route 9, and the moving object 6. FIG.

The movement control unit 711 performs a process of moving the moving body 6 to the monitored object 8 that is an imaging target based on the trajectory of the travel route 9 . Specifically, the movement control unit 711 can recognize the travel route 9 by cooperating with a device (for example, a camera) provided on the mobile body 6 . The movement control unit 711 performs processing to stop the moving body 6 when moving to the monitored object 8 . When the moving body 6 stops, the movement control unit 711 performs a process of starting imaging with respect to the imaging device 1 . When the imaging process of the imaging device 1 ends, the movement control unit 711 performs a process of moving the moving body 6 to the next monitored object 8 .

The movement control unit 711 performs processing to stop the moving body 6 when there is an abnormality in the moving body 6 . The movement control unit 711 performs processing for moving the moving body 6 until the imaging device 1 completes imaging processing of the monitored object 8 to be imaged.

A travel route 9 is set along an object to be monitored 8 in the facility. For example, the travel route 9 is set using a vinyl tape (for example, black). Note that the travel route 9 may be configured with a material other than a vinyl tape, and the trajectory may be set with a marker or paint.

Next, operation of the imaging system 100 of the second embodiment will be described with reference to FIG. FIG. 11 is a flow chart for explaining an example of the operation of the imaging system 100 of the second embodiment.

First, the movement control unit 711 moves the moving object 6 to the monitored object 8 to be imaged based on the trajectory of the travel route 9 (step S21). After the movement process is performed, the process proceeds to step S22.

Next, the movement control unit 711 determines whether the moving body 6 has moved to the imaging location. If it is determined that the moving body 6 has moved to the shooting location (step S22: Yes), the process proceeds to step S23. If it is determined that the moving body 6 has not moved to the shooting location (step S22: No), the process proceeds to step S21. The movement control unit 711 performs determination processing until the moving body 6 moves to the imaging location.

When the movement control unit 711 moves in front of the monitored object 8, which is the imaging location, the movement control unit 711 performs processing to stop the moving body 6 (step S23). After the stop processing is performed, the process proceeds to step S24.

When the moving body 6 stops, the movement control unit 711 performs a process of starting imaging with respect to the imaging device 1 . The first imaging unit 111 captures the visible light of one or more monitored objects 8 via a transparent member. Also, the second imaging unit 112 captures the visible light of one or more monitored objects 8 via the transparent member (step S24). After the imaging process is performed, the process proceeds to step S25.

The acquisition unit 113 acquires first image data captured by the first image capturing unit 111 and second image data captured by the second image capturing unit 112 (step S25). After the acquisition process is performed, the process proceeds to step S26.

The transmission unit 114 transmits the first image data and the second image data acquired by the acquisition unit 113 (step S26). After the transmission process is performed, the process proceeds to step S27.

The reception unit 211 transmits the first image data and the second image data transmitted by the transmission unit 114 (step S27). After the transmission process is performed, the process proceeds to step S28.

The detection unit 212 detects the temperature of the monitored object 8 based on the first image data and the second image data received by the reception unit 211 (step S28). After the detection process is performed, the process proceeds to step S29.

The output control unit 213 controls the output of the information regarding the temperature detected by the detection unit 212 (step S29). After the output control process is performed, the moving body 6 moves to the next monitored object. The moving object 6 is moved until the imaging device 1 completes the imaging process of the monitored object 8 to be imaged. When the imaging process is completed, this process ends.

As described above, in the second embodiment, the imaging device is provided in a mobile body, and includes a monitoring unit that monitors the operating state of the mobile body.

According to the configuration of the second embodiment described above, it is possible to move to an object to be monitored by providing the imaging device on the moving object. After the movement, the object to be monitored can be imaged by the imaging device through the transmissive member, so the burden of directly viewing the object to be monitored can be reduced. Since the operator's visual burden can be reduced, it is possible to monitor objects that have been difficult to monitor unattended.

(Modification 1)
For example, the imaging device 1 may be configured to include a rotation mechanism capable of changing the imageable area in the horizontal direction (or the vertical direction). In this case, the first imaging unit 111 and the second imaging unit control the rotation mechanism of the imaging device 1 and rotate the imaging device 1 to change the imaging region of the imaging device 1 . By equipping the imaging device 1 with a rotation mechanism, it is possible to monitor a wide range of objects to be monitored with a smaller number of imaging devices.

It should be noted that the program executed by the imaging system of the present embodiment is pre-installed in a ROM or the like and provided. Further, the program executed by the imaging system of the present embodiment is a file in an installable format or an executable format, and can be stored on a CD-ROM, a flexible disk (FD), a CD-R, a DVD (Digital Versatile Disk), or the like. It may be configured to be recorded on a computer-readable recording medium and provided.

Further, the program executed by the imaging system of the present embodiment may be stored on a computer connected to a network such as the Internet, and may be provided by being downloaded via the network. Also, the program executed by the imaging system of the present embodiment may be provided or distributed via a network such as the Internet.

1 imaging device 2 information processing device 3 display device 4 external device 5 transmission member 6 moving object 7 moving object processing device 111 first imaging unit 112 second imaging unit 113 acquisition unit 114 transmission unit 211 reception unit 212 detection unit 213 output Control unit 214 Color pixel output unit 215 Detection frame output unit 216 Temperature output unit 711 Movement control unit

JP 2017-167616 A

Claims (5)

a transparent member that transmits visible light and infrared light;
a first imaging unit that captures visible light of one or more monitored objects via the transparent member;
a second imaging unit that captures an infrared light image of the monitored object through the transmission member;
Based on the first imaging data captured by the first imaging unit and the second imaging data captured by the second imaging unit,
a detection unit that detects the temperature of the object to be monitored;
An imaging system comprising: an output control unit that controls output of the information about the temperature detected by the detection unit;
The imaging system of claim 1, comprising: The first imaging unit and the second imaging unit are provided in a moving object,
a monitoring unit that monitors the operating state of the moving body;
3. The imaging system according to claim 1 or 2, comprising: The first imaging unit and the second imaging unit each have a rotation mechanism for changing the imaging range,
4. The imaging system according to any one of claims 1-3. a first imaging step of imaging visible light of one or more monitored objects through a transparent member that transmits visible light and infrared light;
a second imaging step of imaging the monitored object with infrared light of the monitored object through the transmission member;
Based on the first imaging data captured by the first imaging step and the second imaging data captured by the second imaging step,
a detection step of detecting the temperature of the object to be monitored;
An imaging method comprising:

Images