JP2022178773A - Image monitoring device, method and program - Google Patents

Abstract

[Problem] The objective is to provide an image monitoring device, method, and program that can automatically detect when a display no longer changes due to an abnormality in the equipment used to display the image on the display. [Solution] The image monitoring device 1 disclosed herein comprises an acquisition unit 2 that acquires images periodically captured of a display used in a signage system, an identification unit 3 that identifies a display image that is included in the acquired images and is displayed on the display, and a determination unit 4 that determines the display state of the display based on the results of a comparison between the display images identified by the identification unit 3 in each of a plurality of images captured at different times. [Selected Figure] Figure 1

Description

The present disclosure relates to an image monitoring device, method and program.

In recent years, digital signage is used as a new form of advertisement and storefront display. Digital signage can display a plurality of contents and moving images on a single display by displaying images on the display, and can be used more widely than display methods using media such as paper. However, there is a possibility that a problem such as not displaying desired video or the like may occur due to display failure. Various events are conceivable as failures, such as failures of displays, terminal devices that control displays, and failures of network devices.

Not only is the value of the advertisement using the display inferior due to the fact that the image, etc. is not displayed on the display, but only one display is not displayed when, for example, multiple displays are used to form one display format. Even so, there are problems such as the appearance value of the advertisement being greatly reduced. Therefore, it is desirable to automatically detect the failure as soon as possible and notify the administrator.

Japanese Patent Application Laid-Open No. 2002-200002 discloses a technique for detecting image stops and black images in a moving image by extracting images from the moving image at predetermined intervals. Further, Patent Literature 2 discloses a technique for determining whether or not the image displayed on the display device is actually displayed by monitoring the image.

Japanese Patent Application Laid-Open No. 2019-212970 JP 2011-135477 A

According to the method of detecting changes in the actual display by extracting images from the moving image at predetermined intervals, that is, by using the captured image, the actual display and the captured image do not necessarily match. Screen capture is performed by the device that performs display processing, and the image may not be displayed correctly on the display. If the display is broken, the actual display and the captured image may not match. Because it is possible. In addition, when a screen abnormality occurs due to a failure of a device performing display processing, a captured image cannot be generated.

The present disclosure has been made to solve such problems, and can automatically detect that the display on the display has stopped changing due to an abnormality in the equipment for displaying on the display. An object of the present invention is to provide an image monitoring device, method, and program.

An image monitoring apparatus according to the present disclosure includes an acquisition unit that acquires an image of a display used in a signage system that is captured periodically, and an identification unit that identifies a display image displayed on the display included in the acquired image. and a determination unit that determines the display state of the display based on a result of comparing the display images specified by the specifying unit in each of the plurality of images captured at different times.

An image monitoring method according to the present disclosure includes the steps of: acquiring an image of a display used in a signage system that is taken periodically; identifying a display image displayed on the display included in the acquired image; A step of determining the display state of the display based on a result of comparing the specified display images in each of the plurality of images captured at different times.

An image monitoring program according to the present disclosure includes a process of acquiring an image of a display used in a signage system that is taken periodically, a process of specifying a display image displayed on the display included in the acquired image, and The image monitoring program causes a computer to execute a process of determining the display state of the display based on the result of comparing the specified display images in each of the plurality of images captured at different times.

Advantageous Effects of Invention According to the present disclosure, it is possible to provide an image monitoring apparatus, method, and program capable of automatically detecting when the display on the display stops changing due to an abnormality in the equipment for displaying on the display.

1 is a configuration diagram of an image monitoring apparatus according to a first embodiment of the present disclosure; FIG. It is a configuration diagram of a display abnormality detection device according to a second embodiment of the present disclosure. FIG. 7 is a diagram showing an outline of an image generated by performing mask processing on an image acquired from a camera according to the second embodiment of the present disclosure; FIG. 7 is a diagram showing an image of calculating a difference value using a comparison image in which mask images are multiplexed according to the second embodiment of the present disclosure; FIG. 10 is a diagram showing an installation image of a camera that captures images of a display according to a second embodiment of the present disclosure; FIG. 7 is a flowchart of abnormality determination processing performed by an abnormality determination unit according to a second embodiment of the present disclosure;

Hereinafter, embodiments will be described with reference to the drawings. Since the drawings are simplified, the technical scope of the embodiments should not be narrowly interpreted based on the description of the drawings. Also, the same elements are denoted by the same reference numerals, and overlapping descriptions are omitted.

<Embodiment 1>
The configuration of the image monitoring apparatus 1 according to this embodiment will be described with reference to FIG. FIG. 1 is a configuration diagram of an image monitoring apparatus 1 according to this embodiment. The image monitoring device 1 includes an acquisition unit 2 , a specification unit 3 and a determination unit 4 .

Acquisition unit 2 periodically acquires an image of a display used in a signage system. "Regularly" also includes what is not at regular time intervals. The specifying unit 3 specifies the display image displayed on the display, which is included in the image acquired by the acquiring unit 2 . The determination unit 4 determines the display state of the display based on the result of comparing the display images identified by the identification unit 3 in each of the plurality of images captured at different times.

According to the image monitoring apparatus 1 of the present embodiment, it is possible to automatically detect that the display on the display has stopped changing due to an abnormality in the equipment for displaying on the display.

<Embodiment 2>
The configuration of the display abnormality detection device 10 according to this embodiment will be described with reference to FIG. FIG. 2 is a configuration diagram of the display abnormality detection device 10 according to this embodiment. The display abnormality detection device 10 includes an image acquisition unit 11, an image storage unit 12, a difference calculation unit 13, an abnormality determination unit 14, and a notification unit 15, and acquires images captured by the camera 20. FIG. The display abnormality detection device 10 automatically detects that the image on the display 16 used in the signage system has stopped changing.

The image acquisition unit 11 acquires images of the display 16 periodically captured by the camera 20 . “Regularly” may be, for example, the number of seconds at which still images are switched when still images are switched and displayed on the display 16 every few seconds, or may be set arbitrarily. can do. In addition, "regularly" also includes what is not at regular time intervals. The timing at which the image acquisition unit 11 acquires an image from the camera 20 is preferably set to a timing that does not acquire the same still image twice or more.

The image acquisition unit 11 identifies the display image displayed on the display 16 included in the acquired image in order to eliminate the influence of the image that has moved from outside the display area of the display 16 . A particular method may perform masking on an image acquired from camera 20 . The image acquisition unit 11 superimposes a frame surrounding an area including the display image displayed on the display 16 on the image of the display 16 photographed by the camera 20, and masks the peripheral portion of the image on the display 16 from the frame. do.

Here, an outline of an image generated by performing mask processing on an image acquired from the camera 20 will be described with reference to FIG. FIG. 3 is a diagram showing an outline of an image generated by performing mask processing on an image acquired from the camera 20. As shown in FIG. As shown in FIG. 3, the image acquiring unit 11 generates a mask image in which the outside of a trapezoidal frame formed by connecting four preset points with straight lines is blacked out. Note that the generation of the mask image is not limited to a shape in which four points are connected by straight lines, and various shapes such as polygons may be used. The mask image generated by the image acquisition unit 11 is used as a comparison image in the difference calculation unit 13, as will be described later.

The image storage unit 12 stores the image displayed on the display 16 acquired by the image acquisition unit 11 as a comparison image. The image storage unit 12 may store a comparison image obtained by masking the image displayed on the display 16 . The image storage unit 12 stores two images, the acquired comparison image and the comparison image acquired at the previous timing. When a new image is saved, the image storage unit 12 may delete the older one of the two stored images.

The difference calculation unit 13 compares the images displayed on the display 16, which are included in each of the two images stored in the image storage unit 12, to calculate a difference value. The difference value is the total value of differences between pixel values obtained by calculating component values corresponding to different colors in pixels in two display images. The pixel value may use each color component of RGB. When values of 0 to 255 are taken as RGB values, for example, white is expressed as RGB=(255, 255, 255). The difference value may be calculated based on the pixel-by-pixel difference between the two images using the pixel values of the RGB color components. Further, the difference value may be calculated from all pixels in the comparison image, or may be calculated from only pixels extracted at regular intervals, for example.

Here, the outline of the process of calculating the difference value using the masked comparison image performed by the difference calculation unit 13 will be described with reference to FIG. 4 . FIG. 4 is a diagram showing an image of calculating a difference value using a comparison image subjected to mask processing. As shown in FIG. 4, the image acquired at the previous timing and the image acquired this time can be compared for calculating the difference value for the unmasked area. This is because the difference value of the mask area is zero.

The difference calculation unit 13 can calculate the difference value only for the unmasked regions included in the previously acquired image and the currently acquired image. The difference value calculated by the difference calculation unit 13 is a value obtained by eliminating the influence of the shift to the image displayed on the display 16 in the image captured by the camera 20 .

The abnormality determination unit 14 determines whether the screen displayed on the display 16 is changing normally based on the difference value calculated by the difference calculation unit 13 . The abnormality determination unit 14 notifies the notification unit 15 of the determination result. The details of the abnormality determination process performed by the abnormality determination unit 14 will be described later.

The notification unit 15 receives notification of the result of determining whether or not the screen displayed on the display 16 is changing normally from the abnormality determination unit 14, and notifies the result. Various means can be used for the notification means, such as text means such as e-mail to the administrator, voice means, and display means.

An installation image of the camera 20 on the display 16 is shown in FIG. As shown in FIG. 5, the camera 20 is installed so as to face the display 16 side, and can capture an image displayed on the display 16 . As shown in FIG. 5, the camera 20 shoots an image displayed on the display 16 directly from above the display 16, but the display 16 can be shot from various directions, not just from above. I wish I could. In addition, depending on the installation location of the display 16, there may be a situation where people come and go in front of the display 16. FIG. Therefore, in the present embodiment, the camera 20 is fixed above the display 16 and installed at an angle that obliquely looks down on the display 16 . In addition, the camera 20 may have a polarizing filter on the lens in order to reduce reflection of illumination or the like in the shooting environment on the display 16 .

Here, the abnormality determination processing performed by the abnormality determination unit 14 will be described with reference to FIG. 6 . FIG. 6 is a flowchart of abnormality determination processing performed by the abnormality determination unit 14. As shown in FIG. α is a difference value threshold for determining whether the screen displayed on the display 16 is changing normally. β is the threshold for the number of abnormal counts, which is the number of times the image is estimated to be unchanged when the difference value is the threshold α. The thresholds α and β can be set arbitrarily.

The abnormality determination unit 14 acquires the difference value calculated by the difference calculation unit 13 (step 101). If the difference value between the previously obtained image and the currently obtained image exceeds the threshold α, the abnormality determination unit 14 determines that the screen is changing normally (NO in step 102), and sets the abnormality count number to 0. Clear (step 103). If the calculated difference value is equal to or less than the threshold value α (YES in step 102), the abnormality determination unit 14 assumes that there is no change in the screen and adds 1 to the abnormality count number (step 104). If the abnormality count number is less than the threshold value β (NO in step 105), the abnormality determination unit 14 holds the current abnormality count number and waits until the next difference value is input. If the abnormality count number is equal to or greater than the threshold value β (YES in step 105), the abnormality determination unit 14 determines that an abnormality has occurred on the screen displayed on the display 16, and notifies the notification unit 15 of the occurrence of the abnormality. is notified (step 106), and the process ends.

As described above, the display abnormality detection device 10 detects changes in the image displayed on the display 16 based on the image captured by the camera 20 installed outside the display 16 . Therefore, it is possible to prevent the processing of the display abnormality detection device 10 from having any influence on the display processing of the display 16 performed by other devices. Moreover, the display abnormality detection device 10 detects an abnormality based on the difference between the images captured by the camera 20 . Therefore, the display abnormality detection device 10 does not need to store in advance a moving image or a still image to be displayed on the display 16, and it is not necessary to store in advance a huge amount of images for comparison with the actually displayed image. . Also, the camera 20 does not have to have a high resolution and does not need to have functions such as distortion correction.

According to the display abnormality detection device 10 of the present embodiment, it is possible to automatically detect that the display on the display 16 has stopped changing due to an abnormality in the equipment for displaying on the display 16 .

<Embodiment 3>
The difference calculation unit 13 of the display abnormality detection apparatus 10 according to the present embodiment divides the image into a plurality of regions, and compares the image acquired by the image acquisition unit 11 and the image stored in the image storage unit 12 for each of the divided regions. are compared and the difference value is calculated. The abnormality determination unit 14 determines whether or not there is a display abnormality on the display 16 based on the difference value calculated by the difference calculation unit 13 .

Therefore, the abnormality determination unit 14 compares the calculated difference value with the difference threshold α provided for each corresponding region in the previously acquired image and the currently acquired image. The abnormality determination unit 14 determines that the image displayed on the display 16 is changing normally only when the difference between all regions included in each image exceeds the threshold α.

With such a configuration, the display abnormality detection device 10 according to the present embodiment can detect an abnormality even when an abnormality occurs in a part of the image displayed on the display 16. can.

<Other embodiments>
The image monitoring apparatus 1 according to the present disclosure includes an embodiment as an image monitoring method, for example. That is, the image monitoring method comprises the steps of: acquiring images taken periodically of a display used in a signage system; identifying display images displayed on the display included in the acquired images; A step of determining the display state of the display based on a result of comparing the specified display images in each of the plurality of captured images.

In the above examples, the programs can be stored and delivered to the computer using various types of non-transitory computer readable media. Non-transitory computer-readable media include various types of tangible storage media. Non-transitory computer-readable media include, for example, magnetic recording media, magneto-optical recording media, CD-ROMs (Read Only Memory), CD-Rs, CD-R/Ws, and semiconductor memories. Semiconductor memories are, for example, mask ROMs, PROMs (Programmable ROMs), EPROMs, flash ROMs, RAMs (Random Access Memories), and the like. The program may also be delivered to the computer on various types of transitory computer readable medium. Examples of transitory computer-readable media include electrical signals, optical signals, and electromagnetic waves. Transitory computer-readable media can deliver the program to the computer via wired channels, such as wires and optical fibers, or wireless channels.

The above program includes a process of acquiring an image of a display used in a signage system that is taken periodically, a process of specifying a display image displayed on the display included in the acquired image, and a process of specifying the display image displayed on the display, which is taken at different times. The image monitoring program causes a computer to execute a process of determining the display state of the display based on the result of comparing the specified display images in each of the plurality of images.

It should be noted that the present disclosure is not limited to the above embodiments, and can be modified as appropriate without departing from the scope of the present disclosure.

1 Image monitoring device 2 Acquisition unit 3 Identification unit 4 Determination unit 10 Display abnormality detection device 11 Image acquisition unit 12 Image storage unit 13 Difference calculation unit 14 Abnormality determination unit 15 Reporting unit 16 Display 20 Camera

Claims (8)

an acquisition unit that acquires an image of a display used in a signage system that is periodically captured;
a specifying unit that specifies a display image displayed on the display, which is included in the acquired image;
a determination unit that determines a display state of the display based on a result of comparing the display images specified by the specifying unit in each of the plurality of images captured at different times;
An image monitoring device. The determination unit compares the display images included in each of the two images captured at different times to calculate a difference value, and determines the display state based on the calculated difference value.
The image monitoring device according to claim 1. The determination unit calculates the number of times the difference value is equal to or less than a threshold, and determines that the display has a display abnormality when the number of times is equal to or greater than the threshold.
The image monitoring device according to claim 2. The determination unit calculates, as a difference value, a total value of differences in pixel values obtained by calculating component values corresponding to different colors in pixels in regions corresponding to the two display images.
4. The image monitoring device according to claim 2 or 3. The determining unit divides each of the two display images into a plurality of regions, and determines whether or not there is a display abnormality on the display by calculating the difference value for each of the corresponding regions.
The image monitoring apparatus according to any one of claims 2-4. The specifying unit superimposes a frame surrounding an area including the display image displayed on the display on the captured image of the display, and masks a peripheral portion of the image on the display from the frame.
The image monitoring apparatus according to any one of claims 1-5. Acquiring periodically captured images of a display used in a signage system;
identifying a display image displayed on the display that is included in the acquired image;
determining the display state of the display based on a result of comparing the specified display images in each of the plurality of images captured at different times;
A method of image monitoring, comprising: A process of acquiring an image of a display used in a signage system, which is periodically captured;
a process of identifying a display image displayed on the display, which is included in the acquired image;
a process of determining a display state of the display based on a result of comparing the specified display images in each of the plurality of images captured at different times;
An image monitoring program that causes a computer to execute

Images