JP7542413B2 - Remote work information protection system - Google Patents

Description

Article 30, paragraph 2 of the Patent Act applies. Date of website posting: October 12, 2020. Publisher: Deep Percept Co., Ltd. Website address: https://www.deep-percept.co.jp/news/service/20201012513/

The present invention relates to a system for protecting information displayed on a computer display during remote work, a method for operating the computer system, and a recording medium for recording data read into the computer system.

In recent years, so-called remote work, where employees work from a location away from the company, such as their own home or a satellite office near their home, rather than at the company office, has become common as part of work style reform and efforts to improve the balance between private and work life (work-life balance). Furthermore, with the COVID-19 pandemic that occurred from the end of 2019 to 2020, more and more companies are promoting remote work to prevent infection, and the government's declaration of a state of emergency has accelerated the spread of remote work.
For employees to work remotely outside the company office, they will need an Internet environment to connect to the company's information system (a high-speed connection such as fiber optic is preferable for sending and receiving large data files and moving images), a computer to connect to the Internet and perform work, a microphone, speakers, and camera to connect to the computer for holding meetings over the network (a laptop with a built-in camera, speaker, and microphone is convenient), and a mobile phone (a smartphone is preferable) to receive calls from business partners, etc.
Security can be maintained by using public LAN lines or lines at employees' homes for Internet lines and connecting to the company via a VPN (Virtual Private Network).
On the other hand, in remote work, there are two main risks associated with work using a computer: work management risks and security risks. Work management risks are that remote work is basically performed in a single employee environment, and most work is done at home, which means that working hours are intermittent and the working hours are different from those in the office, making it difficult to grasp the overall working status. As for security risks, there is the risk of information leaking from computers used for work. Digital copying of information from computers was initially attracting attention. However, in addition to copying information, there are also risks such as "peeping" where information displayed on a computer screen is secretly seen from behind while working in a coffee shop, "impersonation" where a third party uses the computer without permission while the employee is away from their desk to go to the bathroom, and information leakage through analog methods such as taking pictures of images displayed on a computer screen with a camera.
It is particularly important to resolve security issues in remote work/work from home contact centers. In work from home contact centers where the department that handles customer inquiries is working from home, personal information of customers is often displayed as viewing information on the screen of the computer used for remote work, and if this information is leaked online, the company's personal information protection will be violated, putting the survival of the company itself at risk.
Patent Document 1 describes a technology that acquires an image of a user using a camera connected to a personal computer, compares the image with stored reference facial image data for authentication, starts taking pictures with the camera when access to a specific file, specific application, or specific network address is initiated, and ends taking pictures with the camera when the access is terminated, and if the image captured by the camera shows the user and a third party other than the user, or if only a third party is captured, makes the personal computer screen invisible and records the camera image.
Patent Document 2 describes a prevention device that is equipped with a device that irradiates infrared rays to an observer along with a displayed image on a display device to prevent photography of a personal computer screen using a camera, and a system that detects reflected light from the photographing device using an infrared filter.

Patent Publication 2017-117155 WO2011/105564A1

The technology of Patent Document 1 detects and responds to the risk of third parties peeking at the computer screen and the risk of impersonation. However, it does not take measures against the surreptitious photography of information displayed on the computer screen. Camera-equipped mobile phones, camera-equipped smartphones, and digital cameras are widely used in general, and photographic equipment is in everyday life. There is a problem that photographs are taken with a camera outside the computer and posted to social networking services or social media, and information about the company itself or its customers that should be kept secret may be revealed, which may cause significant disadvantages.
The technology of Patent Document 2 prevents unauthorized capture of the display screen and detects the photographing device that is attempting to take the picture, but in order to prevent the screen from being photographed itself, it is necessary to provide an invisible light emitting unit that superimposes infrared light on the screen display. In addition, to detect photographing devices, a device that detects the reflection of infrared light from a camera lens with an infrared filter is required. Therefore, there is a problem that the function cannot be achieved with a general laptop computer with a camera or a web camera connected to an external USB for desktop use, and that a special device must be added.

The present invention aims to provide a remote work information protection system that can prevent a computer screen from being photographed by external imaging equipment without the need for additional special equipment.

In order to solve the above problems related to a charging control device, the present application provides, as a first invention,
A display unit (A) having a display for displaying an image;
a business use state detection unit (B) that constantly detects whether the display unit is in a business use state indicating that the display unit is being used for business while the display is being driven;
a viewing side image acquisition unit (C) that acquires a viewing side image, which is an image of a side that views the display, while the business use state detection unit (B) detects that the display is in a business use state;
a storage unit (D) for storing voyeurism usable device discrimination information, which is information for discriminating voyeurism usable device;
a device presence determination unit (E) for determining whether a device available for voyeurism is present in the viewing side video using the obtained viewing side video and the voyeurism available device discrimination information;
a detection information output unit (F) that outputs detection information indicating the presence of the device when the determination result of the device presence determination unit (E) indicates the presence of the device;
The present invention provides a remote work information protection system having the above-mentioned features.

Furthermore, as a second invention, based on the first invention,
a face part image determination unit (G) for determining whether a face part image, which is a partial image including a face, is present in the acquired viewer side image;
a face part image configuration information acquisition unit (H) for acquiring face part image configuration information which is information constituting the face part image when the face part image determination unit (G) determines that a face part image is present;
A face part image composition information storage unit (J) for storing the acquired face part image composition information;
The present invention provides a remote work information protection system further comprising:

Furthermore, as a third invention, based on either the first or second invention,
When the detection information output unit (F) outputs the detection information, a danger information storage unit (K) stores danger information that is information that associates a danger image, which is all or a part of a viewer's image, with viewing information identification information that identifies information displayed on the display,
The present invention further provides a remote work information protection system.

Furthermore, as a fourth invention, based on any one of the first to third inventions,
A remote work information protection system is provided which has a device presence determination unit (E) and a photographing permission state determination means (M) which determines whether the state of the device capable of being used for voyeurism relative to the display unit (A), including its position and orientation, is in a state in which photographing is possible, based on the image of the device capable of being used for voyeurism in the viewing side image and the information on the device capable of being used for voyeurism discrimination.

Furthermore, the present invention provides respective operation methods and respective operation programs corresponding to the remote work information protection systems of the first to fourth inventions. Furthermore, the respective operation programs may be recorded on a recording medium.

The present invention, which has the above-mentioned main configuration, can provide a remote work information protection system that allows a computer screen to be photographed using an external imaging device without the need for additional special equipment.

Functional block diagram of the invention according to the first embodiment FIG. 1 is a flowchart showing the operation of the invention according to the first embodiment. Hardware diagram of the invention according to the first embodiment Functional block diagram of the invention according to embodiment 2 FIG. 11 is a flowchart showing the operation of the invention according to the second embodiment. Hardware diagram of the invention according to the second embodiment Functional block diagram of the invention according to embodiment 3 Operational flow chart of the invention according to the third embodiment Hardware diagram of the invention according to the third embodiment Functional block diagram of the invention according to the fourth embodiment FIG. 11 is a flowchart showing the operation of the invention according to the fourth embodiment. Hardware diagram of the invention according to the fourth embodiment Schematic diagram of an example of the overall configuration of the present invention 4 is a schematic diagram of a user image example 1 input to a viewer-side image acquisition unit in the present invention. FIG. 4 is a schematic diagram of a user image example 1 acquired by a viewer-side video acquisition unit in the present invention. FIG. 13 is a schematic diagram of a second example of a user image input to a viewer-side image acquisition unit in the present invention. FIG. 13 is a schematic diagram of a user image example 2 acquired by a viewer-side video acquisition unit in the present invention. FIG. 11 is a schematic diagram of a user image example 3 input to a viewer-side image acquisition unit in the present invention. FIG. 13 is a schematic diagram of a user image example 3 acquired by a viewer-side video acquisition unit in the present invention. FIG. 13 is a schematic diagram of an example of an image input to a viewer-side image acquisition unit in the present invention when a user leaves his/her seat. 6 is a schematic diagram of an example of an image acquired by a viewer-side image acquisition unit in the present invention when a user leaves his/her seat. 1 is a schematic diagram of an example 1 of a user and a third party image input to a viewer-side image acquisition unit in the present invention. 1 is a schematic diagram of a first example of a user and a third party image acquired by a viewer-side image acquisition unit in the present invention. 5A and 5B are schematic diagrams illustrating an example of a third-party image input to a viewer-side image acquisition unit in the present invention. 5A and 5B are schematic diagrams illustrating examples of third-party images acquired by a viewer-side video acquisition section in the present invention. 13 is a schematic diagram of a fourth example of a user image input to a viewer-side image acquisition unit in the present invention. FIG. 13 is a schematic diagram of a user image example 4 acquired by a viewer-side video acquisition unit in the present invention. FIG. 13 is a schematic diagram of an example 2 of user and third party images input to a viewer-side image acquisition unit in the present invention. FIG. 13 is a schematic diagram of a second example of a user and a third party image acquired by a viewer-side image acquisition unit in the present invention. FIG. FIG. 2 is a front view of an example 1 of a device that can be used for secret photography in the present invention. FIG. 2 is a right side view of an example 1 of a device usable for secret photography in the present invention. FIG. 13 is a front view of an example 2 of a device that can be used for secret photography in the present invention. FIG. 13 is a right side view of an example 2 of a device that can be used for voyeurism in the present invention. 11A and 11B are schematic diagrams showing examples of the shooting permission status of devices usable for voyeurism with respect to a PC display in the present invention; 11A and 11B are schematic diagrams showing examples of the photographing permission status of devices available for voyeurism with respect to a notebook PC according to the present invention; 1 is a schematic diagram showing an example of secret photography from a blind spot of a PC display in the present invention. 1 is a schematic diagram showing an example of surreptitious photography from a blind spot of a notebook PC according to the present invention;

<Prerequisites for the Description of All Embodiments>
<Hardware that can configure the present invention>

While the present invention is in principle an invention that utilizes an electronic computer, it is also realized in part at least through software, hardware, and the collaboration of software and hardware. In this case, the software uses hardware resources to perform various calculations and realize various functions through the required data and information. It can be said that information processing by software is specifically realized using hardware resources.

The hardware that realizes all or part of the constituent elements of the present invention is composed of the basic components of a computer, such as a CPU, memory, bus, input/output devices, various peripheral devices, and user interfaces. The various peripheral devices may include storage devices, Internet interfaces, Internet devices, LAN devices, Wi-Fi devices, displays, HDMI (registered trademark) interfaces, keyboards, mice, speakers, microphones, cameras, videos, televisions, CD devices, DVD devices, Blu-ray devices, USB memory, USB memory interfaces, removable hard disks, general hard disks, projector devices, SSDs, telephones, fax machines, copy machines, printing devices, movie editing devices, and various sensor devices.

The system does not necessarily have to be made up of a single box, but may be made up of multiple boxes connected through communication. The communication may be LAN, WAN, Wi-Fi, Bluetooth (registered trademark), infrared communication, or ultrasonic communication, and some of the system may be installed across national borders.

<Fulfillment of the law of nature in all embodiments of the present invention>

The present invention functions through the cooperation of a computer and software. Conventionally, employees have worked in a workplace such as an office prepared by the company providing the labor, using information devices such as a personal computer in a secure environment protected by the company's internal communication line. However, the present invention prevents information leakage from the employee's personal computer when the employee works by connecting to the company's internal communication line via an external general line, or/and works only on the employee's personal computer in a stand-alone manner without connecting to the company's internal communication line. In particular, the invention prevents information leakage from the personal computer used by the employee by performing processing on the personal computer side to prevent so-called voyeurism, in which the image displayed on the display unit is photographed using an external camera, rather than by copying the digital data itself in the personal computer device. Since it includes processing unique to ICT, it is a so-called business model patent. In addition, information on devices available for voyeurism, detection information, face image composition information, and viewing information identification information are stored and processed in each part, and from this perspective, if the resources of the present invention, such as a computer, are judged based on the matters described in the claims and specification and the technical common sense related to those matters, the present invention utilizes the law of nature.

<Applicant's understanding of the significance of utilizing the laws of nature required by patent law>

The use of the laws of nature required by the Patent Act is required to ensure that the invention can be used industrially and usefully, based on the purpose of the law, from the perspective that the invention must have industrial applicability and contribute to the development of industry. In other words, it is required that the invention be industrially useful, that is, that the effect of the invention declared at the time of application can be reproduced with a certain degree of certainty by implementing the invention. From this perspective, the use of the laws of nature is interpreted as the function of each of the invention-specific matters (invention constituent elements) that constitute the invention to achieve the effect of the invention being performed by using the laws of nature. Furthermore, the effect of the invention should be that it is possible to provide a certain usefulness to the user who uses the invention, and should not be viewed from the perspective of how the user feels or thinks about that usefulness. Therefore, even if the effect that the employees and companies who use this remote work information protection system gain from it is a psychological effect (such as a sense of security), that effect itself is not subject to the criteria for determining whether or not the required use of the laws of nature is available.

＜Hardware configuration＞

Figure 3 is a diagram showing an example of a hardware configuration in this embodiment 1. The hardware configuration of the remote work information protection system in this embodiment will be described with reference to Figure 3.

As shown in FIG. 3, the computer is composed of a chipset, CPU, non-volatile memory, main memory, various buses, BIOS, various interfaces such as USB, HDMI (registered trademark) and LAN, a real-time clock, etc., which are configured on a motherboard. These operate in cooperation with an operating system, device drivers (for various interfaces such as USB and HDMI (registered trademark), cameras, microphones, speakers or headphones, displays, and other built-in devices), various programs, etc. The various programs and data that make up the present invention are configured to efficiently utilize these hardware resources to execute various processes.

≪Chipset≫

A "chipset" is a set of large-scale integrated circuits (LSIs) that are mounted on a computer's motherboard and integrate a communication function between the CPU's external bus and the standard bus that connects memory and peripheral devices, in other words a bridge function. There are cases where a two-chipset configuration is used, and cases where a single chipset configuration is used. The north bridge is located on the side closest to the CPU and main memory, and the south bridge is located on the side furthest away and interfaces with the relatively slow external I/O.

(North Bridge)
The north bridge includes a CPU interface, a memory controller, and a graphics interface. Most of the functions of a conventional north bridge may be performed by the CPU. The north bridge is connected to the memory slot of the main memory via a memory bus, and is connected to the graphics card slot of the graphics card via a high-speed graphics bus (AGP, PCI Express).

(South Bridge)
The south bridge is connected to the PCI interface (PCI slot) via a PCI bus, and is responsible for I/O functions and sound functions with the ATA (SATA) interface, USB interface, Ethernet interface, etc. Incorporating circuits that support PS/2 ports, floppy disk drives, serial ports, parallel ports, and ISA buses, which do not require or are impossible to operate at high speed, will hinder the speed of the chipset itself, so they may be separated from the south bridge chip and placed in a separate LSI called a super I/O chip. A bus is used to connect the CPU (MPU) to peripheral devices and various control units. The bus is connected by the chipset. The memory bus used to connect to the main memory may instead adopt a channel structure to increase speed. A serial bus or a parallel bus can be used as the bus. While a serial bus transfers data one bit at a time, a parallel bus transmits the original data itself or multiple bits extracted from the original data as a single block and transmits it simultaneously over multiple communication paths. A dedicated line for the clock signal is set up in parallel with the data line to synchronize the data demodulation on the receiving side. It is also used as a bus to connect the CPU (chipset) to external devices, and includes GPIB, IDE/(Parallel) ATA, SCSI, PCI, etc. Since there is a limit to how fast it can be made, in PCI Express, an improved version of PCI, and Serial ATA, an improved version of Parallel ATA, the data line can be a serial bus.

≪CPU≫

The CPU sequentially reads, interprets, and executes a sequence of instructions called a program stored in the main memory, outputting information consisting of signals to the main memory. The CPU functions as the center of calculations within the computer. The CPU is composed of a CPU core, which is the center of calculations, and its peripheral parts, and includes registers, cache memory, an internal bus connecting the cache memory and the CPU core, a DMA controller, a timer, and an interface with a connection bus to the north bridge. A single CPU (chip) may have multiple CPU cores. In addition to the CPU, processing may be performed by a graphic interface (GPU) or FPU. Although the embodiment is described as a two-core type, this is not limiting. Programs may also be built into the CPU.

≪Non-volatile memory≫

(HDD)

The basic structure of a hard disk drive consists of a magnetic disk, a magnetic head, and an arm on which the magnetic head is mounted. SATA (previously ATA) can be used as the external interface. A high-performance controller, such as SCSI, is used to support communication between hard disk drives. For example, when copying a file to another hard disk drive, the controller can read the sectors and transfer them to the other hard disk drive for writing. At this time, the host CPU's memory is not accessed. Therefore, there is no increase in the CPU load.

≪Main memory≫

The CPU directly accesses and executes various programs in the main memory. The main memory is a volatile memory and uses DRAM. Programs in the main memory are expanded from the non-volatile memory to the main memory upon receiving a program start command. The CPU then executes the program according to various execution commands and execution procedures within the program.

≪Operating System (OS)≫

The operating system is used to manage the resources available to applications on the computer, to manage various device drivers, and to manage the computer itself (the hardware). Small computers sometimes use firmware as the operating system.

≪BIOS≫

The BIOS causes the CPU to execute procedures for starting up the computer hardware and running the operating system, and is most typically the hardware that the CPU reads first when it receives a computer startup command. The BIOS records the address of the operating system stored on the disk (non-volatile memory), and the operating system is sequentially loaded into main memory by the BIOS loaded into the CPU and becomes operational. The BIOS also has a check function that checks whether various devices are connected to the bus. The check results are stored in main memory and are made available to the operating system as appropriate. The BIOS may be configured to check external devices, etc. The above is the same for all embodiments.

≪Camera≫

When using a personal computer for business purposes outside a company's office, an external camera connected via a USB terminal or a camera built into a laptop computer is used as a device for acquiring the viewing side image, which is the image of the side viewing the display. In Windows (registered trademark), when one software is using the camera or microphone, other software may not be able to use the camera or microphone at the same time. The remote work information protection system of the present invention (hereinafter abbreviated as the information protection system) can be configured to provide an image and audio distribution unit in software, and distribute and distribute image and audio when a web conference software or the like tries to use the camera or microphone. This is because by using a camera, microphone, speaker, earphones, etc., it is possible to have a conversation including not only images but also audio with other people in the company (such as a superior) or customers outside the company via the Internet as remote work. In this specification, when an employee (hereinafter referred to as a remote worker) who works remotely outside a company's office connects his or her own personal computer to the Internet to access the company's internal line or internal server in the course of work, a VPN connection may be used. The benefits of this invention can be achieved even if the connection is made without using a VPN or other security measures.

≪Display≫

A display is a device that displays various information and processing results when using a PC for business or non-business tasks. A PC and a display are connected via a display connection terminal on the PC's motherboard or an expansion board connected by a bus such as PCI Express. A variety of terminals have been used for connection, including HDMI (registered trademark), USB, Display port, DVI-D, and Dsub. The appropriate terminal can be selected according to the connection terminals of the PC and display to be used for business purposes. A device driver for the terminal (interface) to be used is installed in the PC. Depending on the capabilities of the PC's graphics processing chip, multiple displays can also be connected and used.

As shown in FIG. 3, the present invention can basically be configured with a general-purpose computer program and various devices. The computer basically operates by loading a program recorded in non-volatile memory into main memory, and then executing processing using the main memory, CPU, and various devices. Communication with devices is performed via an interface connected to a bus line. Possible interfaces include a display interface, a keyboard, and a communication buffer. Below, an embodiment of the present invention will be described with reference to illustrated examples.

<Overview of embodiment 1> Mainly claim 1
The remote work information protection system of embodiment 1 is configured to acquire viewing side video, which is video from the side viewing the display that shows the video output from the computer, while an employee is using a computer or the like for work at home or the like for remote work, determine whether a device that can be used for voyeurism is present in the viewing side video, and output detection information if present.
In general, when working remotely, companies lend employees the computers, tablet devices, smartphones, etc. required for work. Although it is possible to use personal devices, this is undesirable from the viewpoint of information leakage, especially for computers, since the security measures are not standardized. In the following description, the remote worker is assumed to use a computer as the device, but the same effect of the present invention can be obtained even if a tablet device, etc. is used.
FIG. 13 shows an example of the configuration of the present invention. Personal computers (hereinafter referred to as PCs) connected to the Internet (1301) as remote work PC1 (1302) and remote work PC2 (1303) are PCs loaned to each remote worker. The remote work PC1 (1302) and remote work PC2 (1303) are described on the assumption that they are used by different employees at their homes. The company's network is also connected to the Internet (1301) via a company connection device (1304). A firewall is installed on the company connection device (1304) used for connection to provide security measures. A company server (1308) is connected to the company's internal line (1305) on which files created by each employee and data processed by the company's core business system are stored. In the company's office, the company's internal PC1 (1306) and company's internal PC2 (1307) are connected to the company's internal line (1305) to carry out business operations. Therefore, for example, a supervisor who is present in the company office can hold a web conference with a subordinate who is working remotely from home, or a supervisor and a subordinate who is working remotely outside the company can mutually process data files on the company's server (1308).

<Functional Configuration of First Embodiment>
1 is a diagram showing the functional blocks of the remote work information protection system of this embodiment. As shown in the figure, the remote work information protection system (0100) of the first embodiment is composed of a display unit (A) (0101), a business use state detection unit (B) (0102), a viewer side image acquisition unit (C) (0103), a voyeurism available device discrimination information storage unit (D) (0104), a device existence determination unit (E) (0105), and a detection information output unit (F) (0106).

The above functional blocks are merely examples for implementing the present invention, and functions may be omitted or new functions may be added as appropriate to the extent that does not contradict the problems to be overcome by the present invention and its effects.

Each of the functional blocks constituting the information protection system described below can be realized by hardware, software, or both. Specifically, in the case of a computer, the following may be used: a CPU, main memory, GPU, image memory, graphic board, bus, or secondary storage device (such as a hard disk, non-volatile memory, storage media such as CDs and DVDs, and read drives for these media), input devices such as operation buttons used for inputting information, a mouse, a touch panel, an electronic pen used solely for touching a touch panel, a joystick or a joystick-like pointer position input device, and other external peripheral devices; interfaces for these external peripheral devices, a GPS receiving interface, a GPS computing device, a gyro sensor, an acceleration sensor, a rotation detection sensor, signal processing devices for these sensors, a camera, an image file processing circuit, a speaker, a microphone, an audio file processing circuit, a communication interface, a barcode reader, an electronic card reader, a POS terminal, a face authentication device, an encryption device, a biometric authentication device such as a fingerprint authentication device, a palm print authentication device, or a retina authentication device; driver programs and other application programs for controlling these hardware devices. In particular, it uses smartphones, tablet devices, mobile phones, smartwatches, personal computers, data center server equipment, wired and wireless networks and interfaces, etc.

The CPU performs calculations according to the programs deployed in the main memory, processing and storing data input from input devices and other interfaces and held in memory and hardware, and generating commands to control the hardware and software. Here, the above programs may be realized as multiple modularized programs, or two or more programs may be combined into one program.

The present invention can also be partially configured as software. Furthermore, storage media on which such software is recorded are naturally included within the technical scope of the present invention (this is not limited to this embodiment, but is the same throughout this specification).

<Description of Configuration of First Embodiment>
<Embodiment 1: Display unit (A) (0101)>
The display unit (A) (0101) is configured to have a display that displays images.
If the PC being used is a notebook PC, the built-in display will be the main display, but a PC display can also be added and used. If the built-in display has a small screen size and limited resolution, using a display with a larger screen will improve work efficiency. Work efficiency can be further improved by working on multiple screens in combination with the notebook PC's built-in display, rather than using only the PC display. Using multiple displays is also common with desktop PCs.
When multiple displays are used, it is desirable that the viewing side video acquired by the viewing side video acquisition unit (C) described below is acquired for all displays. When multiple displays are used on a PC for business use, in order to detect which display is displaying a business file, a means must be taken to analyze the signal from the video signal processing circuit of the PC, which increases the cost of constructing the means and the processing load on the PC. If a PC with sufficient processing power is used, it may be configured to detect by switching between acquisition and non-acquisition on a display-by-display basis as needed.

<First embodiment: Business usage status detection unit (B) (0102)>
The business use state detection unit (B) (0102) is configured to constantly detect whether the display unit is in a business use state, which indicates that the display unit is being used for business, while the display is being driven. It is not necessary to use a display signal or a display drive circuit signal to detect whether the display unit is being used for business, but rather, to constantly detect whether the PC is being used to access a business folder or whether the PC is connected to a corporate line via the Internet while the display is being driven. The method of detecting whether the display unit is in a business use state can be, for example, by detecting access to a specific folder in the PC (such as a document folder, a download folder, or a folder that stores only business software or business data with a specified name), access to a specific URL, a specific server-side file, access to a specific file in the PC, or a connection to a corporate line via the Internet by connecting to a VPN server.
Alternatively, if the PC used for business purposes is within a company network, the information protection system can be made not to operate since it is under the company's security management, or conversely, the information protection system can be made to operate continuously while the PC is turned on, since it is always used for business purposes.
A management server that acquires and manages the status of a PC used for business, whether it is inside or outside the corporate network, may be provided outside or inside the corporate network, the PC may be started, and the management server may request and acquire device identification information associated with connection status information such as the IP address of the PC from the PC that is connected to the corporate network or the Internet outside the corporate network after the information protection system is started. The management server may further include a network discrimination unit that determines whether the PC is connected to the corporate network or the Internet outside the corporate network based on the device identification information associated with the connection status information provided by the PC, a network judgment result information transmission unit that transmits network judgment result information, which is information indicating the judgment result, to the PC, and a business use status detection unit may acquire the network judgment result information in the PC used for business and determine whether it is in a business use state (outside the company).
If a remote worker and the company have agreed that the use of the PC outside of work is prohibited, the entire period from when the PC is started to when it is shut down or goes into sleep mode may be subject to detection. "Continuous detection" refers to a detection state in which the detection cycle is at most a few seconds or less. It is preferable that the cycle is no longer than 10 seconds. If it exceeds 10 seconds, voyeurism and the like may be possible during that time.

<Embodiment 1: Viewer-side video acquisition unit (C) (0103)>
The viewer's side image acquisition unit (C) (0103) is configured to acquire the viewer's side image, which is the image of the side viewing the display, while the business use state detection unit (B) (0102) detects that the display is in a business use state.
This system acquires video within the range where the display of a PC used for remote work can be viewed, and captures video information from a camera connected to the PC. When one real camera is currently being used for work, various known methods can be used to capture video information from the camera, such as providing a virtual camera using software or using a Windows Camera Frame Server (registered trademark) to acquire two videos, one for business use and one for viewing, from one real camera. In the method using a virtual camera, in addition to the information protection system of the present invention, for example, when a web conference app requests access to the camera, the video information is distributed from the virtual camera. When two real cameras are connected to one PC, it may not be necessary to use a method such as a virtual camera. Although one camera is sufficient, it is preferable to add more cameras if there is a blind spot in the range where the display used for work can be viewed. In particular, when multiple displays are used, it is preferable to add more cameras or install them in a place where there is no blind spot in order to more accurately determine whether or not a device that can be used for voyeurism, as described below, is present within the viewing range.

The blind spots of the display unit (A) and the viewer's image acquisition unit (C) will be explained using FIG. 25. FIG. 25a shows the blind spots of the camera (2502a) on the top of the display unit (A) (2501a) of a PC display used for an additional display of a notebook PC or a desktop PC, and a smartphone (2503a), and FIG. 25b shows the blind spots of the camera (2502b) on the top of the display unit (A) (2501b) of a notebook PC, and a smartphone (2503b). Generally, the angle of view (2504a) of a web camera connected to a PC is about 75 degrees, and the angle of view (2504b) of a camera built into a notebook PC is about 75 to 90 degrees. There are also wide-angle web cameras with an angle of view of about 150 degrees for the purpose of capturing images of multiple people, but they are not suitable for use by a single employee. An example of an angle of view of about 75 degrees will be explained, but the same applies even if the specifications of the angle of view change. FIG. 25a will be used as an example, but the following description is similar to FIG. 25b, and each symbol a may be read as b. The angle of view of the camera of the smartphone (2503a) located in front of the display unit (A) (2501a) is θ (in FIG. 25a, θ=75 degrees), and the range is indicated by a dotted line. In FIG. 25a, the area immediately below the front of the display unit (A) (2501a) is outside the angle of view (2504a) of the camera (2502a), and is an area that cannot be acquired as the viewing side image. Even if the smartphone (2503a) that can capture images is present in the area, it cannot be detected. Therefore, it is preferable to take measures such as adding a camera (not shown) that acquires the viewing side image of the lower blind spot of the camera (2502a) to the side or below the display unit (A) (2501a). In this case, the image of the added camera is not used for business purposes, but is used only as the viewing side image. Therefore, in this case, two types of images are used for the viewing side image. Since the two types of video have different shooting ranges and angles, the information for identifying devices usable for voyeurism must be prepared according to the respective cameras. For example, when capturing the viewer's video from the side of the display unit, the image of the device usable for voyeurism will be from the side, so it is preferable to prepare information for identifying devices usable for voyeurism, such as smartphones, from the side as the information for identifying devices usable for voyeurism.

<Embodiment 1: Secret photography available device discrimination information storage unit (D) (0104)>
The secret photography available device discrimination information holding unit (D) (0104) is configured to hold secret photography available device discrimination information, which is information for discriminating devices that can use secret photography.
The devices that can be determined as devices that can be used for voyeurism are devices that have a photographing function, such as smartphones, camera-equipped mobile phones, digital cameras, tablet terminals, etc. It may or may not be the device presence determination unit (E) described later that determines whether or not these devices are in a state where they can photograph the display that shows the image from the PC used for business.

<Embodiment 1: Device presence determination unit (E) (0105)>
The device presence determination unit (E) (0105) is configured to determine whether or not a device capable of using secret photography is present in the viewing side video, using the obtained viewing side video and the secret photography available device discrimination information.
During remote work, cases where a remote worker's smartphone receives a call, receives an e-mail, or searches the web on the smartphone are assumed to be examples of cases where a device is present in the viewing side video. Since these situations are considered to occur frequently, if it is determined that all devices with a shooting function are in a shooting-enabled state and that a device that can be used for voyeurism exists, this may reduce work efficiency. For this reason, it is preferable to configure the device presence determination unit (E) (0105) to determine whether or not the device is in a shooting-enabled state, but this is not limited to this. The device presence determination unit may first determine that a device that can be used for voyeurism exists in the viewing side video, and then transfer the determination result to another functional unit that determines whether the device that can be used for voyeurism is in a voyeur-enabled state. In other words, with regard to a device that can be used for voyeurism, the device presence determination unit may determine in one go that a device that can be used for voyeurism exists and is in a voyeur-enabled state, or it may determine whether or not a voyeur-enabled device exists, but not determine that the device that can be used for voyeurism is in a voyeur-enabled state, and leave that function to another functional unit. In this case, the functional part may exist on the PC used for remote work, or the functional part may be configured to exist on a business use server device that is capable of communicating with the remote work PC.

<Detection of devices that can be used for voyeurism>
The following method is a known technique for detecting an object in a viewing side image. A window-shaped image of a fixed size is acquired from the acquired viewing side image at all possible positions. An example of the fixed size is the size of the image data containing the retained device discrimination information usable for voyeurism. The image is classified by an image classification means for classifying whether or not the acquired image area contains the retained device discrimination information usable for voyeurism. Since the size in the viewing side image varies with distance, image data is prepared by enlarging or reducing the image data containing the device discrimination information usable for voyeurism to a plurality of sizes. The object is completely contained within a window of a size selected from any of these images. If a window that completely contains the object can be specified, the position information of the object can be obtained from the acquisition position of the window, and a classification result of whether or not the object is a device usable for voyeurism can be obtained from the image classification means. The HOG (Histogram of Oriented Gradients) method can be used for classification by the image classification means.
The detection of an object and the determination of whether or not it is in a state where surreptitious photography is possible based on its orientation and distance may be processed consistently by the device presence determination unit (E) as described above, or the function may be separated.
If the system is configured to detect and judge an object in the acquired viewer-side video for each still image frame, it may be detected even if a family member with a smartphone happens to pass behind the remote worker working from home. It may also be detected even if the remote worker takes out his or her smartphone to make a phone call and operates it in front of the PC. Therefore, it is preferable to configure the system to detect the presence of a device if the same device usable for voyeurism is detected consecutively over 3 to 10 or more frames of images when acquiring images at 30 frames per second.
Also, a still image of a frame spanning one to two or three seconds may be analyzed to analyze the motion of the device usable for voyeurism in the viewing-side video and the person holding it, and the device presence determination unit may determine the presence of a device usable for voyeurism based on the motion analysis result. Motion analysis refers to a case where the camera lens of the device usable for voyeurism faces the display surface for a predetermined time, or a case where the thumb or the fingers of the hand not holding the camera can recognize the presence of a motion of operating the touch panel or buttons arranged on the side of the device usable for voyeurism while facing the display surface. This recognition determination is performed by using artificial intelligence or a function similar to artificial intelligence to retain recognition information through machine learning (using multiple human models to take voyeur poses and repeatedly providing feedback to the judgment result of artificial intelligence, etc.), and when the motion has a high similarity to this recognition information, it is determined that a device usable for voyeurism exists.

<Embodiment 1: Detection information output unit (F) (0106)>
The detection information output unit (F) (0106) is configured to output detection information, which is information indicating the presence, when the determination result of the device presence determination unit (E) is that the device exists.
When a device capable of being used for voyeurism is present in the viewing-side video, the detection information may be output to the PC used by the remote worker, the superior of the company to which the remote worker provides labor, or a monitoring device, PC, server, smartphone, landline phone, etc. in the system management department of the company. The output method may be, for example, a display on the display of the PC used by each person, delivery of an email to an address that can be received on a smartphone or PC, or an automatic phone call to a smartphone or mobile phone. The information protection system of the present invention may be appropriately set by the company that uses it.
In addition, it is preferable to immediately switch the display to another image or to stop displaying the image when the detection information is output, in order to reduce the risk of information leakage. Examples of other images include a black screen, a viewer's image obtained from a camera, a Windows (registered trademark) login screen, and a warning screen such as "This is a voyeuristic video!". There is also a method of repeatedly displaying information that can identify the source of information leakage, such as the PC user, the PC's device management number, or the device's IP address, over the entire screen like a watermark.
There are three possible timings for outputting the detection information: outputting the detection information when a device that can be used for secret photography is detected, outputting the detection information when a device that can be used for secret photography is detected and it is determined that photography is possible, and two-stage output that outputs a warning when a device that can be used for secret photography is detected and then outputs the detection information when it is determined that photography is possible.
In addition, a microphone connected to a PC may be used in combination to detect voyeurism using a device that can be used for voyeurism. When taking a picture with a camera on a mobile phone, including a smartphone, sold in Japan, a shutter sound or other sound is emitted due to self-regulation by the telecommunications carrier so that the user can know that a picture has been taken. If an app that blocks the shutter sound or other sound is used, it cannot be detected, but if no app is used and the PC microphone is not muted, the shutter sound is an effective means of detecting whether or not a picture has been taken.

The business usage status detection unit (B) (0102), the viewing side video acquisition unit (C) (0103), the voyeurism available device discrimination information storage unit (D) (0104), the device presence determination unit (E) (0105), and the detection information output unit (F) (0106) constituting the remote work information protection system (0100) of the first embodiment of the present invention may be located in either the remote worker PC 1 (1302) or the remote worker PC (1303), or the company side server (1308) shown in FIG. 13. Although not shown in FIG. 13, they may also be located on a cloud server of another company that is contracted and used by the company. All of them may be located together in one of the three locations, or they may be located separately, with some on the remote worker PC and the others on the company side (including cloud servers of other companies). For example, the viewer-side video acquisition unit (C) (0103) may be configured to acquire video captured from a camera used by a remote worker to the company-side server (1308). The same applies to other configurations. The same applies to the configurations described in the second and subsequent embodiments.

<Processing flow of embodiment 1>
2 is a flowchart of the operation process of the information protection system of embodiment 1. As shown in this figure, the information protection system of embodiment 1 has a business use state detection step (b) (S0201), a viewer side video acquisition step (c) (S0202), a voyeurism available device determination information retention step (d) (S0203), a device presence determination step (e) (S0204), and a detection information output step (f) (S0205).

Here, the method of operating the remote work information protection system including a computer having a display unit (A) equipped with a display that displays an image is as follows:
The business use state detection step (b) (S0201) performs a process of constantly detecting and determining whether the display unit is in a business use state, which indicates that the display unit is being used for business, while the display is being driven;
The viewer side image acquisition step (c) (S0202) performs a process of acquiring a viewer side image, which is an image on the side of a viewer viewing the display, while the business use state detection step (b) (S0201) detects that the display is in a business use state;
The secret photography available device determination information holding step (d) (S0203) performs a process of holding secret photography available device determination information, which is information for determining a secret photography available device,
The device presence determination step (e) (S0204) performs a process of determining whether a device available for voyeurism is present in the viewing side image using the obtained viewing side image and the voyeurism available device discrimination information,
A detection information output step (f) (S0205) performs a process of outputting detection information indicating the presence of the device when the determination result in the device presence determination step (e) (S0204) indicates the presence of the device,
The business use status detection step (b) (S0206) constantly detects and determines whether the display unit is in a business use status, which indicates that the display unit is being used for business, while the display is in operation, and if the display unit is in a business use status, the process returns to before the viewer side image acquisition step (c) (S0202).
If it is determined in device presence determination step (e) (S0204) that the device is not present, the next step of detection information output step (f) (S0205) is not performed, and processing is performed to skip to before business usage status detection step (b) (S0206).
This is an operating method for causing the information protection system to execute such a series of processes.

<First embodiment: Description of hardware>
FIG. 3 is a diagram for explaining hardware configured by a main board and its peripheral devices of the information protection system, which is a computer (calculator) of the first embodiment. It is configured by a "CPU", a "chipset" consisting of a north bridge and a south bridge, a "non-volatile memory", a "main memory", an "I/O controller", a "USB, IEEE1394, HDMI (registered trademark), LAN terminal, etc.", a "BIOS", a "PCI slot", a "real-time clock", and the like. In this embodiment, a camera, a microphone, and a speaker configure the hardware of a PC by USB connection. For example, the microphone and the speaker may be input and output via a sound board connected to a PCI slot, and the camera does not have to be via USB. As long as video input and audio input and output can be performed in the same way as when connected via USB, the effect of the present invention can be obtained in the same way. It is assumed that a display is also connected using an HDMI (registered trademark) terminal. It may be connected via a connection terminal on a graphics card. Also, a known display connection terminal other than HDMI (registered trademark) may be used.

The various programs and data (information) stored in the non-volatile memory are expanded into the main memory when the system is started, and the programs are configured to perform calculations using the data in sequence by the CPU upon receiving execution commands.

In other words, the non-volatile memory contains, in addition to the OS (operating system) and device drivers, a business use status detection program, a viewing side video acquisition program, a program for retaining information on devices available for voyeurism, a program for judging the presence of a device, a detection information output program, a business program, and a non-business program, and stores the following data: viewing side video, information on devices available for voyeurism, detection information, business data, and non-business data. These are expanded into the main memory when the system is started up on a computer, and when a start-up command is received, the CPU sequentially performs calculations using the programs and data. The non-volatile memory stores not only the programs and data related to the remote work information protection system of the present invention, but also at least business programs necessary for remote work (e.g., web conferencing programs, programs for connecting to intranets, programs for creating materials, virus detection programs, etc.), non-business programs (e.g., browser programs for browsing Internet websites, game programs, etc.), data necessary for the operation of business programs and data created using business programs, data necessary for the operation of non-business programs and data created by operating them, etc. These are also loaded into the main memory when started on the computer, and when a start command is received, the CPU sequentially performs calculations using the programs and data.

<Outline of the second embodiment> Mainly claim 2 (face part image judgment)
The second embodiment is based on the information protection system of the first embodiment, and is configured to have a facial portion image determination unit (G) that determines whether a partial image including a face is present in the image viewed from the display unit (A) that displays the image from the PC, a facial portion image composition information acquisition unit (H) that acquires facial portion image composition information if present, and a facial portion image composition information storage unit (J) that stores the acquired facial portion image composition information.

<Configuration of Second Embodiment>
Fig. 4 shows a functional block diagram of an information protection system of the second embodiment based on the first embodiment. As described above, in addition to the first embodiment, the second embodiment further includes a face part image determination unit (G) (0407), a face part image composition information acquisition unit (H) (0408), and a face part image composition information storage unit (J) (0409). Since the components other than the face part image determination unit (G) (0407), the face part image composition information acquisition unit (H) (0408), and the face part image composition information storage unit (J) (0409) are the same as the first embodiment, only the face part image determination unit (G) (0407), the face part image composition information acquisition unit (H) (0408), and the face part image composition information storage unit (J) (0409) will be described below.

<Description of Configuration of Second Embodiment>
<Embodiment 2: Facial image determination unit (G) (0407)>
The 'face portion image determination unit (G)' (0407) is configured to determine whether or not a face portion image, which is a partial image including a face, exists in the acquired viewer side image.
It is determined whether there is an image including the face of the remote worker or a third party in the viewing side image obtained from the camera connected to the PC. Here, a third party refers to a person other than the remote worker who is in a state where he or she can intentionally or accidentally see the display surface (A). For example, FIG. 14a is an example of an image obtained from a camera. A bust-up image of the remote worker (1401) is shown. The face part image (1402), which is a partial image including the face, corresponds to the rectangular dotted frame area around the face of the remote worker (1401) shown in FIG. 14b. In FIG. 14b, the face part image area is shown as a rectangle, but it may be a circle or an ellipse, or the outline of the face may be detected and only the image within the outline may be extracted. When a person leaves the display and there is no face part image, there is only the background as shown in FIG. 17a and there is no rectangular dotted frame area showing the face part image.
The following methods are available for recognizing the face part. The viewer's video is captured at a speed according to the processing capacity of the device, such as 30 frames per second. Each captured frame is a still image, and two types of discrimination are performed: a face area cut out to a specific size and a non-face area. A face detection unit detects a face by using an extraction unit that extracts components of a figure called a face using known machine learning and pattern recognition techniques, and a discrimination unit that discriminates an area including the extracted components. Various face images are read and the extracted components are learned, but a face can only be recognized from an evaluation image of the same size as the face image used as the learning material. A binary detection is performed on the entire area of a still image in a frame of the viewer's video to be detected, whether or not there is a face for each area of the size that has been learned. A binary detection is performed on the entire area for each image in which the size of the viewer's video to be detected is enlarged or reduced in several steps. This method integrates the results of binary detection performed on one image by enlarging or reducing it to obtain the result of face detection.

<Embodiment 2: Facial image configuration information acquisition unit (H) (0408)>
The "facial part image composition information acquisition unit (H)" (0408) is configured to acquire facial part image composition information, which is information constituting the facial part image, when the facial part image judgment unit (G) judges that a facial part image exists.
In Fig. 14b, the region of the face part image is shown as a rectangle, but it may be a circle or an ellipse, or the contour of the face may be detected and only the image within the contour may be extracted and acquired as face part image configuration information. Alternatively, instead of the face image information as it is, it may be information only of numerical data such as the relative position coordinates and size of the shapes of the parts that constitute the face, such as the eyes, nose, mouth, and cheekbones, which are prominent features and are required for recognizing the face using artificial intelligence (hereinafter referred to as AI). Alternatively, standard coordinate data of the parts that constitute the face, such as the eyes, nose, and mouth, may be stored, and the acquired data may be numerical data indicating the difference between the obtained numerical data and the standard data.
If all the images obtained from the camera as shown in FIG. 14a are acquired in addition to the facial image configuration information, problems such as an increase in storage capacity when stored in the facial image configuration information storage unit (J) described later, and concerns about the infringement of the privacy of the remote worker due to the storage of images other than the face of the remote worker (such as clothing and the state of the work space reflected behind). In particular, problems are likely to occur when information showing personal preferences (posters on the wall, titles of books on the shelves, how tidy the room is, etc.) is reflected behind the remote worker, or when the remote worker's working hours are irregular and he or she wears casual clothing that does not conform to business practices (clothing that exposes a lot of skin, pajamas, etc.). Therefore, if priority is given to storing the entire viewer-side image so that the position of the facial image can be identified rather than increasing the storage capacity, it is preferable to store the parts other than the facial image by performing soft focus processing, mosaic processing, or masking processing. In FIG. 14b, for convenience of explanation, the above processing is expressed by darkening the background as the processing of the parts other than the face. This also applies to FIG. 15 to FIG. 21, not just FIG.

<Second embodiment: Facial image configuration information storage unit (J) (0409)>
The 'storage unit for face part image configuration information (J)' (0409) is configured to store the acquired face part image configuration information.
Examples of images obtained from a camera connected to a PC when a remote worker is positioned in front of a display unit that displays an image from the PC will be described with reference to FIGS. 14 to 17. FIG.
FIG. 14ab shows a normal remote worker state. FIG. 17ab shows a state where the remote worker has left his/her desk. FIG. 15ab shows an image in which the remote worker wears glasses for business reasons. FIG. 16ab shows an image in which the remote worker wears a mask for health reasons. Although not shown in particular, when the remote worker wears makeup, the appearance changes depending on the degree of makeup. In order to obtain a dangerous image to be described later in the third embodiment, it is desirable to configure the remote worker so that the same person can be recognized even if the remote worker's appearance changes slightly due to wearing glasses or a mask or makeup, as described above. The remote worker may be identified using AI, or images for each case may be prepared and registered in advance. Alternatively, if it is difficult to authenticate the remote worker as the person in question, the company's superior or the system department may be contacted each time to have the image of the face that is difficult to authenticate approved and registered.

<Processing flow of embodiment 2>
5 is a flowchart of the operation process of the information protection system of embodiment 2. As shown in this figure, the information protection system of embodiment 2 includes a business use state detection step (b) (S0501), a viewer side image acquisition step (c) (S0502), a face part image determination step (g) (S0503), a face part image configuration information acquisition step (h) (S0504), a face part image configuration information storage step (j) (S0505), a voyeurism available device discrimination information storage step (d) (S0506), a device presence determination step (e) (S0507), and a detection information output step (f) (S0508).

Here, the method of operating the remote work information protection system including a computer having a display unit (A) equipped with a display that displays an image is as follows:
The business use state detection step (b) (S0501) performs a process of constantly detecting and determining whether the display unit is in a business use state, which indicates that the display unit is being used for business, while the display is being driven;
A viewer-side image acquiring step (c) (S0502) performs a process of acquiring a viewer-side image, which is an image on the side of a viewer viewing the display, while the business use state detecting step (b) (S0501) detects that the display is in a business use state;
A face part image determination step (g) (S0503) performs a process of determining whether a face part image, which is a partial image including a face, exists in the acquired viewer side image. A face part image configuration information acquisition step (h) (S0504) performs a process of acquiring face part image configuration information, which is information that configures the face part image, when the face part image determination step (g) (S0503) determines that a face part image exists.
A face part image composition information holding step (j) (S0505) performs a process of holding the acquired face part image composition information,
The secret photography available device determination information holding step (d) (S0506) performs a process of holding secret photography available device determination information, which is information for determining a secret photography available device,
The device presence determination step (e) (S0507) performs a process of determining whether a device available for voyeurism is present in the viewing side image using the obtained viewing side image and the voyeurism available device discrimination information,
In the detection information output step (f) (S0508), when the determination result in the device presence determination step (e) (S0507) is that the device exists, detection information indicating the existence of the device is output.
The business use status detection step (b) (S0509) constantly detects and determines whether the display unit is in a business use status, which indicates that the display unit is being used for business, while the display is in operation, and if the display unit is being used for business purposes, the process returns to before the viewer side image acquisition step (c) (S0502).
If it is determined in device presence determination step (e) (S0507) that the device does not exist, the next step of detection information output step (f) (S0508) is not performed, and processing is performed to skip to before business usage status detection step (b) (S0509).
This is an operating method for causing the information protection system to execute such a series of processes.

<Description of Hardware in Second Embodiment>
FIG. 6 is a diagram for explaining hardware configured by a main board and its peripheral devices of the information protection system, which is a computer (calculator) of the second embodiment. It is configured by "CPU", "chip set" consisting of a north bridge and a south bridge, "non-volatile memory", "main memory", "I/O controller", "USB, IEEE1394, HDMI (registered trademark), LAN terminal, etc.", "BIOS", "PCI slot", "real-time clock", etc. In this embodiment, a camera, a microphone, and a speaker configure the hardware of a PC by USB connection. For example, the microphone and the speaker may be input and output via a sound board connected to a PCI slot, and the camera does not have to be via USB. As long as video input and audio input and output can be performed in the same way as when connected via USB, the effect of the present invention can be obtained in the same way. It is assumed that a display is also connected using an HDMI (registered trademark) terminal. It may be connected via a connection terminal on a graphics card. Also, a known display connection terminal other than HDMI (registered trademark) may be used.

The various programs and data (information) stored in the non-volatile memory are expanded into the main memory when the system is started, and the programs are configured to perform calculations using the data in sequence by the CPU upon receiving execution commands.

In other words, the non-volatile memory contains, in addition to the OS (operating system) and device drivers, a business use status detection program, a viewing side video acquisition program, a voyeurism usable device discrimination information retention program, a device presence determination program, a detection information output program, a facial part video determination program, a facial part video composition information acquisition program, a facial part video composition information retention program, a business program, and a non-business program, and stores the viewing side video, voyeurism usable device discrimination information, detection information, facial part video composition information, business data, and non-business data as data. These are expanded into the main memory when the system is started up on the computer, and by receiving a start-up command, the CPU sequentially performs calculations using the programs and data. The non-volatile memory stores not only the programs and data related to the remote work information protection system of the present invention, but also at least business programs necessary for remote work (e.g., web conferencing programs, programs for connecting to intranets, programs for creating materials, virus detection programs, etc.), non-business programs (e.g., browser programs for browsing Internet websites, game programs, etc.), data necessary for the operation of business programs and data created using business programs, data necessary for the operation of non-business programs and data created by operating them, etc. These are also loaded into the main memory when started on the computer, and when a start command is received, the CPU sequentially performs calculations using the programs and data.

<Outline of the third embodiment> Mainly claim 3 (holding dangerous video and viewing information identification information in association with each other)
The third embodiment is based on the information protection system of either the first or second embodiment,
When the detection information output unit (F) outputs detection information, it is configured to have a danger information holding unit (K) that holds danger information, which is information that associates a danger image, which is all or part of the viewing side image, with viewing information identification information that identifies the information being shown on the display.

<Configuration of Third Embodiment>
Fig. 7 shows a functional block diagram of an information protection system of embodiment 3 based on embodiment 1. As described above, in addition to embodiment 1, it is configured to have a danger information holding unit (K) (0710). Since everything other than the danger information holding unit (K) (0710) is the same as embodiment 1, only the danger information holding unit (K) (0710) will be described below. Note that the same effect can be obtained even if embodiment 2 is used as a base.

<Description of the configuration of embodiment 3>
<Third embodiment: Danger information storage unit (K) (0710)>
The "danger information storage unit (K)" (0710) is configured to store danger information, which is information that correlates a dangerous image, which is all or part of the viewing side image, with viewing information identification information that identifies the information being shown on the display, when the detection information output unit (F) (0706) outputs detection information.
Since the detection information is output when a device that can be used for voyeurism is detected and a security risk occurs, in order for the company to understand what information may have been captured under what circumstances, it is necessary to hold face part configuration information and dangerous images of the part including the device that can be used for voyeurism, and in order to understand what information may have been leaked, the viewing information identification information shown on the display is associated and held. Examples of viewing information identification information include a captured image of the entire display screen, and a list of the software names and data file names that were running at the time.

<Embodiment 3: Dangerous Information Storage Unit (K) Dangerous Image>
18 to 21 show examples of dangerous images.

<Embodiment 3: Danger information storage unit (K) Danger image Unclaimed invention>
Although not described in the claims, Fig. 18a and 18b are examples showing the risk of shoulder hacking, where a third party (1802) peeks at the display over the shoulder of a remote worker (1801). If the viewer's video captured by the camera is not displayed on the display unit (A), the remote worker (1801) cannot notice the third party (1802) behind him. In order to avoid this risk, an employee facial part composition information holding unit is provided which registers and holds the facial part composition information of the remote worker in advance in the information protection system of the present invention, an employee facial part composition information authentication unit which compares and authenticates the facial part composition information of the remote worker acquired during business use of the PC, and a non-employee detection information output unit which outputs face detection information which is detection information indicating that facial part composition information which cannot be authenticated by the employee facial part composition information authentication unit has been detected, and when facial part composition information of a third party (1802) who cannot be authenticated by the employee facial part composition information authentication unit, in addition to the remote worker (1801), is detected in the viewing side video, the non-employee detection information output unit outputs the face detection information, and dangerous video including the facial part video or facial part composition information (1804) of the third party (1802) in the viewing side video can be held as dangerous information associated with viewing information identification information. When the face part configuration information of a remote worker is registered and stored in the employee face part configuration information storage unit in advance, it is preferable to register and store not only one face part configuration information but also face part configuration information according to changes in information when wearing glasses or a mask, face direction (sideways, up, down, etc.), and makeup level (depth and presence or absence of makeup). If an image of a person wearing a mask is not registered in advance and the person is not recognized as an employee, the face part configuration information for authentication may be registered and stored in the employee face part configuration information storage unit after notifying an authorized superior or a system management department and receiving confirmation and approval each time. Alternatively, the face part configuration information may be determined to be authenticated using a known face authentication means using AI from information on parts that do not change depending on the presence or absence of makeup, such as the position coordinates or relative coordinates of parts such as the eyes, nose, and mouth, and if authentication is still not possible, the decision may be left to a human such as a superior.

In order to detect the face part configuration information of the third party as described above, it is necessary to constantly detect in the viewing side video. This "constant detection" refers to a detection state in which the detection cycle is at most a few seconds or less. It is preferable that the cycle is no longer than 10 seconds. If it exceeds 10 seconds, it is possible for someone to peek during that time. On the other hand, if the remote worker himself or a third party in the acquired viewing side video is detected and judged for each still image frame, it may be detected even if a family member happens to pass behind the remote worker who is working from home. In addition, even if the remote worker turns around for a moment when called by a family member, stares at the ceiling for a few seconds thinking, or covers and rubs his eyes with his hands, it may be detected if it is a mode that has not been registered and stored as face part configuration information in advance. Therefore, for example, when images are acquired at 30 frames per second, it is preferable to configure it so that the presence of the remote worker himself or a third party is detected if clothing or a body shape that can be determined to be that of the same person is detected continuously over 3 to 10 or more frames of images.

It may also be configured to analyze still images of frames over a period of about 1 to 2 or 3 seconds, analyze the motion of the remote worker himself or a family member passing by by chance in the viewing side video, and determine the presence of the remote worker himself or a third party based on the motion analysis results. Motion analysis refers to a case where the remote worker himself or a third party faces the display screen for a predetermined period of time, or a case where a person in front of the display screen turns their head, looks to the side, looks down, covers their face with their hands, or other temporary movements while looking at the display screen during face authentication to determine whether they are facing the display screen. This recognition judgment is made by using artificial intelligence or a function similar to artificial intelligence to retain recognition information through machine learning (multiple human models are made to pose and feedback is repeatedly given to the judgment results of the artificial intelligence, etc.), and when the motion is highly similar to this recognition information, it is determined that the remote worker himself or a third party is present.

Although not stated in the claims, Figure 19a and 19b are examples showing the risk that a third party (1901) may view a PC screen while a remote worker is away from his/her desk when working remotely in a public or semi-public place, such as a coffee shop or a shared office. To avoid this risk, an employee facial part configuration information storage unit is provided that registers and stores the facial part configuration information of the remote worker in advance in the information protection system of the present invention; an employee facial part configuration information authentication unit compares and authenticates the facial part configuration information of the remote worker acquired during business use of the PC; and a non-employee detection information output unit outputs face detection information, which is detection information indicating that the employee facial part configuration information authentication unit has detected facial part configuration information that cannot be authenticated. When the facial part configuration information of the remote worker (1901) is not detected in the viewing side video and the facial part configuration information of a third party (1902) that cannot be authenticated by the employee facial part configuration information authentication unit is detected, the non-employee detection information output unit outputs face detection information, and dangerous video including the facial part video or facial part configuration information (1902) of the third party in the viewing side video can be stored as dangerous information associated with viewing information identification information.

When the detection information or face detection information is output, it is preferable to make the display on the display unit (A) non-displayed, but the response can be changed depending on the content of the business information being displayed. The non-displayed state here refers to stopping the output of the display signal to the display unit (A) or displaying a different screen (a warning message against voyeurism, a viewer's video, a black display, a login screen for a PC, etc.) that is different from the screen used for business purposes.
It is also possible to configure the system so that different security measures are used depending on whether only detection information indicating that a device that can be used for voyeurism is present in the viewing-side video is output, face detection information indicating that face-part configuration information of a non-employee third party is detected is output, or both are output. For example, when general business information such as a remote worker's schedule is displayed, the business screen can be configured to remain in the same state even if detection information is output due to the determination that a device that can be used for voyeurism is present. On the other hand, it is possible to configure the screen to be hidden when both detection information and face detection information are output, even if the schedule is displayed.
Furthermore, when business information including personal information of employees such as customers or remote workers is being displayed, if either the detection information or the face detection information is output, the information is not displayed.
To achieve this configuration, an information identifier that identifies the information displayed on the display can be associated with the business information, and optimal security can be activated by a combination of detection information, face detection information, and the information identifier.
In other words, security processing can also be configured so that a security identifier that identifies the security processing (security processing unit) is associated with the security processing (stored in a security identifier storage unit), a security identifier is selected based on the detection information, face detection information, and information identifier (security identifier selection unit), and the security processing corresponding to the selected security identifier is activated (the security processing unit starts processing when activated by the security processing activation unit).
In addition, when the detection information or face detection information is no longer detected, it is preferable to configure the display so that a predetermined time (about 3 to 5 seconds) is measured by a timer, and if there is no detection after the measurement, the hidden business information is restored to the display. In addition, when the hidden business information is restored, it can be configured so that it is restored on the condition that a password or biometric information or the like used by the employee who performed the work is input, or the employee's facial part configuration information is detected.

<Embodiment 3: Danger information storage unit (K) Danger image Part of the present invention>
20a and 20b show the risk of a remote worker (2001) taking a picture of a PC screen with a smartphone. The remote worker (2001) has a smartphone (2002) at hand.
Since a smartphone (2002), which is a device that can be used for voyeurism, is detected, facial portion configuration information (2003) of the remote worker (2001) present in the viewing side video and a partial video (2004) including the smartphone (2002) are retained as danger information.
FIG. 21a and FIG. 21b show the risk of a variation of shoulder hacking in which a third party (2102) takes a surreptitious photograph of a PC display unit with a smartphone (2103) while peeking. The third party (2102) is holding the smartphone (2103) while peeking at the display display over the shoulder of the remote worker (2101). If the viewing side image obtained by the camera is not displayed on the display unit, the remote worker (2101) cannot notice the third party (2102) behind him or the smartphone (2103) he is holding. Since the monitoring device of the present invention detects the smartphone (2002), which is a device that can be used for surreptitious photography, the face part configuration information (2104) of the remote worker (2102), the face part configuration information (2105) of the third party (2102), and the dangerous image (2106) including the smartphone (2103) present in the viewing side image are associated with a viewing information identifier and stored as danger information.
In carrying out business, a remote worker may take an image of the display unit (A) with a smartphone under the approval of the superior. In such a case, by providing an employee face part configuration information storage unit that registers and stores the face part configuration information of the remote worker in advance in the information protection system of the present invention, and an employee face part configuration information authentication unit that collates and authenticates the face part configuration information of the remote worker acquired during business use of the PC, as shown in Fig. 20, the face part configuration information in the viewing side image is only the face part configuration information (2003) of the remote worker (2001), and if the smartphone (2002) is captured to take an image of the display unit (A) with the approval or instruction of the superior for business reasons, it is possible to configure not to store it as dangerous information. This type of exception processing can be configured by outputting a photography permission application requesting permission to take photographs in association with an information identifier to a superior or a decision server that can make the actual decision (photography permission application output unit), and when photography permission information indicating that photography is permitted is returned in response (photography permission information acquisition unit), the exception processing can be performed as described above while the information identified by that information identifier is being displayed on the display (exception control unit).

When these dangerous images are retained, it is preferable to not retain images other than those extracted as dangerous images, or, when retaining the entire viewer-side image, to perform soft focus processing, mosaic processing, masking processing, etc. on the parts other than the dangerous images before retaining them to take privacy into consideration. Alternatively, as described above, it is possible to configure the system to normally process images other than the face-part configuration information, and to retain the background as evidence without image processing when acquiring dangerous information. However, in that case, special care must be taken in handling the retained dangerous information so as not to violate privacy. This is because, as described above, objects reflected behind the remote worker, such as items that indicate the remote worker's living environment and personal preferences, and the remote worker's own clothing, are also information that expresses privacy.
Needless to say, companies must also place importance on protecting the privacy of employees working remotely or in home-based contact centers.

<Processing flow of embodiment 3>
Fig. 8 is a flowchart of the operation process of the information protection system of embodiment 3. As shown in this figure, the information protection system of embodiment 3 has a business use state detection step (b) (S0801), a viewer side video acquisition step (c) (S0802), a voyeurism available device discrimination information storage step (d) (S0803), a device presence determination step (e) (S0804), a detection information output step (f) (S0805), and a danger information storage step (k) (S0806).

Here, the method of operating the remote work information protection system including a computer having a display unit (A) equipped with a display that displays an image is as follows:
The business use state detection step (b) (S0801) performs a process of constantly detecting and determining whether the display unit is in a business use state, which indicates that the display unit is being used for business, while the display unit is in operation;
A viewer-side image acquisition step (c) (S0802) performs a process of acquiring a viewer-side image, which is an image on the side of a viewer viewing the display, while the business use state detection step (b) (S0801) detects that the display is in a business use state;
The secret photography available device determination information holding step (d) (S0803) performs a process of holding secret photography available device determination information, which is information for determining a secret photography available device,
The device presence determination step (e) (S0804) performs a process of determining whether a device available for voyeurism is present in the viewing side image using the obtained viewing side image and the voyeurism available device discrimination information,
A detection information output step (f) (S0805) performs a process of outputting detection information which is information indicating the presence of the device when the determination result in the device presence determination step (e) is that the device exists;
A danger information holding step (k) (S0806) performs a process of holding a danger image, which is all or a part of the viewer-side image, in association with viewing information identification information that identifies information shown on the display when the detection information output step (f) outputs the detection information,
The business use status detection step (b) (S0807) constantly detects and determines whether the display unit is in a business use status, which indicates that the display unit is being used for business, while the display is in operation, and if it is in a business use status, the process returns to before the viewer side image acquisition step (c) (S0802).
If it is determined in device presence determination step (e) (S0804) that the device does not exist, the next detection information output step (f) (S0805) and danger information retention step (k) (S0806) are not performed, and the process skips to before business usage status detection step (b) (S0807).
This is an operating method for causing the information protection system to execute such a series of processes.

Third Embodiment Hardware Description
FIG. 9 is a diagram for explaining hardware configured by a main board and its peripheral devices of the information protection system, which is a computer (calculator) of the third embodiment. It is configured by "CPU", "chip set" consisting of a north bridge and a south bridge, "non-volatile memory", "main memory", "I/O controller", "USB, IEEE1394, HDMI (registered trademark), LAN terminal, etc.", "BIOS", "PCI slot", "real-time clock", etc. In this embodiment, a camera, a microphone, and a speaker configure the hardware of a PC by USB connection. For example, the microphone and the speaker may be input and output via a sound board connected to a PCI slot, and the camera does not have to be via USB. As long as video input and audio input and output can be performed in the same way as when connected via USB, the effect of the present invention can be obtained in the same way. It is assumed that a display is also connected using an HDMI (registered trademark) terminal. It may be connected via a connection terminal on a graphics card. Also, a known display connection terminal other than HDMI (registered trademark) may be used.

The various programs and data (information) stored in the non-volatile memory are expanded into the main memory when the system is started, and the programs are configured to perform calculations using the data in sequence by the CPU upon receiving execution commands.

In other words, the non-volatile memory contains, in addition to the OS (operating system) and device drivers, a business use status detection program, a viewing side image acquisition program, a voyeurism usable device discrimination information retention program, a device presence determination program, a detection information output program, a facial part image judgment program, a facial part image composition information acquisition program, a facial part image composition information retention program, a danger information retention program, a business program, and a non-business program, and stores the viewing side image, voyeurism usable device discrimination information, detection information, facial part image composition information, viewing information identification information, danger information, business data, and non-business data as data. These are expanded into the main memory when the system is started up on a computer, and when a start-up command is received, the CPU sequentially performs calculations using the programs and data. The non-volatile memory stores not only the programs and data related to the remote work information protection system of the present invention, but also at least business programs necessary for remote work (e.g., web conferencing programs, programs for connecting to intranets, programs for creating materials, virus detection programs, etc.), non-business programs (e.g., browser programs for browsing Internet websites, game programs, etc.), data necessary for the operation of business programs and data created using business programs, data necessary for the operation of non-business programs and data created by operating them, etc. These are also loaded into the main memory when started on the computer, and when a start command is received, the CPU sequentially performs calculations using the programs and data.

<Outline of the fourth embodiment> Mainly in claim 4 (Determination of whether photography is possible or not)
The fourth embodiment is based on any one of the information protection systems of the first to third embodiments,
The device presence determination unit (E) is configured to have a photographing availability state determination means (M) for determining whether the state of the device available for voyeurism use relative to the display unit (A), including its position and orientation, is in a state in which photographing is possible, based on the image of the device available for voyeurism use in the viewing side image and the voyeurism availability device discrimination information.

<Configuration of Fourth Embodiment>
Fig. 10 shows a functional block diagram of an information protection system of the fourth embodiment based on the first embodiment. As described above, in addition to the first embodiment, the fourth embodiment is configured to have a photographing permission status determination means (M) (1011) in the device presence determination unit (E). Since everything other than the photographing permission status determination means (M) (1011) is the same as the first embodiment, only the photographing permission status determination means (M) (1011) will be described below. The same effect can be obtained even if either the second or third embodiment is used as a basis.

<Fourth embodiment: Description of configuration>
<Embodiment 4: Image capture availability determination means (M) (1011)>
The "means for determining whether or not photographing is possible (M)" (1011) is configured within the equipment presence determination unit (E) so as to determine whether or not the state of the device available for use as a voyeur device, including its position and orientation relative to the display unit (A), is in a state in which photographing is possible, based on the image of the device available for use as a voyeur device in the viewing side image and the information on the device available for use as a voyeur device.
The device presence determination unit (E) (1005) determines whether a device available for voyeurism is present in the viewer's video. Next, the means for determining whether or not the device is available for voyeurism (M) (1011) determines whether or not the device is available for voyeurism (A) with its lens facing the display unit (A) and within the viewing angle.
During remote work, it is expected that remote workers' smartphones will receive calls, emails, and web searches. Since these situations are likely to occur frequently, if it is determined that all devices with a shooting function are in a shooting-enabled state and that there are devices that can be used for voyeurism, this could reduce work efficiency. For this reason, it is desirable to configure the system so that it can determine whether or not a device is in a shooting-enabled state.

<How to determine whether photography is permitted or not>
The method of determining whether or not the camera is ready to take a picture can be the same as that used for general object detection. Various methods have already been published, and the following method using AI-based BoVW (Bag of Visual Word) can be used.
As learning data, prepare images of devices such as smartphones and digital cameras that can be used for voyeurism. It is better to prepare images with different orientations and sizes. Extract local features from those images using known techniques such as SIFT (Scale-Invariant Feature Transform) and SURF (Speed Up Robust Features). The images of the extracted local features are characteristic parts such as the lens of the smartphone and the edge of the smartphone nearby, the corners of the smartphone, and the manufacturer's logo. When the above-mentioned SIFT is used, the local features are expressed as a 128-dimensional vector quantity. The extracted local features are grouped (clustered) using the k-means method. The vector at the center of each of the k obtained clusters (the centroid of each cluster) is set as the visual word of each cluster.
Image data containing the detected device available for voyeurism is converted into a histogram with visual words as a dimension. This is done by counting the frequency of visual words close to each local feature in the image. This histogram becomes the feature vector of the image. The more similar the images are, the more similar the histograms are (the feature vectors indicate similar directions). If the system is configured to determine whether an image in the learning data is similar to an image in an orientation that allows shooting, it can be configured to determine whether the device available for voyeurism allows shooting.
Another method is to extract structural features of smartphones and digital cameras, and store the positional relationship between features as configuration information. For example, a smartphone has a roughly rectangular plate shape and at least one circular camera lens on the back side (opposite the display). Depending on the placement and number of cameras, several types of configuration information are generated. Similarly, a digital camera has a roughly rectangular shape and one protruding cylindrical lens. The configuration information is stored as three-dimensional data, and the figure in the viewer's video whose presence is detected is matched and searched by changing the angle and size. Since the search is performed by the characteristics of the device, the amount of device information stored in advance is not huge. When matching with three-dimensional data, information on the orientation of the device can be obtained.
Regarding the distance of the device, it is possible to grasp the change in the shape of the blur of the image before and after the focus position by processing it with AI. It is easier to grasp the distance using a stereo camera, but since this is equipment that is not generally used for remote work, there is a problem that it is costly to prepare. Even if a monocular camera is used, distance measurement places a heavy processing load, so if it is determined that the device in the viewer's image is oriented in a way that allows it to shoot, it can also be determined that it is in a state where it can shoot.
In addition, if the processing load for determining whether or not photography is possible on the PC used by the remote worker for work is too heavy, the processing can be shared so that it is performed by a faster device with higher processing capacity, such as the company's server.

For example, as shown in Figures 22a and 22b, the smartphone needs to be upright and the camera lens part needs to face the display part to take a picture. Therefore, as shown in Figure 23a, if the smartphone is tilted downward at a depression angle α so that the camera lens faces downward, or if the camera is rotated horizontally at an azimuth angle β as shown in Figure 23b, or if the display part (A) is not within the shooting range of the smartphone camera due to a position shifted from the front of the display (not shown), it is not possible to take a voyeur picture. Figure 24a shows the relationship between the position and orientation of a PC display (used for an additional display of a notebook PC or a desktop PC) and a smartphone, and Figure 24b shows the relationship between the position and orientation of a notebook PC and a smartphone, as in Figure 24a. Figures 24a and 24b show the case where the smartphone is tilted in the depression and elevation angles, but the same applies when the smartphone is rotated in the azimuth angle direction as shown in Figure 23b.
The standard specification of the camera lens of a smartphone is often equivalent to a wide-angle lens of a general camera (angle of view of about 75 degrees). Three examples of smartphones (2403, 2404, 2405) with different orientations are shown. Figures 24a and 24b show the cases of a PC display and a notebook PC, respectively. The following explanation will be given using Figure 24a as an example. Although the shape of the display unit (A) is different, the following explanation regarding whether or not the smartphone can take a picture is the same, so the a in each symbol can be read as b. The smartphone (2403a) is located near the display unit (A) (2401a) and facing it directly, and is in a position where it can take a picture. The sector of the angle θ (angle of view of the smartphone camera) opened from the top of the smartphone toward the display unit (A) (2401a) and the dotted line range indicate the field of view of the smartphone camera. The smartphone (2403a) is in a state where it can take a picture of the display unit (A) (2401a). The smartphone (2404a) is within a range where the display unit (A) (2401a) can be photographed, but is facing downward and is out of the angle of view θ of the camera lens of the smartphone (2404a), making it impossible to photograph it. The state of the smartphone (2404a) is frequently detected because it is an action that a remote worker may frequently take when using a smartphone to search the web or check email. The display unit (A) (2401a) of the smartphone (2405a) is within the range in terms of the angle of view, but since it is far away, it can also be determined that the risk of being photographed is low.

22 to 24, an outer camera of a smartphone (opposite the display unit of the smartphone) is described as an example. Since an inner camera is generally installed on the display side of a smartphone for taking pictures of the operator himself, it can also be configured to detect the inner camera. Since a smartphone usually does not have an optical viewfinder, when taking pictures with the camera, the image captured by the camera is displayed on the smartphone's display. When the display side of the smartphone is facing the display unit (A) of the PC, it may be determined whether the smartphone is in a state where it can take pictures based on whether the image of the display unit (A) captured by the inner camera of the smartphone is displayed.
When the camera of a device with a photographing function is pointed toward the display unit (A) of a PC and is in a position where it can take a photograph, it is difficult to judge from the outside whether the power of the device is on, so it may be determined that the device is in a photographing-enabled state and that a device that can be used for voyeurism exists based on whether the camera lens is oriented in a way that it can take a photograph of the display unit. In simple image matching, the number of reference images to be matched is enormous, so it is preferable to use AI to detect candidate devices from their characteristics as described above and to determine their angle from the external image.
In addition, a camera may be arranged behind the employee facing the display, and the camera may obtain an image on the screen of the smartphone and calculate the matching rate with the information displayed on the display. In this case, the screen of the smartphone or tablet terminal faces the face of the employee, and the camera lens on the back side of the display faces the display. In this case, the image from the camera may be configured to be transmitted to a management server or the like via the terminal used by the employee, and the management server may calculate the matching rate and activate security processing if the matching rate exceeds a predetermined value. In addition, this processing may be configured to be processed within the terminal used by the employee, rather than an external device such as a management server.
Alternatively, a configuration is possible in which a smartphone is connected to a remote work PC via a USB terminal or the like, and the smartphone camera is accessed and controlled by the information protection system of the present invention. In that case, the display unit (A) can be photographed only when business information or non-business information that the remote worker is permitted to photograph is displayed. Even if it is not under the control of the information protection system, the remote work PC may be configured to import video information acquired by the smartphone camera and determine whether the smartphone is in a state in which it can photograph the display unit (A). In addition, a dedicated control application may be installed on the smartphone, and when an invisible control pattern that is difficult for humans to see is superimposed on the display of the display unit (A), the image information imported via the camera can be used to prevent the shutter from being released.

If the system is configured to detect and judge an object in the acquired viewer-side video for each still image frame, it may be detected even if a family member with a smartphone happens to pass behind the remote worker working from home. It may also be detected even if the remote worker takes out his or her smartphone to make a phone call and operates it in front of the PC. Therefore, it is preferable to configure the system to detect the presence of a device if the same device usable for voyeurism is detected consecutively over 3 to 10 or more frames of images when acquiring images at 30 frames per second.
Also, the apparatus may be configured to analyze frames over a period of 1 to 2 or 3 seconds, analyze the motion of the device usable for voyeurism in the viewing side video and the person holding it, and determine the device presence determination unit to determine the presence of a device usable for voyeurism based on the motion analysis result. Motion analysis refers to a case where the camera lens of the device usable for voyeurism for a predetermined time period faces the display surface, or a case where the thumb or the fingers of the hand not holding the camera can recognize the presence of a motion of operating the touch panel or buttons arranged on the side of the device usable for voyeurism while facing the display surface. This recognition determination is performed by using artificial intelligence or a function similar to artificial intelligence to retain recognition information through machine learning (using multiple human models to take voyeur poses and repeatedly providing feedback to the judgment result of artificial intelligence, etc.), and when the motion has a high similarity to this recognition information, it is determined that a device usable for voyeurism exists.

<Processing flow of embodiment 4>
Fig. 11 is a flowchart of the operation process of the information protection system of embodiment 4. As shown in this figure, the information protection system of embodiment 4 has a business use state detection step (b) (S1101), a viewer side video acquisition step (c) (S1102), a surreptitious shooting available device discrimination information retention step (d) (S1103), a device presence determination step (e) (S1104), a shooting permission/prohibition state determination sub-step (m) (S1105), and a detection information output step (f) (S1106).

Here, the method of operating the remote work information protection system including a computer having a display unit (A) equipped with a display that displays an image is as follows:
The business use state detection step (b) (S1101) detects and judges whether the display unit is in a business use state, which indicates that the display unit is being used for business, while the display unit is being driven.
A viewer-side image acquisition step (c) (S1102) performs a process of acquiring a viewer-side image, which is an image on the side of a viewer viewing the display, while the business use state detection step (b) (S1101) detects that the display is in a business use state;
The step (d) of storing information on a device that can be used for voyeurism (S1103) includes a process of storing information on a device that can be used for voyeurism, which is information for determining a device that can be used for voyeurism;
The device presence determination step (e) (S1104) performs a process of determining whether a device available for voyeurism is present in the viewing side image using the acquired viewing side image and the voyeurism usable device discrimination information, and the recording permission status determination sub-step (m) (S1105) in the device presence determination step (e) (S1104) performs a process of determining whether the state of the device available for voyeurism, including its position and orientation relative to the display unit (A), is in a state where recording is possible, based on the image of the device available for voyeurism in the viewing side image and the voyeurism usable device discrimination information,
In the detection information output step (f) (S1106), when the determination result in the device presence determination step (e) is that the device exists, a process of outputting detection information which is information indicating that the device exists is performed;
The business use state detection step (b) (S1107) performs processing for constantly detecting and determining whether the display unit is in a business use state, which indicates that the display unit is being used for business, while the display is being driven.
If it is determined in the device presence determination step (e) (S1104) that the device does not exist, or if it is determined in the photography status determination sub-step (m) (S1105) that the device is not in a state where photography can be performed, the next detection information output step (f) (S1106) is not performed, and processing is performed to skip to before the business use status detection step (b) (S1107).
This is an operating method for causing the information protection system to execute such a series of processes.

<Fourth embodiment: Description of hardware>
FIG. 12 is a diagram for explaining hardware configured by a main board and its peripheral devices of the information protection system, which is a computer (calculator) of the fourth embodiment. It is configured by "CPU", "chip set" consisting of a north bridge and a south bridge, "non-volatile memory", "main memory", "I/O controller", "USB, IEEE1394, HDMI (registered trademark), LAN terminal, etc.", "BIOS", "PCI slot", "real-time clock", etc. In this embodiment, a camera, a microphone, and a speaker configure the hardware of a PC by USB connection. For example, the microphone and the speaker may be input and output via a sound board connected to a PCI slot, and the camera does not have to be via USB. As long as video input and audio input and output can be performed in the same way as when connected via USB, the effect of the present invention can be obtained in the same way. It is assumed that a display is also connected using an HDMI (registered trademark) terminal. It may be connected via a connection terminal on a graphics card. Also, a known display connection terminal other than HDMI (registered trademark) may be used.

The various programs and data (information) stored in the non-volatile memory are expanded into the main memory when the system is started, and the programs are configured to perform calculations using the data in sequence by the CPU upon receiving execution commands.

In other words, the non-volatile memory contains, in addition to the OS (operating system) and device drivers, a business use status detection program, a viewing side video acquisition program, a program for retaining information on devices available for voyeurism, a program for judging the presence of devices, a detection information output program, a subprogram for judging whether or not filming is possible, a business program, and a non-business program, and stores the following data: viewing side video, information on devices available for voyeurism, detection information, business data, and non-business data. These are expanded into the main memory when the system is started up on a computer, and when a start-up command is received, the CPU sequentially performs calculations using the programs and data. The non-volatile memory stores not only the programs and data related to the remote work information protection system of the present invention, but also at least business programs necessary for remote work (e.g., web conferencing programs, programs for connecting to intranets, programs for creating materials, virus detection programs, etc.), non-business programs (e.g., browser programs for browsing Internet websites, game programs, etc.), data necessary for the operation of business programs and data created using business programs, data necessary for the operation of non-business programs and data created by operating them, etc. These are also loaded into the main memory when started on the computer, and when a start command is received, the CPU sequentially performs calculations using the programs and data.

When a device with a photographing function is used in front of the display unit (A) for purposes other than photographing, the monitoring device of the present invention outputs detection information, reducing the disruption of work on the PC. For example, this applies when viewing emails or making phone calls on a smartphone while working remotely. While looking at the information displayed on the display unit (A), the person in question can explain things to a customer over the phone.

With regard to each of the elements constituting the present invention, namely the display unit (A), business use status detection unit (B), viewing side video acquisition unit (C), voyeurism usable device discrimination information retention unit (D), device presence determination unit (E), detection information output unit (F), facial portion video judgment unit (G), facial portion video composition information acquisition unit (H), facial portion video composition information retention unit (J), danger information retention unit (K), and recording applicability status determination means (M), the other 10 elements except for the display unit (A) may be located either on the remote work PC or on the company's server (including other contracted servers). For example, all 10 elements may be located on the remote work PC, or the display unit (A), business use status detection unit (B), and viewing side image acquisition unit (C) may be located on the remote work PC, and the remaining voyeurism available device discrimination information storage unit (D), device presence determination unit (E), detection information output unit (F), face part image determination unit (G), face part image configuration information acquisition unit (H), face part image configuration information storage unit (J), danger information storage unit (K), and shooting permission/non-permission state determination means (M) may be located on the company side server, and so on. By simple calculation, the number of combinations is 2 to the power of 10 (1024 combinations). The display unit (A) and other elements may be located on the company side server. In that case, one component may be allocated to the remote work PC and the company side server, such as the viewing side image acquisition unit (C), where the camera that captures the viewing side image is connected to the remote work PC and located at the remote worker's workplace, and the part that acquires and processes the viewing side image is located on the company side server.

<5. Effects>
By using the remote work information protection system having the above configuration, it is possible to detect voyeurism by devices that can be used for voyeurism outside the PC, which has been recognized as a risk in the past but no measures have been taken. Since dangerous images that indicate a possible voyeurism state are associated with viewing information identification information that identifies the information that was displayed on the display at that time and are stored, it is effective in identifying the voyeur. In addition, since it is possible to grasp information that may be voyeurism, it is possible for the company to take countermeasures quickly. By switching the display display to notify not only employees but also the voyeur that voyeurism has occurred, it is expected to have a deterrent effect of making them hesitate to leak or spread information.

Remote work information protection system...0100
Display section (A)...0101
Business usage status detection unit (B) ... 0102
Viewing side image acquisition unit (C) ... 0103
Secret photography available device discrimination information storage unit (D) ... 0104
Device presence determination section (E)...0105
Detection information output unit (F)...0106

Claims (9)

A display unit (A) having a display for displaying an image;
a business use state detection unit (B) that constantly detects whether the display unit is in a business use state indicating that the display unit is being used for business while the display is being driven;
a viewing side image acquisition unit (C) that acquires a viewing side image, which is an image of a side that views the display, while the business use state detection unit (B) detects that the display is in a business use state;
a storage unit (D) for storing voyeurism usable device discrimination information, which is information for discriminating voyeurism usable device;
a device presence determination unit (E) for determining whether a device available for voyeurism is present in the viewing side video using the obtained viewing side video and the voyeurism available device discrimination information;
a detection information output unit (F) that outputs detection information indicating the presence of the device when the determination result of the device presence determination unit (E) indicates the presence of the device;
A remote work information protection system having:
a danger information holding unit (K) for holding danger information, which is information associating a danger image, which is all or a part of a viewer's image, with viewing information identification information for identifying information shown on the display, when the detection information output unit (F) outputs the detection information;
a security processing unit that performs security processing such as keeping the display unit (A) in a display state or not displaying the display unit (A);
The security processing unit includes:
When the display on the display unit (A) identified by the viewing information identification information associated with the stored risk information is general business information, the display on the display unit (A) is not turned to a non-display state, and when the display on the display unit (A) is business information including personal information, security processing is performed to turn the display on the display unit (A) to a non-display state.
Remote work information protection system . a face part image determination unit (G) for determining whether a face part image, which is a partial image including a face, is present in the acquired viewer side image;
a face part image configuration information acquisition unit (H) for acquiring face part image configuration information which is information constituting the face part image when the face part image determination unit (G) determines that a face part image is present;
A face part image composition information storage unit (J) for storing the acquired face part image composition information;
The remote work information protection system of claim 1 , further comprising: A remote work information protection system as described in either claim 1 or claim 2, wherein the equipment presence determination unit (E) has a photographing availability state determination means (M) which determines whether the state of the device capable of being used for voyeurism relative to the display unit (A), including its position and orientation, is in a state in which photographing is possible, based on the image of the device capable of being used for voyeurism in the viewing side image and the voyeurism availability device discrimination information. A method for operating a remote work information protection system that is a computer having a display unit (A) equipped with a display that displays an image, comprising:
a business use state detection step (b) for constantly detecting whether the display unit is in a business use state indicating that the display unit is being used for business while the display is being driven;
a viewing side image acquiring step (c) of acquiring a viewing side image which is an image of a side viewing the display while the business use state detecting step (b) detects that the display is in a business use state;
a secret photography available device discrimination information holding step (d) for holding secret photography available device discrimination information which is information for discriminating a secret photography available device;
a device presence determination step (e) of determining whether a device available for voyeurism is present in the viewing side video using the obtained viewing side video and the voyeurism available device discrimination information;
a detection information output step (f) of outputting detection information indicating the presence of the device when the determination result in the device presence determination step (e) indicates the presence of the device;
A method for operating a remote work information protection system, the remote work information protection system being a computer having
a danger information holding step (k) of holding danger information which is information associating a danger image which is all or a part of a viewer's image with viewing information identification information which identifies information shown on a display when the detection information output step (f) outputs the detection information;
a security processing step of performing security processing such as continuing the display state of the display unit (A) or making the display unit (A) non-display;
This security processing step comprises:
When the display on the display unit (A) identified by the viewing information identification information associated with the stored risk information is general business information, the display on the display unit (A) is not turned to a non-display state, and when the display on the display unit (A) is business information including personal information, security processing is performed to turn the display on the display unit (A) to a non-display state.
A method for operating a remote work information protection system that is a computer. a face partial image determination step (g) of determining whether a face partial image, which is a partial image including a face, is present in the acquired viewer side image;
a face part image composition information acquisition step (h) of acquiring face part image composition information which is information constituting the face part image when the face part image determination step (g) determines that a face part image is present;
a face part image composition information holding step (j) of holding the acquired face part image composition information;
5. A method for operating a remote work information protection system, the computer being as recited in claim 4, further comprising: A method for operating a remote work information protection system which is a computer as described in either claim 4 or claim 5, wherein the device presence determination step (e) includes a photographing availability state determination substep (m) for determining whether the state of the device capable of being used for voyeurism relative to the display unit (A), including its position and orientation, is in a state in which photographing is possible, based on the image of the device capable of being used for voyeurism in the viewing side image and the voyeurism available device discrimination information. A program to be executed by a remote work information protection system, which is a computer, having a display unit (A) equipped with a display that displays an image,
a business use state detection step (b) for constantly detecting whether the display unit is in a business use state indicating that the display unit is being used for business while the display is being driven;
a viewing side image acquiring step (c) of acquiring a viewing side image which is an image of a side viewing the display while the business use state detecting step (b) detects that the display is in a business use state;
a secret photography available device discrimination information holding step (d) for holding secret photography available device discrimination information which is information for discriminating a secret photography available device;
a device presence determination step (e) of determining whether a device available for voyeurism is present in the viewing side video using the obtained viewing side video and the voyeurism available device discrimination information;
a detection information output step (f) of outputting detection information indicating the presence of the device when the determination result in the device presence determination step (e) indicates the presence of the device;
A program for executing a remote work information protection system that is a computer , comprising:
a danger information holding step (k) of holding danger information which is information associating a danger image which is all or a part of a viewer-side image with viewing information identification information which identifies information shown on a display when the detection information output step (f) outputs the detection information;
a security processing step of performing security processing such as continuing the display state of the display unit (A) or making the display unit (A) non-display;
This security processing step comprises:
When the display on the display unit (A) identified by the viewing information identification information associated with the stored risk information is general business information, the display on the display unit (A) is not turned to a non-display state, and when the display on the display unit (A) is business information including personal information, security processing is performed to turn the display on the display unit (A) to a non-display state.
A program to be executed by a remote work information protection system which is a computer . a face partial image determination step (g) of determining whether a face partial image, which is a partial image including a face, is present in the acquired viewer side image;
a face part image composition information acquisition step (h) of acquiring face part image composition information which is information constituting the face part image when the face part image determination step (g) determines that a face part image is present;
a face part image composition information holding step (j) of holding the acquired face part image composition information;
The program according to claim 7, further comprising : The device presence determination step (e) includes a photographing permission state determination sub-step (m) for determining whether or not the state of the device available for voyeurism relative to the display unit (A), including the position and orientation of the device available for voyeurism relative to the display unit (A), is in a state in which photographing is possible, based on the image of the device available for voyeurism in the viewing side image and the information on the device available for voyeurism discrimination.
The program according to claim 7 or 8, which is executed .

Images