JP2007003745A - Image display device and image display system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To easily perform user authentication when displaying an image transmitted from a PC or the like on an image display device such as an eyeglass type or a head-mounted type.
When a sensor 111 detects that a user Y wears an image display device 101 on his / her head, a CCD image sensor 105 captures an image of the right eye of the user Y and a PC connected to the image display device 101. Send to. In the PC, the iris authentication of the user Y is performed with the image of the right eye of the user Y. Then, image data to be displayed on the LCD 104 is transmitted to the image display device 101 on the condition that the iris authentication has been performed.
[Selection] Figure 2

Description

  The present invention relates to an image display device and an image display system that are mounted on a user's head and display an image in front of the user's eyes.

  In recent years, with the spread of mobile computing, there are increasing opportunities to use notebook PCs, sub-notebook PCs, etc. in trains, stations, coffee shops, hotels, and other public places. The widespread use of public wireless LAN services in stations, coffee shops, etc. has spurred the spread of mobile computing.

  As described above, when using a notebook PC, a sub-notebook PC, etc. in a train, a coffee shop, etc., there are many cases where there are others near the user, and others can easily look into the display of the PC. Therefore, there is a risk that confidential information such as companies (customer information that should be protected by the Personal Information Protection Law) may be leaked, or personal privacy may be violated.

  Therefore, in order to solve such a problem, it is conceivable to use an eyeglass-type or head-mounted image display device (refer to Patent Documents 1 and 2 and the like. The image display device is a device for experiencing virtual reality). This eyeglass-type and head-mounted image display device is either worn on the head by the user wearing it on the ear like glasses (glass-type) or directly on the head (head-mounted) The image is displayed on a display provided in a housing disposed in front of the user's eyes.

  If such an eyeglass-type or head-mounted image display device is used, it is difficult for others to look into the display even if an image is displayed in front of the user's eyes. Can be prevented.

  However, in the mobile computing environment as described above, while the user is out of his / her seat, other people use a PC and glasses-type or head-mounted image display device to collect confidential information and privacy information. There is a risk of seeing. Further, there is a risk that secret information and privacy information may be stolen because the PC, glasses-type, and head-mounted image display devices are stolen.

Conventionally, in order to prevent others from stealing information from a PC, it has been generally possible to view data in the PC on condition that the user has been authenticated by inputting a password or fingerprint authentication. It was the target.
Japanese Patent Laid-Open No. 11-265248 JP 2000-278713 A

  However, as described above, it is possible to use a PC, glasses-type, or head-mounted image display device on condition that a password is input with an input device such as a keyboard or a fingerprint is read with a fingerprint reader. , Because it requires extra operations such as a password input operation (you have to tap the keyboard) and a fingerprint reading operation (you need to touch your fingertip to the fingerprint reader), it is cumbersome for the user. .

  In addition, public wireless LAN services such as in station premises and coffee shops are systems that share hotspots (registered trademark of NTT Communications), so even if you enter passwords or fingerprint authentication, you can use PCs or glasses Even if a type and head-mounted image display device can be used, information is intercepted by others while the PC and the eyeglass-type and head-mounted image display device exchange information. There is also a problem that there is a fear.

Accordingly, an object of the present invention is to make it possible to easily perform user authentication when displaying an image transmitted from a PC or the like on an image display device such as an eyeglass type or a head-mounted type.
Another object of the present invention is to prevent others from intercepting information exchange after user authentication.

  (1) The present invention provides a support body mounted on a user's head, a main body supported by the support body and disposed in front of the user's eyes, and provided in the main body to display an image in front of the user's eyes. A display and an image sensor provided on the support or the main body and disposed at a position where the image of at least a part of the user's face can be taken from the support or the main body in the mounted state; The image display apparatus is provided.

  (2) Further, the present invention provides the image display device according to (1), an authentication unit that performs biometric authentication of a user using an image captured by the imaging device, and the display on the condition that the biometric authentication is performed. An image display system provided with permission means for permitting display of an image at.

  (3) Another aspect of the present invention is the image display system according to (2), wherein the permission unit transmits image data to be displayed on the display to the image display device on condition that the biometric authentication has been performed. Transmission means, further comprising reception means for receiving the image data transmitted by the image display device.

  (4) Another aspect of the present invention is the image display system according to (3), wherein an encryption unit that encrypts the image data before the transmission and a decryption unit that decrypts the received image data. And further.

  (5) Another aspect of the present invention is the image display system according to any one of (2) to (4), wherein the sensor is provided on the support body or the main body to detect the mounting. And control means for executing photographing by the image sensor and authentication by the authentication means when the attachment is detected.

  (6) Another aspect of the present invention is the image display system according to any one of (2) to (4), wherein the sensor detects that the support is detached from the user's head. Further includes prohibiting means for prohibiting display of an image on the display until the biometric authentication is performed again.

  (1) According to the invention of (2), since the image pickup device that captures an image of at least a part of the user's face from the main body side or the support body side where the display is provided is provided, at least one of the user's face is provided. The biometric authentication of the user using the captured image (the biometric authentication for capturing at least a part of the user's face includes iris authentication, retina authentication, facial copying (morphing), skeleton authentication, etc.) It is possible to permit the image display on the display on the condition that the biometric authentication of the user has been performed, and it is possible to protect confidential information and privacy information.

  In addition, since the imaging element is disposed at a position where an image of at least a part of the user's face can be taken from the support side or the main body side when the image display device is mounted on the head, the image display device is placed on the head. It is possible to perform biometric authentication by taking a necessary facial image without performing any extra operation other than wearing. Therefore, unnecessary operations such as password input operations and fingerprint reading operations are unnecessary, and user authentication can be performed easily.

  According to the invention of (3), when biometric authentication is not performed and image display is not permitted, image data is not transmitted to the image display device. There is no fear that information will be intercepted by others while the image display device exchanges information.

  According to the invention of (4), since the image data is encrypted and transmitted, information is intercepted by another person while biometric authentication is performed and the image data transmission side and the image display device exchange information. There is no fear of it.

  According to the invention of (5), if the sensor detects that the image display device is mounted on the user's head, it is possible to automatically perform photographing and biometric authentication. It is possible to easily perform user authentication without the need for such extra operations.

  According to the invention of (6), when the user removes the image display device from the head, the image is not displayed on the display unless biometric authentication is performed again. Therefore, even when the user temporarily stands. It is possible to prevent others from seeing confidential information and privacy information.

The best mode for carrying out the present invention will be described below with reference to the drawings.
FIG. 1 is an external perspective view of an image display system 1 of the present embodiment. The image display system 1 includes an image display device 101 and a PC (Personal Computer) 201.

  First, the configuration of the image display apparatus 101 will be described. As illustrated in FIG. 1, the image display apparatus 101 includes a housing 102 that is a main body and a support body 103 that supports the housing 102. The support body 103 of this example is attached to the head of the user Y (see FIG. 2), and the housing 102 can be disposed in front of the user Y in this state. The illustrated example is a head-mounted type that is directly attached to the head of the user Y. However, the support 103 is attached to the user Y's ear by putting the support 103 in a shape like a vine of eyeglasses, etc. You may comprise in the spectacles type with which a head is mounted | worn.

  FIG. 2 is a side view showing the arrangement of each device in the housing 102 of the image display device 101, and FIG. 3 is a rear view of the same (viewed inside the housing 102 from the user Y side). As shown in FIGS. 2 and 3, a liquid crystal display device (LCD) 104 serving as a display is provided in the housing 102. The LCD 104 can display an image in front of the user's eyes while the image display device 101 is mounted on the user Y's head. Since the LCD 104 is disposed at the innermost part of the housing 102 and is blocked by the housing 102, the display image on the LCD 104 is difficult to be seen by persons other than the user Y.

  A CCD (Charge Coupled Device) image sensor 105 serving as an imaging device is provided on the top plate side in the housing 102. A half mirror 106 is provided below the CCD image sensor 105. The half mirror 106 is normally housed at a position immediately below the CCD image sensor 105, but the rotation provided at the edge portion farther from the user Y by driving the motor 108 through a predetermined coupling mechanism 107. By rotating around the axis, the edge side closer to the user Y can be lowered so as to block the front of the user Y, and can be stored in the original position again. When the half mirror 106 is lowered, the half mirror 106 captures at least a part of the face of the user Y, forms an image on the CCD image sensor 105 via a predetermined imaging optical system 141, and at least one of the face of the user Y. (In this example, only the right eye image of user Y needs to be taken). That is, the CCD image sensor 105, the half mirror 106, and the connection are positioned so that the required portion of the face of the user Y can be photographed by the CCD image sensor 105 only by the user Y wearing the image display device 101 correctly on the head. The image optical system 141 is arranged. The lamp 109 is provided at a predetermined position in the housing 102 and illuminates at least a part of the face of the user Y so that the CCD image sensor 105 can photograph the face of the user Y in the housing 102.

  In addition, the support body 103 or the housing 102 (in this example, the inner surface of the support body 103) is provided with a sensor 111 that detects that the image display device 101 is mounted on the head of the user Y. (See FIG. 2). As the sensor 111, specifically, a pressure sensor, an optical sensor, or the like can be used.

  FIG. 4 is a block diagram for explaining the electrical connection of the image display apparatus 101. As shown in FIG. 4, the image display apparatus 101 is a CPU 121 that centrally controls the entire image display apparatus 101, a ROM 122 that stores various control programs executed by the CPU 121 and fixed data, and a work area for the CPU 121. A RAM 123 is connected by a bus 124 to constitute a microcomputer.

  The bus 124 further includes the LCD 104, the CCD image sensor 105, the motor 108, the lamp 109, and the sensor 111, which are a liquid crystal driving circuit 131, a CCD driving circuit 132, a motor driving circuit 133, a lighting circuit 134, and a sensor driving circuit 135, respectively. Connected through. Further, an image processing circuit 136, an image memory 137, and an interface (I / F) 138 are also connected to the bus 124. The image processing circuit 136 performs necessary image processing such as γ correction on the image data to be displayed on the LCD 104. The image memory 137 serves as a buffer for image data for displaying a moving image or the like on the LCD 104.

  The I / F 138 is an interface for the image display apparatus 101 to communicate with the PC 201, and various standards such as D-Sub15pin, USB, and IEEE1394 can be used. Note that the I / F 138 may be connected to the image display apparatus 101 and the PC 201 by wire or may be connected by wireless communication.

  The power source of the image display apparatus 101 may be supplied from the PC 201 when the USB is used for the interface between the image display apparatus 101 and the PC 201, may be supplied from a commercial AC power source, You may receive supply from the battery incorporated in the apparatus 101. FIG. The power supply circuit 139 generates a power supply voltage necessary for the image display apparatus 101 from these power supplies and supplies the power supply voltage to each unit of the image display apparatus 101.

  Next, the configuration of the PC 201 will be described. FIG. 5 is a block diagram for explaining the electrical connection of the PC 201. As shown in FIG. 5, the PC 201 is connected by a bus 204 to a CPU 202 that performs various calculations and centrally controls the entire PC 201 and a memory 203 including various ROMs and RAMs.

  The bus 204 further includes a magnetic storage device (HDD: Hard Disk Drive) 205, a display device 206 such as an LCD, an input device 207 such as a keyboard and a mouse, and a memory for reading data recorded in the storage medium 208. A medium reading device 209, a communication control device 211 that communicates with a network 210 such as the Internet, and an interface (I / F) 212 that allows the PC 201 to communicate with the image display device 101 are connected. Specifically, the storage medium reading device 209 is a DVD drive device, a CD drive device, or a flexible disk drive depending on which medium the storage medium 208 is (DVD, CD, flexible disk, etc.). A device or the like is used. The I / F 212 can use various standards such as D-Sub15pin, USB, and IEEE1394.

  The PC 201 reads a program (driver software) for driving the image display device 101 from the storage medium 208 by the storage medium reading device 209, downloads it from the network 210 such as the Internet, or stores it in the ROM 122 of the image display device 101. By reading the driver software stored in advance and installing it in the HDD 205, the operation described below becomes possible. The driver software operates on a predetermined OS (Operating System).

Next, the operation of the image display system 1 will be described.
6, 8, and 9 are flowcharts of processing executed by the CPU 121 of the image display apparatus 101 in accordance with a program stored in the ROM 122, and FIGS. 7 and 10 show that the CPU 202 of the PC 201 is installed in the HDD 205. It is a flowchart of the process performed according to driver software.

  First, as shown in FIG. 6, when the CPU 121 of the image display apparatus 101 detects the user Y by the sensor 111 continuously for a predetermined time (for example, 2 seconds) (Y of S1), the user Y displays the image. If it is determined that the device 101 is mounted on the head and no image is currently being displayed on the LCD 104 of the image display device 101 based on the image data sent from the PC 201 (N in S2), the motor 108 is driven. Then, the half mirror 106 is lowered in front of the user Y (S3), the lamp 109 is turned on (S4), and at least a part of the face of the user Y (only the right eye part is photographed in this example) by the CCD image sensor 105. An image is taken (S5). The image data of the photographed image is transmitted to the PC 201 (S6), the lamp 109 is turned off (S7), the motor 108 is driven, and the half mirror 106 is returned to the original position (S8).

  Next, as shown in FIG. 7, when the CPU 202 of the PC 201 receives the image data transmitted by the image display apparatus 101 in S5 (Y in S11), the CPU 202 performs biometric authentication of the user Y using the received image data ( S12). The biometric authentication in this case includes well-known iris authentication, retina authentication, facial copying (morphing), skeleton authentication, and the like. Here, an outline of a specific example in the case of iris authentication will be described later. S12 implements the authentication means of the present invention. Further, the control means is realized by S4 and S12.

  When the user Y is authenticated by biometric authentication in S12 (Y in S12), browsing of the image on the LCD 104 of the display device 101 is permitted (S13), and transmission of the desired image of the user Y is started. As a result, the user Y can view various images on the LCD 104 of the display device 101 by operating various applications of the PC 201 such as a Web browser. This S13 realizes a permission unit and a transmission unit. In this case, the image data is transmitted from the PC 201 to the display device 101 after the target image data is encrypted by a predetermined encryption method. This realizes encryption means.

  If the user Y is not authenticated by biometric authentication in S12 (N in S12), image browsing on the LCD 104 of the display device 101 is prohibited (S14). In this case, no matter how the user Y operates the PC 201, the image cannot be viewed on the LCD 104 of the display device 101.

  Then, as shown in FIG. 8, when transmission of image data from the PC 201 is started (Y in S21), the CPU 121 of the image display apparatus 101 receives this image data and receives an image based on this image data. Is displayed on the LCD 104 (S22). The receiving means is realized by S22. The received image data is once buffered in the image memory 137, decoded by a predetermined decoding program (decoding means is realized by this decoding), and buffered again in the image memory 137. (Especially for moving image data with a large amount of data). The image data is displayed on the LCD 104 after necessary image processing such as γ correction is performed by the image processing circuit 136.

  With the processing in FIG. 8, the user Y can view a desired image that can be viewed on the PC 201. During the image browsing, as shown in FIG. 9, when the CPU 121 of the image display apparatus 101 cannot detect the user Y by the sensor 111 continuously for a predetermined time (for example, 2 seconds) (Y in S31), If it is determined that the user Y has detached the image display device 101 from the head and an image based on the image data sent from the PC 201 is currently displayed on the LCD 104 of the image display device 101 (Y in S32). The prohibit command is transmitted to the PC 201 (S33).

  As shown in FIG. 10, when the CPU 202 of the PC 201 receives a prohibit command from the image display apparatus 101 (Y in S41), the CPU 202 stops transmitting image data to the image display apparatus 101 (S42). The prohibiting means is realized by S33 and S42.

  When the user Y wears the image display device 101 on the head again after the transmission of the image data is stopped, the processing after S1 is executed again, biometric authentication is performed again (S12), and the image display device 101 The image display on the LCD 104 is resumed.

  Note that when a display device other than the image display device 101 is connected to the PC 201, the driver software installed in the HDD 205 always prohibits display of an image on the display device (the PC 201 in the illustrated example of FIG. Although a display device is provided, images are not always displayed on this display device).

Next, iris authentication will be described as an example of biometric authentication (S12).
Iris recognition uses the characteristics of the iris of the human eye to convert the iris pattern image captured by the camera into data and identify the individual by comparing it with the pre-registered iris data. Personal authentication technology. This is realized by sequentially executing the following processes (1) to (4).

(1) Localization processing An iris portion is cut out from an eye image obtained by photographing with the CCD image sensor 105. This can be done by detecting the outer boundary of the iris (white eye and iris) and the inner boundary of the iris (iris and pupil) using a difference in image brightness.

(2) Feature extraction An analysis band is determined by dividing the iris from the outer boundary to the inner boundary into eight layers. At this time, in order to prevent erroneous iris data due to part of the iris being hidden by the eyelids or eyelashes, the range to be used is limited using feature extraction.

  Next, polar coordinates are considered with the iris center as the origin. A change in iris density in each analysis zone is extracted by a predetermined filter. Then, 256-byte iris data can be generated by digitally encoding the change in shading.

(3) Matching The encoded data is used as input iris data, and the Hamming distance value is calculated by matching with the iris data of the user Y registered in advance.

(4) Identity determination When the normalized distance value (Hamming distance value) is smaller than a predetermined value (threshold value), it is determined as the user Y (Y in S12), and when it is larger (N in S12) Is determined. The threshold value for this determination can be determined by collecting a large amount of iris data and statistically processing it.

  The above is the outline of the iris authentication, but the iris authentication technique is a well-known technique, and detailed description thereof will be omitted. Further, in the present invention, as biometric authentication, retina authentication, facial copying (morphing), skeleton authentication, and the like can be used in addition to iris authentication. However, these biometric authentication techniques are also well-known techniques and will not be described here. Note that biometric authentication techniques that can be applied to the present invention include all techniques that can perform authentication using at least some images of the face of the user Y.

  According to the image display system 1 described above, the CCD image sensor 105 that captures an image of at least a part of the face of the user Y from the housing 102 side where the LCD 104 is provided (or from the support body 103 side). Therefore, at least a part of the face of the user Y can be photographed, and the user's biometric authentication (iris authentication, retina authentication, facial copying (morphing), skeleton authentication, etc.) can be performed using the captured image (S12). ) Since the image display on the LCD 104 can be permitted on the condition that the biometric authentication of the user has been made (Y in S12), confidential information and privacy information can be protected.

  The CCD image sensor 105 is arranged at a position where at least a part of the face of the user Y can be taken from the support 103 side or the housing 102 side when the image display device 101 is mounted on the head. It is possible to perform biometric authentication by photographing a necessary facial image without performing any extra operation other than mounting the image display device 101 on the screen. In this embodiment, if the sensor 111 detects that the image display device 101 is mounted on the head of the user Y (Y in S1), the photographing (S5) and the biometric authentication (S12) are automatically performed. ing.

  As a result, the user Y only has to wear the image display device 101 on the head, and after that, the user Y only has to wait for the biometric authentication to be automatically displayed on the LCD 104.

  In contrast, conventional password input and fingerprint authentication require complicated operations such as entering a password with a keyboard or having the fingerprint reader read your fingerprint. The advantages of are obvious.

  Further, as a means for prohibiting browsing of images without biometric authentication (S14), in this embodiment, transmission of image data from the PC 201 is prohibited, but no restriction is provided from the PC 201. Alternatively, the image data may be transmitted to the image display apparatus 101 so that the display on the LCD 104 is rejected.

  However, if the transmission of the image data from the PC 201 itself is prohibited as in the present embodiment, the image data transmission side PC 201 and the image display device 101 exchange information even though biometric authentication is not performed. There is no risk of information being intercepted by others while you are doing it.

  Further, since the image data transmitted on the PC 201 side is encrypted and decrypted on the image display apparatus 101 side, information is transferred to another person while biometric authentication is performed and the PC 201 side and the image display apparatus 101 exchange information. There is no fear of being intercepted.

  Further, when the user Y detaches the image display device 101 from the head during image display (Y in S31, Y in S32), the image is displayed on the LCD 104 unless biometric authentication is performed again (S1, S2, S12). Since this is not done (S33), it is possible to prevent others from seeing secret information, privacy information, etc. even when the user Y stands temporarily.

  In this embodiment, the biometric authentication is executed by the CPU 202 of the PC 201. However, the present invention is not limited to this, and may be executed by the CPU 121 of the image display apparatus 101. In this case, the image display system 1 can be implemented with the image display apparatus 101 alone.

  In this embodiment, a series of processing is executed by the exchange between the CPU 202 of the PC 201 and the CPU 121 of the image display apparatus 101. However, the present invention is not limited to this, and the image display apparatus 101 includes A microcomputer composed of the CPU 121, ROM 122, and RAM 123 is not provided, and the liquid crystal driving circuit 131, the CCD driving circuit 132, the lighting circuit 134, the motor driving circuit 133, and the sensor circuit 135 can be directly connected to the PC 201. It may be executed by the CPU 202. With such a configuration, the manufacturing cost of the image display apparatus 101 can be reduced, and the weight of the image display apparatus 101 can be reduced.

1 is an external perspective view of an image display system according to an embodiment of the present invention. It is a side view showing the arrangement of each device in the housing of the image display device constituting the image display system, It is the same rear view. It is a block diagram explaining the electrical connection of an image display apparatus. It is a block diagram explaining the electrical connection of PC which comprises an image display system. It is a flowchart of the process which CPU of an image display apparatus performs. It is a flowchart of the process which CPU of PC performs. It is a flowchart of the process which CPU of an image display apparatus performs. It is a flowchart of the process which CPU of an image display apparatus performs. It is a flowchart of the process which CPU of PC performs.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Image display system 101 Image display apparatus 102 Main body 103 Support body 104 Display 105 Image sensor

Claims (6)

A support worn on the user's head;
A main body supported by the support and disposed in front of the user's eyes;
A display provided on the main body and displaying an image in front of the user's eyes;
An image sensor that is provided at the support body or the main body and is mounted at a position where the image of at least a part of the user's face can be taken from the support body side or the main body side in the mounted state;
An image display device comprising: An image display device according to claim 1;
Authentication means for performing biometric authentication of a user by an image captured by the image sensor;
Permission means for permitting display of an image on the display on the condition that the biometric authentication is performed;
An image display system. The permission means is a transmission means for transmitting image data to be displayed on the display to the image display device on condition that the biometric authentication has been performed,
The image display system according to claim 2, further comprising receiving means for receiving the image data transmitted by the image display device. An encryption means for encrypting the image data before the transmission;
Decoding means for decoding the received image data;
The image display system according to claim 3, further comprising: A sensor that is provided on the support or the main body to detect the mounting;
Control means for performing photographing by the image sensor and authentication by the authentication means when the attachment is detected;
The image display system according to any one of claims 2 to 4, further comprising:   6. The apparatus according to claim 5, further comprising prohibiting means for prohibiting display of an image on the display until the biometric authentication is performed again when the sensor detects that the support is detached from the user's head. The image display system described.