JP2011053925A - Image display method, program and image display apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image display method, a program and an image display apparatus which includes a display function of a digital image or the like and an e-mail receiving function and can recognize a viewer and display e-mails corresponding to the viewer. <P>SOLUTION: The image display apparatus 100 is composed of: a storage part 300 for storing image data, e-mails and face authentication information; a display 214; an image display processing part 230 for reading out image data, e-mails and face authentication information from the storage part 300 and displaying the image data on a display 214; a face authentication part 240 for authenticating a face of a person photographed by an imaging part 212 based on the face authentication information; an e-mail processing part 250 for displaying e-mails specifying the person authenticated by the face authentication part 240 as a destination on the display 214; and a control part 220 for controlling respective parts. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an image display method, a program, and an image display apparatus.

  In recent years, for the purpose of displaying a digital image taken by a digital camera, an image display device such as a so-called “digital photo frame” obtained by digitizing a conventional photo frame has been commercialized. Among such image display devices, a high-performance processor is mounted, and there are products having not only a digital image display function but also a network connection function. Using this network connection function, we have also commercialized an image display device that has been widely used in devices such as personal computers and mobile phones, that is, a function that receives e-mail and displays it on the display unit. Has been. Furthermore, even simple image display devices that do not have an input device such as a keyboard are provided with an e-mail receiving function and products that display received e-mail and image data have increased.

  On the other hand, in a personal computer or the like, when a specific user is using the user, the user is identified and mail or data is displayed while the user is in use. There has been proposed an image display system which is intended to detect and prevent display when a user looks into a screen (see, for example, Patent Document 1).

JP 2000-322358 A

  However, in a simple image display device having only an image display function such as a digital photo frame and a function for receiving and displaying an e-mail, the e-mail acquired by accessing the mail server periodically using a preset account is obtained. It just displays email and images as they are. Therefore, a device having a function of individually identifying a user and displaying a message corresponding to the user has not been proposed yet, and development is desired.

  The present invention has been made in view of such problems, and has a display function for digital images and an e-mail reception function, and recognizes a user while displaying a digital image, and an e-mail corresponding to the user. An object of the present invention is to provide an image display method, a program, and an image display apparatus capable of displaying the image.

  In order to solve the above-described problems, an image display method according to the present invention includes a storage unit that stores image data, email, and registered user authentication information, a display unit that displays image data or email, and a display. A detection unit that detects information on a person around the unit and the image data is read from the storage unit and displayed on the display unit, or when the human information is detected by the detection unit, The identification information for identifying the person is detected, and it is determined whether or not the detected identification information matches the authentication information of the registered user read from the storage unit. A control unit that displays mail on the display unit and displays image data on the display unit if they do not match.

  In such an image display device, when a plurality of registered users are detected, the control unit displays an e-mail addressed to all of the plurality of registered users on the display unit, and at least of the detected registered users It is preferable that the electronic mail is not displayed when one person is not included in the destination.

  In such an image display device, the detection unit is an imaging device that captures authentication image data of a person around the display unit, and the control unit is based on the authentication image data captured by the imaging device. It is preferable that a face is detected as the identification information, and it is determined whether or not the face matches the authentication information read from the storage unit.

  When the image display method, the program, and the image display device according to the present invention are configured as described above, the image display device is provided with a display function for digital images and an e-mail receiving function. Can be displayed and an e-mail corresponding to the viewer can be displayed, and not only an image display function but also an information terminal function can be improved.

It is a block diagram showing the hardware constitutions of the image display apparatus of this embodiment. It is a block diagram showing the function structure of the program performed with a microprocessor. It is a figure which shows the data structure of the system information list memorize | stored in the memory | storage part. It is a flowchart which shows the procedure of the image display method performed with an image display apparatus. It is a flowchart which shows the procedure of the initialization process. It is a flowchart which shows the procedure of an image display process. It is a flowchart which shows the procedure of an imaging | photography process. It is a flowchart which shows the procedure of a face authentication process. It is a flowchart which shows the procedure of an electronic mail display process.

  Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings. First, the hardware configuration of the image display apparatus 100 according to the present embodiment will be described with reference to FIG. As shown in FIG. 1, the image display apparatus 100 includes a microprocessor 201, an operation control unit (operation panel) 202 having an operation switch 202a, a ROM (Read Only Memory) 203, a RAM (Random Access Memory) 204, a network. A control unit 205, a FLASH memory 209, an external memory terminal 210, an imaging unit (detection unit) 212, a video controller 213, a display 214, and the like are connected to each other or via a data bus 215. ing.

  The microprocessor 201 controls the entire operation of the image display apparatus 100, and is configured to operate based on a program code stored in the ROM 203 or the like and data referred to by this program code. The ROM 203 is a storage unit that stores program codes and data necessary for the microprocessor 201 to perform a predetermined operation. The RAM 204 functions as a volatile memory for holding temporary storage of data necessary for the operation of the microprocessor 201 and the video controller 213.

  The operation control unit 202 is used by a person (user) who wants to visually recognize image data and e-mail to the microprocessor 201 for instructions necessary for various operations. The operation control unit 202 is provided with an operation switch 202a. When the viewer operates the operation switch 202a, a viewer's operation input is realized. Further, the display 214 may be a touch panel, and the touch panel may be connected to the operation control unit 202.

  The FLASH memory 209 is an internal storage medium, and is a non-volatile memory that stores images for display on the display 214, registered user setting data, e-mail, and the like. The external memory terminal 210 is connected to an external storage medium 211 such as a memory card, and image data photographed by the photographing device 212 via the external memory terminal 210 and its attribute information, and image data received via the network 206. And e-mail are stored in the external storage medium 211. The external storage medium 211 is a non-volatile memory that can be removed and taken outside. Once image data or the like is stored, the external storage medium 211 is stored even when the power of the image display device 100 main body is shut off. Image data and the like can be held. Similarly to the FLASH memory 209, the image data and electronic mail stored in the external storage medium 211 are used for display on the display 214 via the external memory terminal 210.

  The network control unit 205 is connected to the image server 207 and the e-mail server 208 via a network 206 (for example, the Internet). In the image display apparatus 100 of the present embodiment, image data is acquired from the image server 207 via the network control unit 205 and the network 206, and stored in the FLASH memory 209, the external storage medium 211, and the like. The acquired image data is also used for display on the display 214 under the control of the microprocessor 201. The e-mail server 208 is a server that accumulates e-mails delivered via the network 206. The image display apparatus 100 periodically accesses the e-mail server 208 via the network 206, receives e-mails of registered users, and stores them in the FLASH memory 209, the external storage medium 211, and the like.

  The imaging unit 212 includes an optical lens and an image sensor (not shown), and forms an object image to be photographed on the image sensor with the optical lens under the control of the microprocessor 201. The imaging unit 212 captures an image around the image display device 100 as a subject. The viewer is authenticated based on the captured image (hereinafter referred to as “authentication image data”). In this embodiment, the image capturing by the imaging unit 212 is controlled to be performed at predetermined capturing intervals. However, the image capturing may be controlled to be performed when a motion of a person is detected by a sensor or the like. . As described above, in the present embodiment, the imaging unit 212 is used as the detection unit, and a face is detected from the captured authentication image data to determine whether the user is a registered user, as described later. It is not limited to. For example, wireless IC tag information carried by the viewer may be detected, and based on this information, it may be determined whether the user is a registered user, or fingerprint, voice, or other personal identification information may be detected and authenticated. You may comprise.

  The video controller 213 generates an image to be displayed on the display 214 as necessary under the control of the microprocessor 201. The display 214 is a display device that displays image data and e-mail, and is attached to the front side of the image display device 100. The display 214 can be a general liquid crystal monitor or the like. Note that the video displayed on the display 214 is output from the video controller 213 via the data bus 215.

  Next, the functional configuration of each unit that executes image display processing in the microprocessor 201 will be described with reference to FIG. As shown in FIG. 2, the image display processing is taken by the control unit 220 that controls each unit, the image display processing unit 230 that displays the image data stored in the storage unit 300 on the display 214, and the imaging unit 212. It is executed by the face authentication unit 240 that analyzes the authentication image and authenticates the viewer, and the e-mail processing unit 250 that determines whether or not the e-mail is visible and displays or hides the e-mail. .

  The storage unit 300 includes the above-described ROM 203, RAM 204, FLASH memory 209, external storage medium 211, and the like, and stores display image data, a system information list 301 (see FIG. 3), e-mail, and other data. In addition, the storage unit 300 stores a user authentication information table in which a facial ID of a user (hereinafter referred to as a “registered user”) registered in advance to permit viewing of an e-mail is associated with a user ID. Is remembered. Also, a detection information table for setting image data for authentication photographed by the imaging unit 212, a face feature amount detected by face authentication processing based on the image data, and a user ID of the corresponding registered user is also stored. Stored in the unit 300.

  The system information list 301 has a data structure as shown in FIG. 3, and for each registered user, a user account, a mail address, a password, a received mail server name, a transmitted mail server name, a mail reception interval, etc. are stored. Account information area 302, network server name, various IP addresses, image server address, server information area 303 for storing other network information, and shooting information area for storing whether or not an image is taken by the image pickup unit 212, a shooting interval, etc. 304 and an area in which various parameters are set, such as a user ID area 305 in which the user ID of the registered user is stored. The user ID in the user ID area 305 is associated with the user ID in the user authentication information table. Therefore, by searching the user authentication information table using the user ID in the user ID area 305 as a key, the feature amount of the user can be acquired. Also, the user authentication information table is searched using the feature quantity detected by the face authentication process as a key, and if it matches the feature quantity of any user ID, it is known which registered user's face is detected by the user ID. The system information of the registered user can be obtained from the system information list 301 based on the user ID. Further, when the feature amounts do not match with each other, it is understood that an unregistered face has been detected.

  Next, image display processing using the image display method by the image display apparatus 100 according to the present embodiment will be described with reference to the flowcharts of FIGS. First, when the power is turned on, the image display apparatus 100 is activated and a reset signal is inputted to the microprocessor 201. By this operation, as shown in FIG. 4, the program code is read from the ROM 203 to the microprocessor 201 and the execution of the image display process is started (step S10). Note that the control of the following processing is performed by the control unit 220 in accordance with the program code read by the microprocessor 201.

  Next, the control unit 220 initializes the system (step S20). This initialization initializes devices that control each function, such as the network control unit 205, the operation control unit 202, the external memory terminal 210, the imaging unit 212, and the video controller 213, and sets parameters necessary for the operation. It is. The procedure of the system initialization process will be described with reference to the flowchart shown in FIG. 5. First, the external memory terminal 210 that controls the external storage medium 211 is initialized (step S201), and then the device is set. The parameters to be read are read from the system information list 301 in the storage unit 300 (step S202). Based on these parameters, initialization of the video controller 213 (step S203), state initialization of the operation control unit 202 (step S204), setting of various data of the network control unit 205 (step S205), and the imaging unit 212 Initialization (step S206) is sequentially executed.

  When the initialization process (step S20) is completed, the control unit 220 checks the storage medium in preparation for image display on the display 214 (step S30). Specifically, in this embodiment, the setting is such that image data is first read from the external storage medium 211 in accordance with the standard value stored in the ROM 203. The image data list stored in the external storage medium 211 is read, and a display list indicating the order for displaying images on the display 214 is created based on the image data list. In addition to the external storage medium 211, image data stored in the ROM 203 and the FLASH memory 209 and image data acquired via the network 206 are also used for display and are added to the display list.

  When various settings are completed in this manner, image display processing (step S40), image shooting and face authentication processing (steps S50 and S60), face authentication result determination processing (step S70), and e-mail processing (step S80) is executed. Although these processes are described in series in the flowchart shown in FIG. 4, they are actually parallel processes, and the control unit 220 constantly monitors interrupt processes and state changes, and in response to these, The process is instructed to execute one of the processes independently or in parallel. Specifically, an image display process (step S40) is normally executed, and in parallel with this, image shooting (step S50) and face authentication process (step S60) are executed at regular intervals. Then, when a person's face is detected or an instruction to execute the e-mail process is given by an instruction from the operation control unit 202, the image display process is interrupted, and the process proceeds to the e-mail process (step S80). . Before this transition, the face authentication result determination process (step S70) is executed. If there is an unregistered user or no face is detected, the e-mail process is not performed and the process returns to the image display process. When the e-mail process is completed, the process returns to the image display process. Also, even when the face authentication process is executed even during the execution of the e-mail process and the authentication state changes due to the change of the number of viewers of the image display device 100 or the increase or decrease of the number of persons, the e-mail to be displayed as necessary It is also possible to perform a process such as changing the e-mail process or returning to the image display process. Further, the entire image display process may be ended when an instruction to end such as power off is given.

  Hereinafter, each process of step S40-step S80 is demonstrated using the flowchart of FIGS. 6-9. First, the image display process (step S40) executed by the image display processing unit 230 will be described. In this process, an image is displayed on the display 214 while replacing the image at regular intervals according to the display list created in step S30. In the present embodiment, images are displayed in a predetermined order according to the display list as described above, but the present invention is not limited to this, and the images may be displayed randomly.

  Details of the image display processing will be described with reference to the flowchart of FIG. First, the image display processing unit 230 reads the system information list 301 in the storage unit 300 and performs settings related to the operation (step S401). This step S401 may be performed every time the image display process is performed, or may be performed only once when the image display apparatus 100 is activated. Next, the control unit 220 determines whether an electronic mail is currently being displayed (step S402). If the electronic mail is being displayed, the image display process is terminated. If the e-mail is not being displayed, the process proceeds to the image data reading process (step S403).

  In step S403, the image display processing unit 230 reads necessary image data from the external storage medium 211 based on the display list. Next, based on the size and format of the read image data, the image data is converted into an image size and format optimal for output from the video controller 213, and an image for display on the display 214 is generated. (Step S404). Next, referring to the system information list 301, it is determined whether or not the image has a display effect setting (step S405). If there is a display effect setting, display processing with an effect is performed on the image ( Step S406) If there is no display effect setting, a simple display process is performed for the image (step S407).

  Next, the image capturing process (step S50) will be described with reference to the flowchart of FIG. First, the photographing information area 304 of the system information list 301 shown in FIG. 3 is confirmed, and the state of the imaging unit 212 and the photographing interval are confirmed (step S501). Based on this information, under the control of the control unit 220, the imaging unit 212 periodically captures an authentication image at every shooting interval (step S502). Then, the authentication image data is converted into a format optimal for acquiring the image feature amount (step S503), and the converted authentication image data is stored in the storage unit 300 (step S504).

  Next, the face authentication process (step S60) executed by the face authentication unit 240 will be described using the flowchart of FIG. First, the face authentication unit 240 acquires necessary information from the user information list 301 (step S601). Next, the authentication image data acquired in step S50 is read from the storage unit 300 (step S602), the authentication image data is analyzed to extract a feature amount, and a person's face is searched based on the feature amount. (Step S603). As the face detection method, a conventionally known appropriate detection method can be used. In the face detection, only the face having a size equal to or larger than the threshold may be detected. This is because a person who wants to view e-mail brings his face close to the image display apparatus 100, and thus the face of the person is detected to a certain extent. Conversely, a person with a face smaller than the threshold is likely to be a person away from the image display device 100 or a person passing by. There is no need to display an e-mail for such a person, and even if an e-mail of a registered user who wishes to view the e-mail is displayed, there is no fear of being read by these persons. In addition, when a person with a face smaller than the threshold value is set as an authentication target, if the person is an unregistered user, the registered user who originally wanted to visually recognize the electronic mail may not be visible.

  Next, the face authentication unit 240 reads the user authentication information table from the storage unit 300, and acquires the user ID and feature amount of the registered user (step S604). Based on the registered user information, the following processing is performed for all the faces detected in step S603. First, if there is a detected face in the determination processing in step S605, the user authentication information table is searched using the feature amount of the face as a key to check whether there is a match in the feature amount (step S606). . As a result of this collation, if there is a matching feature amount, the user ID of the registered user having the feature amount is acquired from the user authentication information table, and the face feature amount and the user ID are associated with each other to detect the information table. To remember. On the other hand, if it does not coincide with any feature amount, for example, “null” is stored in the detection information table as the feature amount of the face and the user ID (step S607). When the processes in steps S606 and S607 are completed for all detected faces, the face authentication process (step S60) is terminated.

  Through the face authentication process (step S60) described above, all detected face feature amounts and user IDs are set in the detection information table. By referring to this user ID, it can be seen which registered user is detected. When the user ID is “null”, it is understood that an unregistered user has been detected. For registered users, the system information list 301 can be searched based on the user ID to obtain user information such as an e-mail address.

  Next, the e-mail process (step S80) executed by the e-mail processing unit 250 will be described with reference to the flowchart of FIG. As described above, when it is determined in step S70 that an unregistered face is detected based on the result of the face authentication process (step S60), that is, a feature amount is set in the detection information table. If there is even one with the user ID “null”, this e-mail processing (step S80) is not executed to prevent the unregistered user from seeing the e-mail, and the detected face It is executed only when everything belongs to a registered user. Also, no e-mail process is performed when no face is detected (see FIG. 4).

  In the e-mail process (step S80), first, for all registered users of the user ID set in the detection information table, from the account information area 302 of the user information list 301, a user account, e-mail address, password, transmission / reception server name, e-mail A reception interval or the like is acquired (step S801). In this embodiment, 208 in FIG. 1 is designated as the electronic mail server. The e-mail processing unit 250 connects to the e-mail server 208 at every acquired e-mail reception interval, and performs e-mail arrival confirmation (step S802). Each time this process is performed, authentication is performed by the e-mail server 208 based on the account information acquired in step S801, and the result is acquired (step S803). Based on this result, it is determined whether or not authentication has been performed (step S804). If authentication has been performed, the presence or absence of an e-mail is confirmed. When arrival of an e-mail is detected, an e-mail is acquired from the e-mail server 308. (Step S805). On the other hand, when the authentication is not performed, that is, when an error such as the communication path is interrupted, the e-mail is not received and error processing corresponding to the error is performed such as notifying of an error message (step S806).

  In step S805, when arrival of an e-mail is notified and an e-mail is received, these e-mails are stored in the storage unit 300 (step S807). Each electronic mail is sorted and stored in a mail box for each user ID (for each mail address). At this time, the sender, destination, sender name and outgoing date / time set in the header part of the e-mail, the text, and the attached file are decoded and stored. Alternatively, the viewer may be notified by displaying a symbol or the like on the display 214 via the video controller 213 through the reception of the e-mail. If it is confirmed in step S805 that no e-mail has arrived, step S807 is not performed.

  Next, the processing of steps S808 to S809 is performed to display the contents of the e-mail on the display 214. This e-mail display displays only e-mails in which the e-mail addresses of all detected registered users are set as destinations (which may be either TO or CC). For this reason, even one of the detected registered users does not display an email that is not designated as the destination. A specific processing procedure will be described below.

  First, a mailbox is searched to obtain an e-mail to be displayed (step S808). As described above, the e-mails to be displayed in this case are only those in which all detected registered users are set as destinations (TO or CC). In addition, when searching for a mailbox, all the registered user mailboxes detected may be search targets, but from the viewpoint of improving processing efficiency, in the present embodiment, the first registered user stored in the detection information table is used. Search only mailboxes for. Because it is possible to determine whether or not the e-mail addressed to all registered users detected this time is referred to by referring to TO and CC that are destinations of e-mails of the registered user. As a result, only e-mails addressed to all registered users It is because it can acquire. When only one registered user is detected, all e-mails in the registered user's mailbox are displayed.

  Next, the e-mail acquired above is displayed on the display 214 via the video controller 213 (step S809). In this display, the title, sender information, recipient information, transmission date and time, etc. are displayed as a list in the same way as general email display methods, and any email can be selected by clicking or touching the screen. Thus, the content of the text can be displayed on the display 214. Furthermore, an attached file such as image data attached to an e-mail may be displayed. Information such as the total number of e-mails that have arrived and the number of e-mails and how many e-mails are displayed may be displayed.

  As described above, a registered user who wishes to view an e-mail can view an e-mail addressed to all the registered users. When there is only one viewer, all e-mails addressed to the viewer can be viewed. In addition, even if an e-mail is addressed to any registered user who is currently viewing, if there is a registered user who is not the addressee among the viewers, the e-mail is not displayed. Can be prevented. On the other hand, if there is no email to be displayed, the email processing (step S80) is immediately terminated. At this time, the viewer may be notified that there is no display target electronic mail.

  Note that the viewer may be changed when the viewer leaves the seat or another viewer joins the viewer. However, since image shooting and face authentication processing are executed at regular intervals in parallel with the email processing, the viewer is re-authenticated and the display content of the email is displayed according to the authentication result. It can also be changed. Further, when all the viewers are away, or when the viewer performs a viewing end operation, the e-mail process (step S80) is ended. Thereby, since it returns to image display mode, it can prevent that an e-mail is visually recognized by others.

  In the above embodiment, the image display method executed by the image display process of the image display device 100 is implemented as a program executed by the microprocessor 201. Here, this program can be stored in a storage medium or storage device (storage unit 300) such as a flexible disk, CD-ROM, magneto-optical disk, semiconductor memory, and hard disk. In some cases, the digital signal is distributed via the network 206 or the like. At this time, intermediate processing results are temporarily stored in a storage device such as a memory. Alternatively, this program may be configured as a dedicated IC chip (for example, ASIC). Moreover, although the example of the digital photo frame was given as the image display apparatus 100, it can also be implemented by a mobile phone or a portable music device.

100 Image Display Device 212 Imaging Device (Detection Unit)
214 Display (display unit) 201 Microprocessor (control unit)
300 storage unit

Claims (3)

A storage unit storing image data, e-mail, and registered user authentication information;
A display unit for displaying the image data or the e-mail;
A detection unit for detecting information of a person around the display unit;
The image data is read from the storage unit and displayed on the display unit, or identification information for identifying the person is detected from the person information when the detection unit detects human information. Determining whether or not the detected identification information matches the authentication information of the registered user read from the storage unit, and if it matches, an e-mail addressed to the registered user is displayed on the display unit And a control unit that displays the image data on the display unit when they do not match,
An image display apparatus.   When a plurality of registered users are detected, the control unit displays the e-mail addressed to all of the plurality of registered users on the display unit, and at least one of the detected registered users is the destination The image display device according to claim 1, wherein the electronic mail is not displayed when it is not included.   The detection unit is an imaging device that captures authentication image data of a person around the display unit, and the control unit is configured to detect a face as the identification information from the authentication image data captured by the imaging device. The image display device according to claim 1, wherein the image display device is configured to detect whether the face matches the authentication information read from the storage unit.

Images