JP3559622B2

JP3559622B2 - Gaze detection device and optical apparatus having gaze detection device

Info

Publication number: JP3559622B2
Application number: JP19498995A
Authority: JP
Inventors: 明彦長野
Original assignee: Canon Inc
Current assignee: Canon Inc
Priority date: 1995-07-31
Filing date: 1995-07-31
Publication date: 2004-09-02
Anticipated expiration: 2015-07-31
Also published as: JPH0938038A

Description

【０００１】
【発明の属する技術分野】
本発明は、使用者または被験者の視線情報を検出する視線検出装置に関し、特に明るさの情報を利用して、正確な視線検出を可能にする視線検出装置に関する。
【０００２】
【従来の技術】
従来より、ファインダーを覗く使用者または被験者の視線を検出してその視線情報に基づいて撮影レンズの自動焦点調節等の機能を制御するカメラが市販されている。
【０００３】
この種のカメラにおいて撮影者の視線情報は、撮影者の眼球を照明した照明光による角膜反射像の位置と、瞳孔の中心位置とのずれより検出される。視線情報の検出方法については特開平２−２６４６３３号公報に開示されている。
【０００４】
特開平２−２６４６３３号公報によると、瞳孔の中心位置は瞳孔と虹彩との境界を検出することによって求めているが、実際には光の強度差の大きいところを瞳孔と虹彩との境界とみなして検出を行っている。したがって、瞼と眼球との境界や強膜と虹彩との境界、あるいはまつげのある部分においても光の強度差が大きいところがあり偽の境界を検出してしまう場合があった。
【０００５】
そこで、本願出願人は特開平２−６５８３６号公報において外光の明るさから使用者または被験者の瞳孔径を推定してその瞳孔径から大きく外れている検出データを排除して瞳孔と虹彩との境界を効率的に抽出する方法を開示している。
【０００６】
さらに、本願出願人は特開平６−１４８５０７号公報において使用者または被験者の眼球の瞳孔径を予め設定しセンサー上での瞳孔と虹彩との境界が存在する範囲を設定して偽の検出データを排除する方法を開示している。
【０００７】
【発明が解決しようとする課題】
しかしながら、特開平２−６５８３６号公報に開示されているようにファインダーを覗く使用者または被験者の瞳孔径を外光の明るさによって推定することができるが、推定した瞳孔径は眼球像の結像倍率が考慮されていないため、受光センサー上で、瞳孔が実際にどの程度の大きさになるか正確には推定できないという欠点があった。
【０００８】
また、使用者または被験者の瞳孔径をファインダーに入射する光量によって推定すると、例えば明るい屋外から暗い室内等を観察する場合には、ファインダーに入射する光量は少ないにもかかわらず使用者または被験者の眼球の瞳孔は周囲の明るさによって小さくなるため、推定した瞳孔径は実際の瞳孔径よりも大きくなってしまい、不要な検出データを正確に排除することができず、瞳孔と虹彩の境界を正確に検出することができないことがあった。
【０００９】
【課題を解決するための手段】
本願の請求項１に記載した発明によれば、実質的に使用者または被験者の眼球に入射する光量を測光する測光手段と、使用者または被験者眼の像を受光する受光手段と、該受光手段で得られた眼球像信号を演算処理して使用者または被験者の視線情報を求める演算処理手段とを有する視線検出装置において、前記測光手段の出力に基づいて推定した使用者または被験者の瞳孔径と、前記受光手段の出力に基づいて推定した使用者または被験者の瞳孔径を比較して、より小さい瞳孔径を使用者または被験者眼の推定瞳孔径として設定する推定瞳孔径設定手段を有し、前記演算処理手段は設定された該推定瞳孔径に基づいて前記受光手段で得られた眼球像信号から偽の抽出データを排除する演算処理をすることによって、より正確に瞳孔径を推定することができるので、瞳孔と虹彩との境界を検出する際の不要な検出データを正確に排除することが可能になる。
【００１０】
また、本願の請求項３に記載した発明による視線検出装置は、使用者または被験者の眼球像を受光する受光面を有する受光手段と、実質的に使用者または被験者の眼球に入射する光量を測光する測光手段と、該測光手段の出力に基づいて使用者または被験者の瞳孔径を推定する瞳孔径推定手段と、使用者または被験者の眼球と受光手段の受光面との距離情報を求める距離情報検出手段と、該瞳孔径推定手段によって推定された使用者または被験者の瞳孔径および該距離情報検出手段によって求めた距離情報から該受光手段の受光面に結像する使用者または被験者の瞳孔径を演算する瞳孔径演算手段と、該瞳孔径演算手段の演算結果に基づいて、該受光手段によって得られた眼球像情報から偽の抽出データを排除する演算処理をすることで使用者または被験者の視線情報を求める演算処理手段とを有することを特徴として、このような構成によって、受光手段の受光面に結像する瞳孔径を求めることができ、より正確に瞳孔径を推定することができるので、瞳孔と虹彩との境界を検出する際の不要な検出データを正確に排除することが可能になる。
【００１１】
【発明の実施の形態】
図１〜図５は本発明の実施の形態で、図１は視線検出装置付き光学機器であるところの一眼レフカメラの要部概略図である。
【００１２】
図中１は撮影レンズであり、便宜上２枚のレンズ１ａ、１ｂで示したが、実際は多数のレンズから構成されている。２は主ミラーで、観察状態と撮影状態に応じて撮影光路へ斜設されあるいは退去される。３はサブミラーで、主ミラー２を透過した光束をカメラボディの下方へ向けて反射する。４はシャッター、５は感光部材で、銀塩フィルムあるいはＣＣＤやＭＯＳ型等の固体撮像素子あるいはビディコン等の撮像管より成っている。
【００１３】
６は焦点検出装置であり、結像面近傍に配置されたフィールドレンズ６ａ、反射ミラー６ｂ及び６ｃ、２次結像レンズ６ｄ、絞り６ｅ、複数のＣＣＤからなるラインセンサー６ｆ等から構成されている周知の位相差方式を採用している。同図の焦点検出装置６は、観察画面内の複数の領域を焦点検出可能なように構成されている。
【００１４】
７は撮影レンズ１の予定結像面に配置されたピント板、８はファインダー光路変更手段であるペンタダハプリズム、９、１０は観察画面内の被写体輝度を測定する為の結像レンズと測光センサーで、結像レンズ９はペンタダハプリズム８内の反射光路を介してピント板７と測光センサー１０を共役に関係付けている。
【００１５】
次にペンタダハプリズム８の射出面後方には、光分割手段であるダイクロイックミラー１６と接眼レンズ１１が配され、撮影者の眼１５によるピント板７の観察に使用される。光分割手段であるダイクロイックミラー１６は、可視光を透過し赤外光を反射する特性を有している。１２は受光レンズ、１４はＣＣＤ等の光電素子列を２次元的に配したイメージセンサーで受光レンズ１２に関して所定の位置にある撮影者の眼１５の虹彩近傍と共役になるように配置されている。
【００１６】
１３ａ〜１３ｄは各々撮影者の目１５の照明光源であるところの赤外発光ダイオード（ＩＲＥＤ）で、接眼レンズ１１の下方に配置されている。
【００１７】
３１は撮影レンズ１内に設けた絞り、３２は絞り駆動回路１０７を含む絞り駆動装置、３３はレンズ駆動用モーター、３４は駆動ギヤ等からなるレンズ駆動部材、３５はフォトカプラーでレンズ駆動部材３４に連動するパルス板３６の回転を検知してレンズ駆動回路１１０に伝えている。レンズ駆動回路１１０は、この情報とカメラ側からのレンズ駆動量の情報に基づいてレンズ駆動用モーターを所定量駆動させ、撮影レンズ１の合焦レンズ１ａを合焦位置に移動させている。３７は公知のカメラとレンズとのインターフェイスとなるマウント接点である。
【００１８】
図２は本発明を実施したカメラの本体に内蔵された電気回路の要部ブロック図である。同図において、図１と同一のものは同一の番号をつけている。カメラ本体に内蔵されたマイクロコンピュータの中央処理装置（以下ＣＰＵ）１００には、視線検出回路１０１、測光回路１０２、焦点検出回路１０３、信号入力回路１０４、絞り駆動回路１０７、シャッター制御回路１０８、レンズ駆動回路１１０が接続されている。また、撮影レンズ内に配置された絞り駆動回路１０７とレンズ駆動回路１１０とは図１で示したマウント接点３７を介して信号の伝達がなされる。
【００１９】
ＣＰＵ１００には、記憶手段としてＥＥＰＲＯＭ１００ａが付随している。
【００２０】
視線検出回路１０１は、ＩＲＥＤ１３ａ〜１３ｄを点灯制御して撮影者の眼球１５を照明し、イメージセンサー１４からの眼球像の出力をＡ／Ｄ変換しこの像情報をＣＰＵ１００に送信する。ＣＰＵ１００は眼球像を画像処理して角膜反射像と虹彩と瞳孔の境界を抽出し、さらに角膜反射像と瞳孔の中心位置のずれから撮影者の視線情報を算出する。
【００２１】
測光回路１０２は、測光センサー１０からの出力を増幅後、対数圧縮、Ａ／Ｄ変換し、各センサーの輝度情報としてＣＰＵ１００に送っている。
【００２２】
ラインセンサー６ｆは画面内の複数の焦点検出領域に対応した公知のＣＣＤラインセンサーである。焦点検出回路１０３はこれらラインセンサー６ｆから得た電圧をＡ／Ｄ変換し、ＣＰＵ１００に送っている。
【００２３】
信号入力回路１０４は、不図示のシャッターレリーズボタンの第１ストローク（スイッチＳＷ−１）と第２ストローク（スイッチＳＷ−２）の信号入力を検知してＣＰＵ１００にその信号を送信している。
【００２４】
絞り駆動回路１０７は、撮影レンズ１の絞り３１が所定の大きさになるように絞り駆動装置３２を制御している。
【００２５】
シャッター制御回路１０８は通電するとシャッター幕を走行させて、感光部材に所定光量を露光させる。
【００２６】
ここで本発明の視線検出装置は、照明手段を構成するＩＲＥＤ１３ａ〜１３ｄ、ＩＲＥＤを駆動する視線検出回路１０１、受光手段を構成する接眼レンズ１１、ダイクロイックミラー１６、受光レンズ１２、イメージセンサー１４、イメージセンサー１４を制御する視線検出回路１０１、演算処理手段を構成する視線検出回路１０１及びＣＰＵ１００、記憶手段であるＥＥＰＲＯＭ１００ａ、測光手段を構成する結像レンズ９、測光センサー１０、測光センサーを制御する測光回路１０２とから構成されている。
【００２７】
次に、視線検出装置を具備した光学機器であるところのカメラの動作のフローチャートを図３に示す。本発明のカメラは、使用者または被験者の視線情報を用いて焦点検出領域の選択と測光値の重み付けが可能となっている。
【００２８】
撮影者がカメラを起動させると、ＣＰＵ１００は信号入力回路１０４を介して視線検出装置の動作状態を設定するための不図示のモード選択スイッチの設定状態を確認する（＃１００）。
【００２９】
モード選択スイッチが視線のキャリブレーションモードに設定されていれば（＃１００）、ＣＰＵ１００は撮影者の視線と注視目標を一致させるための視線のキャリブレーションを実行する（＃１０１）。視線のキャリブレーション方法については、特開平６−８６７５９号公報において詳細に説明されているので、ここでは説明を省略する。
【００３０】
またモード選択スイッチが通常の撮影モードに設定されていれば、ＣＰＵ１００は、視線のキャリブレーションを行うモード（ＣＡＬモード）に設定されていないことを認識し（＃１００）、さらに信号入力回路１０４を介してシャッターレリーズボタンの第１ストローク、スイッチＳＷ−１の状態を確認する（＃１０２）。スイッチＳＷ−１がＯＦＦ状態であれば、ＣＰＵ１００はスイッチＳＷ−１がＯＮされるまで待機する（＃１０２）。
【００３１】
撮影者によってスイッチＳＷ−１がＯＮされれば（＃１０２）、ＣＰＵ１００は先に検知したモード選択スイッチの設定状態を確認し（＃１０３）、撮影者の視線情報を用いずに制御手段を制御するモードに設定されていなければ、ＣＰＵ１００は記憶手段であるＥＥＰＲＯＭ１００ａから撮影者の視線のキャリブレーションデータを読み出す（＃１０５）。
【００３２】
さらにＣＰＵ１００は、視線検出回路１０１に信号を送って撮影者の眼球１５をＩＲＥＤ１３にて照明し、その反射光を接眼レンズ１１、ダイクロイックミラー１６、受光レンズ１２を介してイメージセンサー１４にて撮像させる。ＣＰＵ１００は、視線検出回路１０１を介してイメージセンサー１４より得られた眼球像を処理して角膜反射像及び虹彩と瞳孔の境界を抽出し、その角膜反射像と瞳孔の中心位置のずれから撮影者の視線情報を算出する（＃１０６）。視線情報の算出のフローは後述する。
【００３３】
さらにＣＰＵ１００は検出された撮影者の視線情報と視線のキャリブレーションデータによりピント板７上の注視点を算出する（＃１０７）。
【００３４】
ここで、撮影者の視線情報を自動焦点調節手段の制御に用いるモードに設定されていれば、ＣＰＵ１００は前記注視点座標より最寄りの焦点検出領域を選択する（＃１０９）。
【００３５】
引き続きＣＰＵ１００は焦点検出回路１０３に信号を送信して撮影者の視線情報に基づいて選択された焦点検出領域の焦点検出を実行する（＃１１０）。選択された焦点検出領域の焦点状態が合焦していなければ（＃１１１）、ＣＰＵ１００はレンズ駆動回路１１０に焦点調節信号を送信して撮影レンズ１レンズ駆動を行う（＃１２０）。焦点調節信号に対応したレンズ駆動が実行されると（＃１２０）再度焦点検出が実行され（＃１２１）。
【００３６】
選択された焦点検出領域の焦点状態が合焦していれば（＃１１１）、ファインダー内に合焦表示を行う（＃１１２）。ファインダー視野内に合焦表示が行われるため、撮影者は撮影レンズが所望の焦点検出領域において合焦していることが認識できる。
【００３７】
また、撮影者の視線情報を露出値の決定に用いるモードに設定されていれば（＃１１３）、ＣＰＵ１００は測光回路１０２に信号を送信して測光を行なう。このとき、撮影画面内の分割された領域の測光値に撮影者の視線情報に基づいた重み付けを行って露出値が決定する（＃１１４）。
【００３８】
引き続きスイッチＳＷ−１がＯＮされていれば（＃１１５）、シャッターレリーズボタンのスイッチＳＷ−２の状態が確認される（＃１１６）。スイッチＳＷ−２がＯＦＦ状態であれば（＃１１６）、再びスイッチＳＷ−１の状態の確認を行う（＃１１５）。スイッチＳＷ−１がＯＦＦ状態であれば（＃１１５）、そのまま初期状態に戻る。
【００３９】
また、スイッチＳＷ−２がＯＮされたならば（＃１１６）、ＣＰＵ１００は絞り駆動回路１０７に信号を送信して絞り３１を所定の開口に設定するとともに、シャッター制御回路１０８に信号を送信しシャッター幕を走行させて撮影を行う（＃１１７）。
【００４０】
カメラのシャッターレリーズ動作が終了すると（＃１１７）、初期状態に戻る。
【００４１】
一方、撮影者によってスイッチＳＷ−１がＯＮされれば（＃１０２）、ＣＰＵ１００は先に検知したモード選択スイッチの設定状態を確認し、撮影者の視線情報を用いて制御手段を制御するモードに設定されていなければ制御手段は独立して制御される（＃１０３）。
【００４２】
ＣＰＵ１００は焦点検出回路１０３に信号を送信して全ての焦点検出領域の焦点検出を実行する（＃１１８）。さらにＣＰＵ１００は、検出された全ての焦点検出領域の焦点検出情報を比較して、被写体距離がもっとも短い焦点検出領域を撮影レンズの焦点調節を行う領域と決定する（＃１１９）。
【００４３】
決定された焦点検出領域に対して合焦していれば（＃１１１）合焦表示を行う（＃１１２）。
【００４４】
また、露出値も撮影者の視線情報を用いずに決定される（＃１１３）。ＣＰＵ１００は測光回路１０２に信号を送信して測光を行なう。このとき、主被写体があると思われる領域が適正な露出量になるように評価を行って露出値が決定される（＃１２３）。
【００４５】
引き続きスイッチＳＷ−１がＯＮされていれば（＃１１５）、シャッターレリーズボタンのスイッチＳＷ−２の状態が確認される（＃１１６）。スイッチＳＷ−２がＯＦＦ状態であれば（＃１１６）、再びスイッチＳＷ−１の状態の確認を行う（＃１１５）。スイッチＳＷ−１がＯＦＦ状態であれば（＃１１５）、そのまま初期状態に戻る。
【００４６】
また、スイッチＳＷ−２がＯＮされたならば（＃１１６）、ＣＰＵ１００は絞り駆動回路１０７に信号を送信して絞り３１を所定の開口に設定するとともに、シャッター制御回路１０８に信号を送信しシャッター幕を走行させて撮影を行う（＃１１７）。
【００４７】
カメラのシャッターレリーズ動作が終了すると（＃１１７）、初期状態に戻る。
【００４８】
図４は視線情報の算出のフローで、以下同図を用いて説明する。
【００４９】
視線検出の実行が指示されると（＃１０６）、ＣＰＵ１００は測光手段である測光回路１０２に信号を送信して測光センサー１０の信号を読みだしてファインダー光学系に入射している光量を測定する（＃１３０）。記憶手段であるＥＥＰＲＯＭ１００ａにはファインダー光学系の測光値に対する使用者または被験者の瞳孔径の情報が記憶されているため、ＣＰＵ１００はＥＥＰＲＯＭ１００ａから測定されたファインダー光学系の測光値に対応する使用者または被験者の瞳孔径情報を読みだして瞳孔径ｒ１と推定する（＃１３１）。このとき、ファインダー光学系の測光値が明るければ瞳孔径ｒ１は小さい値となり、またファインダー光学系の測光値が暗ければ瞳孔径ｒ１は大きい値となる。
【００５０】
さらにＣＰＵ１００は、受光手段である視線検出回路１０１に信号を送信してイメージセンサーであるＣＣＤ１４を所定時間蓄積させ信号を出力させる（＃１３２）。記憶手段であるＥＥＰＲＯＭ１００ａにはイメージセンサー１４の出力に対する使用者または被験者の瞳孔径情報が記憶されているため、ＣＰＵ１００はＥＥＰＲＯＭ１００ａから測定されたイメージセンサー１４の光出力値に対応する使用者または被験者の瞳孔径情報を読みだして瞳孔径ｒ２と推定する（＃１３３）。このとき、使用者または被験者の前眼部が明るければ瞳孔径ｒ２は小さい値となり、また使用者または被験者の前眼部が暗ければ瞳孔径ｒ２は大きい値となる。受光手段を用いて使用者または被験者の前眼部の明るさを測定する際、照明手段によって使用者または被験者の眼球を照明しない方が望ましい。
【００５１】
ＣＰＵ１００は、ファインダー光学系に入射する光量に基づいて推定された瞳孔径ｒ１と、使用者または被験者の前眼部の明るさに基づいて推定された瞳孔径ｒ２とを比較してより小さい瞳孔径の方を最終的な設定値とする（＃１３４）。たとえば、使用者または被験者が暗い屋内から明るい屋外を観察している場合は、ｒ１＜ｒ２となるため、瞳孔径はｒ１と設定される。逆に、太陽光下の明るい屋外から暗い室内等を観察する場合は、ｒ１＞ｒ２となるため、瞳孔径はｒ２と設定される。このように、より小さい瞳孔径を最終的な設定値にすることで推定瞳孔径はより実際の瞳孔径に近づけることができる。したがって、どのような場合にも正確に不要なデータを排除することができる。
【００５２】
ファインダー光学系に入射する光量と使用者または被験者の前眼部の明るさに基づいて使用者または被験者の瞳孔径が設定されると（＃１３４）、ＣＰＵ１００は照明手段である視線検出回路１０１に信号を送信してＩＲＥＤ１３によって使用者または被験者の眼球を照明するとともに、受光手段であるイメージセンサー１４にて眼球からの反射光を受光する（＃１３５）。ＣＰＵ１００は、イメージセンサー１４にて撮像された眼球像を視線検出回路１０１を介してＡ／Ｄ変換して画像処理を行い、角膜反射像と瞳孔と虹彩の境界を抽出する（＃１３６）。
【００５３】
図５はイメージセンサー１４に形成される使用者または被験者の眼球像と画像処理結果を示したものである。同図において２個の黒丸（・）が角膜反射像である。角膜反射像は、光強度が強いためほぼ正確に検出される。また、図中 × と □ で示したのが光の強度差が大きいために瞳孔と虹彩との境界として抽出されたところである。
【００５４】
さらにＣＰＵ１００は、２個の角膜反射像の間隔から使用者または被験者眼の受光手段に対する距離を求めて、その距離から使用者または被験者眼の結像倍率βを算出する（＃１３７）。結像倍率βは、２個の角膜反射像の間隔をΔＰとすると、
【００５５】
【外１】

と表される。ただし、ａ、ｂ、ｃ、ｄは照明手段、受光手段の設定状態で決まる定数である。
【００５６】
先に設定された使用者または被験者眼の瞳孔径ｒと使用者または被験者眼の結像倍率βより、イメージセンサー１４上での瞳孔の大きさが求められる。さらに、２個の角膜反射像の位置とイメージセンサー１４上での瞳孔の大きさに基づいて瞳孔の存在範囲が設定される（＃１３８）。図５に示した眼球像の中で、瞳孔の存在範囲を示したのが点線である。
【００５７】
ＣＰＵ１００は瞳孔の存在範囲に基づいて、瞳孔と虹彩との境界に関する偽の抽出データを排除する（＃１３９）。このとき図５において、×印で示したまぶたと眼球との境界で抽出されたデータや強膜と虹彩との境界で抽出されたデータが排除される。さらにＣＰＵ１００は、瞳孔と虹彩との境界として抽出された残りのデータ（図５における□印のデータ）に基づいて瞳孔の中心位置を算出する（＃１４０）。
【００５８】
瞳孔の中心位置が求まると、角膜反射像との位置の差より使用者または被験者眼の回転角が算出される（＃１４１）。
【００５９】
本実施例において、瞳孔径に基づく瞳孔の存在範囲の設定は、使用者または被験者眼の回転を考慮して幾分大きめに設定されるのはいうまでもない。
【００６０】
また本実施例において、角膜反射像及び瞳孔と虹彩との境界を抽出後に瞳孔の存在範囲を設定して偽の抽出データを排除する方法を例示したが、先に角膜反射像を抽出して瞳孔の存在範囲を設定後、瞳孔と虹彩との境界とを抽出してもかまわない。
【００６１】
【発明の効果】
以上説明したように、本願の請求項１に記載した発明によれば、実質的に使用者または被験者の眼球に入射する光量を測光する測光手段と、使用者または被験者眼の像を受光する受光手段と、該受光手段で得られた眼球像信号を演算処理して使用者または被験者の視線情報を求める演算処理手段とを有する視線検出装置において、前記測光手段の出力に基づいて推定された使用者または被験者の瞳孔径と、前記受光手段の出力に基づいて推定された使用者または被験者の瞳孔径の少なくともどちらか一方を利用して使用者または被験者眼の推定瞳孔径を設定する瞳孔径設定手段を有し、前記演算処理手段は設定された該推定瞳孔径に基づいて前記眼球像信号を演算処理することによって、より正確に瞳孔径を推定することができる。したがって、瞳孔と虹彩との境界を検出する際の不要な検出データをより正確に排除することが可能になり、短時間で正確に瞳孔形状を検出することができるので、視線検出の高速化、高精度化を図ることができる。
【００６２】
また、本願の請求項５に記載した発明による視線検出装置は、該使用者または被験者の眼球像を受光する受光面を有する受光手段と、実質的に使用者または被験者の眼球に入射する光量を測光する測光手段と、該測光手段の出力に基づいて該使用者または被験者の瞳孔径を推定する瞳孔径推定手段と、該使用者または被験者の眼球と受光手段の受光面との距離情報を求める距離情報検出手段と、該瞳孔径推定手段によって推定された該使用者または被験者の瞳孔径および該距離情報検出手段によって求めた距離情報から該受光手段の受光面に結像する該使用者または被験者の瞳孔径を演算する瞳孔径演算手段と、該瞳孔径演算手段の演算結果に基づいて、該受光手段によって得られた眼球像情報を演算処理することで使用者または被験者の視線情報を求める演算処理手段とを有することを特徴として、このような構成によって、受光手段の受光面上の推定瞳孔径を求めることができ、より正確に瞳孔径を推定することができる。したがって、瞳孔と虹彩との境界を検出する際の不要な検出データをより正確に排除することが可能になり、短時間で正確に瞳孔形状を検出することができるので、視線検出の高速化、高精度化を図ることができる。
【図面の簡単な説明】
【図１】一眼レフカメラの要部概略図
【図２】カメラの電気回路ブロック図
【図３】カメラの動作フロー説明図
【図４】視線算出フロー説明図
【図５】眼球像説明図
【符号の説明】
１撮影レンズ
２主ミラー
３サブミラー
４シャッター
５フィルム
６焦点検出装置
７ピント板
８ペンタダハプリズム
９結像レンズ
１０測光センサー
１１接眼レンズ
１２受光レンズ
１３赤外発光ダイオード
１４ＣＣＤ
１５眼球
３２絞り駆動装置
１００ＣＰＵ
１０１視線検出回路
１０２測光回路
１０３焦点検出回路
１０４信号入力回路
１０７絞り駆動回路
１０８シャッター制御回路
１１０レンズ駆動回路[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a gaze detection device that detects gaze information of a user or a subject, and more particularly to a gaze detection device that enables accurate gaze detection by using brightness information.
[0002]
[Prior art]
2. Description of the Related Art Conventionally, cameras that detect a line of sight of a user or a subject who looks through a viewfinder and control functions such as automatic focusing of a photographing lens based on the line of sight information are commercially available.
[0003]
In this type of camera, the line-of-sight information of the photographer is detected from the deviation between the position of the corneal reflection image due to the illumination light illuminating the photographer's eyeball and the center position of the pupil. A gaze information detection method is disclosed in Japanese Patent Application Laid-Open No. 2-264633.
[0004]
According to JP-A-2-264633, the center position of the pupil is determined by detecting the boundary between the pupil and the iris. However, in practice, a place where the difference in light intensity is large is regarded as the boundary between the pupil and the iris. Detection. Therefore, even at the boundary between the eyelid and the eyeball, the boundary between the sclera and the iris, or the portion with eyelashes, there is a place where the difference in light intensity is large and a false boundary may be detected.
[0005]
In view of this, the present applicant estimates the pupil diameter of a user or a subject from the brightness of external light in Japanese Patent Application Laid-Open No. 2-65836 and excludes detection data that largely deviates from the pupil diameter to remove the pupil and iris. A method for efficiently extracting boundaries is disclosed.
[0006]
Further, the applicant of the present application disclosed in Japanese Patent Application Laid-Open No. 6-148507, presets the pupil diameter of the eyeball of the user or the subject, sets the range where the boundary between the pupil and the iris exists on the sensor, and obtains false detection data. A method of elimination is disclosed.
[0007]
[Problems to be solved by the invention]
However, as disclosed in Japanese Patent Application Laid-Open No. 2-65836, the pupil diameter of a user or a subject who looks through a finder can be estimated by the brightness of external light. Since the magnification is not taken into consideration, there is a disadvantage that it is not possible to accurately estimate the actual size of the pupil on the light receiving sensor.
[0008]
Further, when the pupil diameter of the user or the subject is estimated based on the amount of light incident on the finder, for example, when observing from a bright outdoor to a dark room, the eyeball of the user or the subject despite the small amount of light incident on the finder. The estimated pupil diameter is larger than the actual pupil diameter because unnecessary pupil diameter is smaller than the actual pupil diameter, and unnecessary detection data cannot be eliminated accurately. Sometimes it could not be detected.
[0009]
[Means for Solving the Problems]
According to the invention described in claim 1 of the present application, photometric means for measuring the amount of light substantially incident on the eyeball of the user or the subject, light receiving means for receiving an image of the eye of the user or the subject, and the light receiving means In a gaze detection device having arithmetic processing means for calculating the eye-gaze information of the user or the subject by arithmetically processing the eyeball image signal obtained in the above, the pupil diameter of the user or the subject estimated based on the output of the photometric means. Comparing the pupil diameter of the user or the subject estimated based on the output of the light receiving means, and having an estimated pupil diameter setting means for setting a smaller pupil diameter as the estimated pupil diameter of the user or the subject eye, The arithmetic processing means estimates the pupil diameter more accurately by performing an arithmetic processing for eliminating false extraction data from the eyeball image signal obtained by the light receiving means based on the set estimated pupil diameter. Since it is Rukoto, it is possible to accurately eliminate an unnecessary detection data in detecting a boundary between the pupil and iris.
[0010]
The gaze detecting apparatus according to the invention described in claim 3 of the present application includes a light receiving unit having a light receiving surface for receiving an eyeball image of a user or a subject, and a photometer that substantially measures the amount of light incident on the eyeball of the user or the subject. Pupil diameter estimating means for estimating the pupil diameter of the user or the subject based on the output of the photometric means, and distance information detecting the distance information between the eyeball of the user or the subject and the light receiving surface of the light receiving means. Means for calculating the pupil diameter of the user or the subject to be imaged on the light receiving surface of the light receiving means from the pupil diameter of the user or the subject estimated by the pupil diameter estimating means and the distance information obtained by the distance information detecting means. Pupil diameter calculating means, and performing calculation processing for eliminating false extraction data from eyeball image information obtained by the light receiving means based on the calculation result of the pupil diameter calculating means. Or arithmetic processing means for obtaining gaze information of the subject, with such a configuration, a pupil diameter to be imaged on the light receiving surface of the light receiving means can be obtained, and the pupil diameter can be more accurately estimated. Therefore, unnecessary detection data for detecting the boundary between the pupil and the iris can be accurately excluded.
[0011]
BEST MODE FOR CARRYING OUT THE INVENTION
1 to 5 show an embodiment of the present invention, and FIG. 1 is a schematic view of a main part of a single-lens reflex camera which is an optical device with a visual line detection device.
[0012]
In the figure, reference numeral 1 denotes a photographing lens, which is shown by two lenses 1a and 1b for convenience, but is actually composed of a large number of lenses. Reference numeral 2 denotes a main mirror which is inclined or retreated to a photographing optical path according to an observation state and a photographing state. A sub-mirror 3 reflects a light beam transmitted through the main mirror 2 downward of the camera body. Reference numeral 4 denotes a shutter, and reference numeral 5 denotes a photosensitive member.
[0013]
Reference numeral 6 denotes a focus detection device, which includes a field lens 6a, reflection mirrors 6b and 6c, a secondary imaging lens 6d, an aperture 6e, a line sensor 6f including a plurality of CCDs, and the like, which are arranged near the image plane. A well-known phase difference method is adopted. The focus detection device 6 in FIG. 1 is configured to be able to detect a focus in a plurality of regions in an observation screen.
[0014]
Reference numeral 7 denotes a focusing plate arranged on a predetermined image forming plane of the photographing lens 1, reference numeral 8 denotes a penta roof prism which is a finder optical path changing unit, reference numerals 9 and 10 denote imaging lenses and photometric sensors for measuring the luminance of a subject in an observation screen. The imaging lens 9 conjugately connects the focusing plate 7 and the photometric sensor 10 via a reflection optical path in the penta roof prism 8.
[0015]
Next, a dichroic mirror 16 and an eyepiece 11 serving as light splitting means are arranged behind the exit surface of the penta roof prism 8, and used for observation of the focus plate 7 by the photographer's eyes 15. The dichroic mirror 16, which is a light splitting unit, has a characteristic of transmitting visible light and reflecting infrared light. Reference numeral 12 denotes a light receiving lens, and reference numeral 14 denotes an image sensor having a two-dimensional array of photoelectric elements such as CCDs, which are arranged so as to be conjugate with the vicinity of the iris of the photographer's eye 15 at a predetermined position with respect to the light receiving lens 12. .
[0016]
Reference numerals 13a to 13d denote infrared light emitting diodes (IREDs), which are illumination light sources for the eyes 15 of the photographer, and are arranged below the eyepiece 11.
[0017]
Reference numeral 31 denotes an aperture provided in the taking lens 1, reference numeral 32 denotes an aperture driving device including an aperture driving circuit 107, reference numeral 33 denotes a lens driving motor, reference numeral 34 denotes a lens driving member including a driving gear and the like, reference numeral 35 denotes a photocoupler which is a lens driving member 34. The rotation of the pulse plate 36 that is linked with the rotation of the pulse plate 36 is detected and transmitted to the lens drive circuit 110. The lens driving circuit 110 drives the lens driving motor by a predetermined amount based on this information and the information on the lens driving amount from the camera side, and moves the focusing lens 1a of the photographing lens 1 to the focusing position. Reference numeral 37 denotes a mounting contact serving as an interface between a known camera and lens.
[0018]
FIG. 2 is a block diagram of a main part of an electric circuit built in a camera body embodying the present invention. In the figure, the same components as those in FIG. 1 are given the same numbers. A central processing unit (hereinafter, CPU) 100 of a microcomputer built in the camera body includes a line-of-sight detection circuit 101, a photometry circuit 102, a focus detection circuit 103, a signal input circuit 104, an aperture drive circuit 107, a shutter control circuit 108, a lens The drive circuit 110 is connected. Also, signals are transmitted between the aperture driving circuit 107 and the lens driving circuit 110 disposed in the taking lens via the mount contact 37 shown in FIG.
[0019]
The CPU 100 has an EEPROM 100a as storage means.
[0020]
The line-of-sight detection circuit 101 controls the lighting of the IREDs 13 a to 13 d to illuminate the eyeball 15 of the photographer, A / D converts the output of the eyeball image from the image sensor 14, and transmits this image information to the CPU 100. The CPU 100 processes the eyeball image to extract the boundary between the corneal reflection image, the iris and the pupil, and further calculates the gaze information of the photographer from the shift between the corneal reflection image and the center position of the pupil.
[0021]
The photometric circuit 102 amplifies the output from the photometric sensor 10, performs logarithmic compression and A / D conversion, and sends the result to the CPU 100 as luminance information of each sensor.
[0022]
The line sensor 6f is a known CCD line sensor corresponding to a plurality of focus detection areas in the screen. The focus detection circuit 103 performs A / D conversion on the voltage obtained from the line sensor 6f and sends the voltage to the CPU 100.
[0023]
The signal input circuit 104 detects a signal input of a first stroke (switch SW-1) and a second stroke (switch SW-2) of a shutter release button (not shown) and transmits the signal to the CPU 100.
[0024]
The aperture driving circuit 107 controls the aperture driving device 32 so that the aperture 31 of the photographing lens 1 has a predetermined size.
[0025]
When the shutter control circuit 108 is energized, the shutter curtain runs so that the photosensitive member is exposed to a predetermined amount of light.
[0026]
Here, the visual axis detection device of the present invention includes the IREDs 13a to 13d constituting the illumination means, the visual axis detection circuit 101 driving the IRED, the eyepiece 11, the dichroic mirror 16, the light receiving lens 12, the image sensor 14, the image sensor 14, constituting the light receiving means. A line-of-sight detection circuit 101 that controls the sensor 14, a line-of-sight detection circuit 101 and CPU 100 that constitute arithmetic processing means, an EEPROM 100a that is storage means, an imaging lens 9 that constitutes light measurement means, a light measurement sensor 10, and a light measurement circuit that controls the light measurement sensor. 102.
[0027]
Next, FIG. 3 shows a flowchart of the operation of a camera, which is an optical device equipped with a visual line detection device. The camera of the present invention is capable of selecting a focus detection area and weighting a photometric value using line-of-sight information of a user or a subject.
[0028]
When the photographer activates the camera, the CPU 100 checks the setting state of a mode selection switch (not shown) for setting the operation state of the eye-gaze detecting device via the signal input circuit 104 (# 100).
[0029]
If the mode selection switch is set to the line-of-sight calibration mode (# 100), the CPU 100 executes line-of-sight calibration for matching the line of sight of the photographer with the gaze target (# 101). The gaze calibration method is described in detail in JP-A-6-86759, and will not be described here.
[0030]
If the mode selection switch is set to the normal photographing mode, the CPU 100 recognizes that the mode is not set to the line-of-sight calibration mode (CAL mode) (# 100). Then, the first stroke of the shutter release button and the state of the switch SW-1 are confirmed (# 102). If the switch SW-1 is in the OFF state, the CPU 100 waits until the switch SW-1 is turned on (# 102).
[0031]
When the switch SW-1 is turned on by the photographer (# 102), the CPU 100 checks the setting state of the mode selection switch detected earlier (# 103), and controls the control means without using the photographer's line of sight information. If the mode is not set, the CPU 100 reads out the calibration data of the line of sight of the photographer from the EEPROM 100a as the storage means (# 105).
[0032]
Further, the CPU 100 sends a signal to the eye-gaze detection circuit 101 to illuminate the photographer's eyeball 15 with the IRED 13, and causes the reflected light to be imaged by the image sensor 14 via the eyepiece 11, the dichroic mirror 16, and the light-receiving lens 12. . The CPU 100 processes the eyeball image obtained from the image sensor 14 via the line-of-sight detection circuit 101 to extract the corneal reflection image and the boundary between the iris and the pupil. Is calculated (# 106). The flow of gaze information calculation will be described later.
[0033]
Further, the CPU 100 calculates the gazing point on the focus plate 7 based on the detected line-of-sight information of the photographer and the line-of-sight calibration data (# 107).
[0034]
Here, if the mode in which the photographer's line of sight information is used to control the automatic focus adjustment means is set, the CPU 100 selects the focus detection area closest to the gazing point coordinates (# 109).
[0035]
Subsequently, the CPU 100 transmits a signal to the focus detection circuit 103 to execute focus detection of the focus detection area selected based on the gaze information of the photographer (# 110). If the focus state of the selected focus detection area is not in focus (# 111), the CPU 100 transmits a focus adjustment signal to the lens drive circuit 110 to drive the taking lens 1 lens (# 120). When the lens drive corresponding to the focus adjustment signal is executed (# 120), focus detection is executed again (# 121).
[0036]
If the focus state of the selected focus detection area is in focus (# 111), focus display is performed in the viewfinder (# 112). Since the in-focus display is performed in the finder visual field, the photographer can recognize that the photographing lens is in focus in a desired focus detection area.
[0037]
If the mode in which the photographer's line of sight information is used to determine the exposure value is set (# 113), CPU 100 transmits a signal to photometric circuit 102 to perform photometry. At this time, the exposure value is determined by performing weighting based on the line of sight information of the photographer to the photometric value of the divided area in the photographing screen (# 114).
[0038]
If the switch SW-1 is continuously turned on (# 115), the state of the switch SW-2 of the shutter release button is confirmed (# 116). If the switch SW-2 is in the OFF state (# 116), the state of the switch SW-1 is confirmed again (# 115). If the switch SW-1 is off (# 115), the process returns to the initial state.
[0039]
When the switch SW-2 is turned on (# 116), the CPU 100 transmits a signal to the aperture driving circuit 107 to set the aperture 31 to a predetermined aperture, and transmits a signal to the shutter control circuit 108 to release the shutter. The curtain is run for shooting (# 117).
[0040]
When the shutter release operation of the camera ends (# 117), the process returns to the initial state.
[0041]
On the other hand, if the switch SW-1 is turned on by the photographer (# 102), the CPU 100 confirms the setting state of the mode selection switch detected earlier, and switches to the mode for controlling the control means using the photographer's line of sight information. If not set, the control means is independently controlled (# 103).
[0042]
The CPU 100 transmits a signal to the focus detection circuit 103 to execute focus detection for all focus detection areas (# 118). Further, the CPU 100 compares the focus detection information of all the detected focus detection areas, and determines the focus detection area having the shortest object distance as the area for performing the focus adjustment of the photographing lens (# 119).
[0043]
If the focus is determined with respect to the determined focus detection area (# 111), focus display is performed (# 112).
[0044]
The exposure value is also determined without using the photographer's line of sight information (# 113). The CPU 100 transmits a signal to the photometry circuit 102 to perform photometry. At this time, the exposure value is determined by performing an evaluation so that the area where the main subject is considered to be has an appropriate exposure amount (# 123).
[0045]
If the switch SW-1 is continuously turned on (# 115), the state of the switch SW-2 of the shutter release button is confirmed (# 116). If the switch SW-2 is in the OFF state (# 116), the state of the switch SW-1 is confirmed again (# 115). If the switch SW-1 is off (# 115), the process returns to the initial state.
[0046]
When the switch SW-2 is turned on (# 116), the CPU 100 transmits a signal to the aperture driving circuit 107 to set the aperture 31 to a predetermined aperture, and transmits a signal to the shutter control circuit 108 to release the shutter. The curtain is run for shooting (# 117).
[0047]
When the shutter release operation of the camera ends (# 117), the process returns to the initial state.
[0048]
FIG. 4 is a flowchart of the calculation of the line-of-sight information, which will be described below with reference to FIG.
[0049]
When the execution of the line-of-sight detection is instructed (# 106), the CPU 100 transmits a signal to the photometric circuit 102, which is a photometric means, reads out the signal of the photometric sensor 10, and measures the amount of light incident on the finder optical system. (# 130). Since the information of the pupil diameter of the user or the subject with respect to the photometric value of the finder optical system is stored in the EEPROM 100a as the storage means, the CPU 100 determines the user or the subject corresponding to the photometric value of the finder optical system measured from the EEPROM 100a. The pupil diameter information is read out and estimated as the pupil diameter r1 (# 131). At this time, the pupil diameter r1 becomes a small value if the photometric value of the finder optical system is bright, and the pupil diameter r1 becomes a large value if the photometric value of the finder optical system is dark.
[0050]
Further, the CPU 100 transmits a signal to the line-of-sight detection circuit 101 serving as a light receiving unit to cause the CCD 14 serving as an image sensor to accumulate for a predetermined time and output a signal (# 132). Since the pupil diameter information of the user or the subject with respect to the output of the image sensor 14 is stored in the EEPROM 100a serving as the storage means, the CPU 100 sets the user or the subject corresponding to the light output value of the image sensor 14 measured from the EEPROM 100a. Pupil diameter information is read out and estimated as pupil diameter r2 (# 133). At this time, the pupil diameter r2 has a small value if the anterior segment of the user or the subject is bright, and the pupil diameter r2 has a large value if the anterior segment of the user or the subject is dark. When measuring the brightness of the anterior segment of the user or the subject using the light receiving means, it is desirable not to illuminate the eyeball of the user or the subject with the lighting means.
[0051]
The CPU 100 compares the pupil diameter r1 estimated based on the amount of light incident on the finder optical system with the pupil diameter r2 estimated based on the brightness of the anterior segment of the user or the subject, and determines a smaller pupil diameter. Is the final set value (# 134). For example, when the user or the subject is observing from a dark indoor to a bright outdoor, r1 <r2, so that the pupil diameter is set to r1. Conversely, when observing from a bright outdoor place to a dark indoor place under sunlight, r1> r2, so that the pupil diameter is set to r2. As described above, by setting the smaller pupil diameter to the final set value, the estimated pupil diameter can be made closer to the actual pupil diameter. Therefore, unnecessary data can be accurately excluded in any case.
[0052]
When the pupil diameter of the user or the subject is set based on the amount of light incident on the finder optical system and the brightness of the anterior segment of the user or the subject (# 134), the CPU 100 causes the line-of-sight detection circuit 101, which is an illumination unit, to The signal is transmitted to illuminate the eyeball of the user or the subject by the IRED 13, and the reflected light from the eyeball is received by the image sensor 14 as the light receiving means (# 135). The CPU 100 performs A / D conversion on the eyeball image captured by the image sensor 14 via the line-of-sight detection circuit 101 to perform image processing, and extracts a corneal reflection image, a boundary between the pupil and the iris (# 136).
[0053]
FIG. 5 shows an eyeball image of the user or the subject formed on the image sensor 14 and an image processing result. In the figure, two black circles (•) are corneal reflection images. The corneal reflection image is detected almost accurately because the light intensity is high. In the figure, x and □ are extracted as boundaries between the pupil and the iris because of a large difference in light intensity.
[0054]
Further, the CPU 100 obtains the distance of the eye of the user or the subject to the light receiving means from the interval between the two corneal reflection images, and calculates the imaging magnification β of the eye of the user or the subject from the distance (# 137). Assuming that the interval between two corneal reflection images is ΔP, the imaging magnification β is
[0055]
[Outside 1]

It is expressed as Here, a, b, c, and d are constants determined by the setting state of the lighting means and the light receiving means.
[0056]
The pupil size on the image sensor 14 is obtained from the previously set pupil diameter r of the user or subject's eye and the imaging magnification β of the user or subject's eye. Further, the pupil existence range is set based on the positions of the two corneal reflection images and the size of the pupil on the image sensor 14 (# 138). In the eyeball image shown in FIG. 5, the dotted line indicates the existence range of the pupil.
[0057]
The CPU 100 eliminates false extraction data relating to the boundary between the pupil and the iris based on the pupil existence range (# 139). At this time, in FIG. 5, the data extracted at the boundary between the eyelid and the eyeball and the data extracted at the boundary between the sclera and the iris, which are indicated by crosses, are excluded. Further, CPU 100 calculates the center position of the pupil based on the remaining data extracted as the boundary between the pupil and the iris (the data indicated by the square in FIG. 5) (# 140).
[0058]
When the center position of the pupil is determined, the rotation angle of the user or the subject's eye is calculated from the difference between the position and the corneal reflection image (# 141).
[0059]
In this embodiment, it goes without saying that the setting of the pupil existence range based on the pupil diameter is set somewhat larger in consideration of the rotation of the eye of the user or the subject.
[0060]
Further, in the present embodiment, the method of setting the pupil existence range after extracting the corneal reflection image and the boundary between the pupil and the iris and excluding the false extraction data has been described. After the existence range of is set, the boundary between the pupil and the iris may be extracted.
[0061]
【The invention's effect】
As described above, according to the invention described in claim 1 of the present application, the light metering means for measuring the amount of light incident on the eyeball of the user or the subject, and the light receiving means for receiving an image of the user or the subject's eye A gaze detection device comprising: means for calculating gaze information of a user or a subject by arithmetically processing an eyeball image signal obtained by the light receiving means; Pupil diameter setting for setting the estimated pupil diameter of the user or the subject's eye using at least one of the pupil diameter of the user or the subject and the pupil diameter of the user or the subject estimated based on the output of the light receiving means. Means for calculating the pupil diameter by calculating the eyeball image signal based on the set estimated pupil diameter. Therefore, it is possible to more accurately eliminate unnecessary detection data when detecting the boundary between the pupil and the iris, and to accurately detect the pupil shape in a short time. High accuracy can be achieved.
[0062]
In addition, the eye-gaze detecting device according to the invention described in claim 5 of the present application includes a light-receiving unit having a light-receiving surface for receiving an eyeball image of the user or the subject, and a light amount substantially incident on the eyeball of the user or the subject. Photometric means for photometric measurement; pupil diameter estimating means for estimating the pupil diameter of the user or subject based on the output of the photometric means; and distance information between the eyeball of the user or subject and the light receiving surface of the light receiving means. Distance information detecting means, and the user or subject forming an image on the light receiving surface of the light receiving means from the pupil diameter of the user or the subject estimated by the pupil diameter estimating means and the distance information obtained by the distance information detecting means A pupil diameter calculating means for calculating the pupil diameter of the eye, and calculating the eyeball image information obtained by the light receiving means based on the calculation result of the pupil diameter calculating means, so that the user or the subject can see Is characterized by having an arithmetic processing means for obtaining information, by such a configuration, it is possible to obtain the estimated pupil diameter on the light receiving surface of the light receiving means, more precisely it is possible to estimate the pupil diameter. Therefore, it is possible to more accurately eliminate unnecessary detection data when detecting the boundary between the pupil and the iris, and to accurately detect the pupil shape in a short time. High accuracy can be achieved.
[Brief description of the drawings]
FIG. 1 is a schematic diagram of a main part of a single-lens reflex camera. FIG. 2 is a block diagram of an electric circuit of the camera. FIG. 3 is an explanatory diagram of an operation flow of the camera. FIG. Explanation of code]
REFERENCE SIGNS LIST 1 photographing lens 2 main mirror 3 sub-mirror 4 shutter 5 film 6 focus detection device 7 focus plate 8 penta roof prism 9 imaging lens 10 photometric sensor 11 eyepiece lens 12 light receiving lens 13 infrared light emitting diode 14 CCD
15 Eyeball 32 Aperture drive device 100 CPU
101 eye-gaze detection circuit 102 photometry circuit 103 focus detection circuit 104 signal input circuit 107 aperture drive circuit 108 shutter control circuit 110 lens drive circuit

Claims

A photometric unit for measuring the amount of light incident on the eyeball of the user or the subject, a light receiving unit for receiving an image of the user or the subject's eye, and an eyeball image signal obtained by the light receiving unit being processed and used A gaze detecting device having an arithmetic processing unit for obtaining gaze information of a user or a subject, wherein the user or the pupil diameter of the subject estimated based on the output of the photometric unit and the user estimated based on the output of the light receiving unit Or comparing the pupil diameter of the subject and having an estimated pupil diameter setting means for setting a smaller pupil diameter as the estimated pupil diameter of the user or the subject's eye, wherein the arithmetic processing means is based on the set estimated pupil diameter. A gaze detection device for performing arithmetic processing for eliminating false extracted data from the eyeball image signal obtained by the light receiving means.

A photometer that substantially measures the amount of light incident on the eyeball of the user or the subject; a first pupil diameter estimator that estimates a pupil diameter of the user or the subject based on the output of the photometer; Light receiving means for receiving the eyeball image of the subject, second pupil diameter estimating means for estimating the pupil diameter of the user or the subject based on the output of the light receiving means, and estimation results of the first pupil diameter estimating means Setting means for comparing the estimation result of the second pupil diameter estimating means and setting a smaller pupil diameter as the estimated pupil diameter of the user or the subject; and setting the pupil diameter based on the estimated pupil diameter set by the setting means. A gaze detecting device, comprising: arithmetic processing means for performing arithmetic processing for eliminating false extraction data from eyeball image information obtained by the light receiving means to obtain gaze information of a user or a subject.

A light-receiving means having a light-receiving surface for receiving an eyeball image of the user or the subject, a photometric means for substantially measuring the amount of light incident on the eyeball of the user or the subject, and the user or the subject based on the output of the photometric means Pupil diameter estimating means for estimating the pupil diameter of the user, distance information detecting means for obtaining distance information between the eyeball of the user or the subject and the light receiving surface of the light receiving means, and the user or the subject estimated by the pupil diameter estimating means A pupil diameter calculating means for calculating a pupil diameter of a user or a subject forming an image on a light receiving surface of the light receiving means from the pupil diameter and the distance information obtained by the distance information detecting means; and a calculation result of the pupil diameter calculating means. Calculation processing means for obtaining gaze information of the user or the subject by performing calculation processing for eliminating false extraction data from the eyeball image information obtained by the light receiving means. Line-of-sight detection apparatus according to claim.

A light receiving means having a light receiving surface for receiving an eyeball image of the user or the subject, a pupil diameter estimating means for estimating a pupil diameter of the user or the subject based on an output of the light receiving means, and an eyeball of the user or the subject Distance information detecting means for obtaining distance information from the light receiving surface of the means, and a light receiving surface of the light receiving means from the pupil diameter of the user or the subject estimated by the pupil diameter estimating means and the distance information obtained by the distance information detecting means. A pupil diameter calculating means for calculating a pupil diameter of a user or a subject who forms an image on a subject; and eliminating false extraction data from eyeball image information obtained by the light receiving means based on a calculation result of the pupil diameter calculating means. A gaze detecting device comprising: an arithmetic processing unit that obtains gaze information of a user or a subject by performing arithmetic processing.

A light receiving unit having a light receiving surface for receiving an eyeball image of the user or the subject; a distance information detecting unit for obtaining distance information between the user or the subject's eyeball and the light receiving surface of the light receiving unit; Photometric means for measuring the amount of light incident on the eyeball; first pupil diameter estimating means for estimating the pupil diameter of the user or subject based on the output of the photometric means; and The second pupil diameter estimating means for estimating the pupil diameter of the subject is compared with the estimation result of the first pupil diameter estimating means and the estimation result of the second pupil diameter estimating means, and a smaller pupil diameter is used. Pupil diameter setting means for setting as an estimated pupil diameter of a user or a subject, and a pupil for calculating a pupil diameter of a user or a subject to be imaged on a light receiving surface of the light receiving means based on the distance information and the estimated pupil diameter. Diameter calculation means Arithmetic processing means for performing an arithmetic processing for eliminating false extraction data based on the output of the pupil diameter calculating means on the eyeball image information obtained by the light receiving means, thereby obtaining visual line information of a user or a subject. Eye-gaze detecting device.

The distance information detecting means has an illuminating means for illuminating a user or a subject's eye with a pair of parallel light beams, and the user or the subject is obtained from a pair of reflected images of the illuminating means generated in an eyeball image received by the light receiving means. 6. The eye-gaze detecting device according to claim 3 , wherein distance information between the eyeball and the light receiving means is obtained.

The pupil diameter calculating means calculates an imaging magnification of an eyeball of the user or the subject that forms an image on the light receiving surface of the light receiving means from the distance information, and calculates a pupil diameter of the eyeball that forms an image on the light receiving surface of the light receiving means. 7. The gaze detection device according to claim 6 , wherein the gaze is calculated.

A photometric unit for measuring the amount of light incident on the eyeball of the user or the subject, a light receiving unit for receiving an image of the user or the subject's eye, and an eyeball image signal obtained by the light receiving unit being processed and used A gaze detecting device having an arithmetic processing unit for obtaining gaze information of a user or a subject, wherein the user or the pupil diameter of the subject estimated based on the output of the photometric unit and the user estimated based on the output of the light receiving unit Or comparing the pupil diameter of the subject and having an estimated pupil diameter setting means for setting a smaller pupil diameter as the estimated pupil diameter of the user or the subject's eye, wherein the arithmetic processing means is based on the set estimated pupil diameter. An optical apparatus having an eye-gaze detecting device for performing a calculation process for eliminating false extraction data from an eyeball image signal obtained by the light-receiving means.

A photometer that substantially measures the amount of light incident on the eyeball of the user or the subject; a first pupil diameter estimator that estimates a pupil diameter of the user or the subject based on the output of the photometer; Light receiving means for receiving the eyeball image of the subject, second pupil diameter estimating means for estimating the pupil diameter of the user or the subject based on the output of the light receiving means, and estimation results of the first pupil diameter estimating means Setting means for comparing the estimation result of the second pupil diameter estimating means and setting a smaller pupil diameter as the estimated pupil diameter of the user or the subject; and setting the pupil diameter based on the estimated pupil diameter set by the setting means. A gaze detecting device for calculating gaze information of a user or a subject by performing arithmetic processing for eliminating false extraction data from eyeball image information obtained by the light receiving means. Manabu equipment.

A light-receiving means having a light-receiving surface for receiving an eyeball image of the user or the subject, a photometric means for substantially measuring the amount of light incident on the eyeball of the user or the subject, and the user or the subject based on the output of the photometric means Pupil diameter estimating means for estimating the pupil diameter of the user, distance information detecting means for obtaining distance information between the eyeball of the user or the subject and the light receiving surface of the light receiving means, and the user or the subject estimated by the pupil diameter estimating means A pupil diameter calculating means for calculating a pupil diameter of a user or a subject forming an image on a light receiving surface of the light receiving means from the pupil diameter and the distance information obtained by the distance information detecting means; and a calculation result of the pupil diameter calculating means. Calculation processing means for obtaining gaze information of the user or the subject by performing calculation processing for eliminating false extraction data from the eyeball image information obtained by the light receiving means. Optical apparatus having a sight line detecting device according to claim.

A light receiving means having a light receiving surface for receiving an eyeball image of the user or the subject, a pupil diameter estimating means for estimating a pupil diameter of the user or the subject based on an output of the light receiving means, and an eyeball of the user or the subject Distance information detecting means for obtaining distance information from the light receiving surface of the means, and a light receiving surface of the light receiving means from the pupil diameter of the user or the subject estimated by the pupil diameter estimating means and the distance information obtained by the distance information detecting means. A pupil diameter calculating means for calculating a pupil diameter of a user or a subject who forms an image on a subject; and eliminating false extraction data from eyeball image information obtained by the light receiving means based on a calculation result of the pupil diameter calculating means. An optical device having a line-of-sight detection device, comprising: a processing unit for calculating line-of-sight information of a user or a subject by performing a calculation process.

A light receiving unit having a light receiving surface for receiving an eyeball image of the user or the subject; a distance information detecting unit for obtaining distance information between the user or the subject's eyeball and the light receiving surface of the light receiving unit; Photometric means for measuring the amount of light incident on the eyeball; first pupil diameter estimating means for estimating the pupil diameter of the user or subject based on the output of the photometric means; and The second pupil diameter estimating means for estimating the pupil diameter of the subject is compared with the estimation result of the first pupil diameter estimating means and the estimation result of the second pupil diameter estimating means, and a smaller pupil diameter is used. Pupil diameter setting means for setting as an estimated pupil diameter of a user or a subject, and a pupil for calculating a pupil diameter of a user or a subject to be imaged on a light receiving surface of the light receiving means based on the distance information and the estimated pupil diameter. Diameter calculation means Arithmetic processing means for performing an arithmetic processing for eliminating false extraction data based on the output of the pupil diameter calculating means on the eyeball image information obtained by the light receiving means, thereby obtaining visual line information of a user or a subject. Optical device having a line-of-sight detection device.