JP4107469B2 - Method for generating 3D shape data - Google Patents

Description

・・・数式１
この数式１により算出された高さｈ、即ち、Ｚ座標値を、メッシュ点ＰのＺ座標値に加算する。
【０１３７】
さらに、同様の処理を鼻外郭形状４００に対応する３次元メッシュの領域内の全てのメッシュ点について繰り返し行う。その結果、鼻外郭形状４００の上端点ＮＵ，下端点ＮＤ，左端点ＮＬならびに右端点ＮＲからなる四角形で取り囲まれた２次元画像の内部領域に対応する３次元メッシュの領域は、鼻の盛り上がりに対応するような形状となる（図２０参照）。
【０１３８】
ステップＳ１６１６の処理が終了すると、ステップＳ１６１８に進み、３次元形状の楕円の盛り上げ処理を行う。この楕円の盛り上げ処理は、３次元形状において顔面全体の丸みを表現するための処理であり、上記した（１−２）３次元化領域設定処理によって設定された楕円３００（図４（ｃ）参照）を用いる。
【０１３９】
具体的には、楕円３００で取り囲まれた２次元画像の内部領域に対応する３次元メッシュの領域を処理対象とする。
【０１４０】
そして、楕円３００に対応する３次元メッシュの領域内の全てのメッシュ点それぞれのＺ座標値に、楕円３００の中心（楕円長軸３００ａ上の点）を頂上とした楕円球を想定して求められる高さ情報、即ち、Ｚ座標値を加算する。
【０１４１】
ステップＳ１６１８の処理が終了すると、ステップＳ１６２０に進み、３次元形状データを対象にフィレット処理を行う。このフィレット処理は、３次元形状の人間の顔面を自然に仕上げるための処理であり、人間の顔面にはありえないような垂直面の角を丸くする（図２１（ａ）（ｂ）参照）。
【０１４２】
具体的には、３次元形状データの３次元メッシュのメッシュ点毎に順次、漏斗様のフィレットメッシュ（図２２（ａ）参照）の中心Ｔを一致させる。そして、３次元メッシュのメッシュ点の高さ情報（Ｚ座標値）が、対応するフィレットメッシュのフィレットメッシュ点の高さ情報よりも高い場合には、その３次元メッシュのメッシュ点のＺ座標値をフィレットメッシュの高さ情報に合わせて、３次元メッシュを変形させる。
【０１４３】
なお、このフィレット処理の開始に際しては、ステップＳ１６０２乃至ステップＳ１６１８の処理によって生成された３次元メッシュのコピー処理を行い、変換元３次元メッシュと変換先３次元メッシュとの２つの３次元メッシュを用意する。つまり、変換元３次元メッシュの高さ情報は変換先３次元メッシュにコピーされており、変換元３次元メッシュと変換先３次元メッシュとは、フィレット処理の開始前は同一なものである。
【０１４４】
また、図２２乃至図２６においては、説明を容易にするために、３次元メッシュのメッシュ点やフィレットメッシュのメッシュ点をバンド状に図示することとする。
【０１４５】
そして、フィレット処理に用いられるフィレットメッシュは、Ｘ軸方向におけるサイズとＹ軸方向におけるサイズとが等しい正方形形状の底面を有している（図２２（ｂ）参照）。なお、フィレットメッシュの半径をＲとした場合に、フィレットメッシュのＸ軸方向およびＹ軸方向におけるサイズＬはＲ×２＋１となされている。
【０１４６】
こうしたフィレットメッシュのフィレットメッシュ点それぞれについて、フィレットメッシュの中心Ｔとの距離ｒを求める（図２３（ａ）（ｂ）参照）。そして、求められた距離ｒがフィレットメッシュの半径Ｒに比べて短いフィレットメッシュ点については、当該フィレットメッシュ点の高さ情報として下記数式２により求められるＺ座標値Ｌｒを設定する。
【０１４７】
【数２】

・・・数式２
一方、求められた距離ｒがフィレットメッシュの半径Ｒに比べて短いフィレットメッシュ点以外のフィレットメッシュ点については、当該フィレットメッシュ点の高さ情報として最高値Ｌｍａｘ（例えば、２５５０００）のＺ座標値を設定する。
【０１４８】
こうして設定されたＺ座標値を有するフィレットメッシュ点からなるフィレットメッシュは、図２２（ａ）に示すような漏斗様体である。そして、このフィレットメッシュの中心Ｔを、変換元３次元メッシュの所定のメッシュ点に一致させ、３次元メッシュのメッシュ点の高さ情報ｂａｓｅＬを取得する（図２４参照）。この高さ情報ｂａｓｅＬは、フィレットメッシュの中心Ｔが一致する変換元３次元メッシュのメッシュ点のＺ座標値に等しいものである。
【０１４９】
それから、フィレットメッシュの全てのフィレットメッシュ点について、フィレットメッシュ点のＺ座標値ｍＬと、当該Ｚ座標値ｍＬのフィレットメッシュ点に対応する変換元３次元メッシュのメッシュ点の高さｓｒｃＬとを求める（図２５（ａ）参照）。
【０１５０】
そして、求められた変換元３次元メッシュの高さｓｒｃＬと変換元３次元メッシュの高さ情報ｂａｓｅＬとを比較し、当該比較処理の結果、高さｓｒｃＬが高さ情報ｂａｓｅＬに比べて大きい（高い）場合には、処理対象の変換元３次元メッシュのメッシュ点に対応するフィレットメッシュの高さｔａｒｇｅｔＬを
ｔａｒｇｅｔＬ＝（ｓｒｃＬ−ｂａｓｅＬ）×ｍＬ／Ｌｍａｘ＋ｂａｓｅＬとする（図２８（ｂ）参照）。
【０１５１】
この後、処理対象の変換元３次元メッシュのメッシュ点でのフィレットメッシュの高さｔａｒｇｅｔＬと、処理対象の変換元３次元メッシュのメッシュ点に対応する変換先３次元メッシュの高さ情報とを比較し、高さｔａｒｇｅｔＬが変換先３次元メッシュの高さ情報に比べて小さい（低い）場合には、変換先３次元メッシュの高さ情報、即ち、変換先３次元メッシュのメッシュ点のＺ座標値を高さｔａｒｇｅｔＬに置き換える（図２６（ａ）参照）。
【０１５２】
そして、こうした処理をフィレットメッシュの全てのフィレットメッシュ点について行うことにより、変換先３次元メッシュの垂直面の角は丸いフィレット面となる（図２６（ｂ）参照）。
【０１５３】
ステップＳ１６２０の処理に続き、ステップＳ１６２２においては、３次元形状データを対象に孤立領域除去処理を行う。この孤立領域除去処理は、３次元形状に孤立した領域の存在を防止するための処理であり、余分な３次元形状メッシュのメッシュ点を削除する。
【０１５４】
３次元形状データの３次元メッシュのメッシュ点の所定のメッシュ点について、当該所定のメッシュ点の周囲８方向に位置するメッシュ点を対象にし、この周囲に有効なデータ、即ち、所定のメッシュ点の周囲に位置する８つのメッシュ点のうち、Ｚ座標値が「０」ではないメッシュ点が１つ以下の場合には、そのメッシュ点のＺ座標値を「０」にする。こうした処理を３次元メッシュの全てのメッシュ点について繰り返し行う。
【０１５５】
こうして孤立領域除去処理が３次元メッシュの全てのメッシュ点について行われると、ステップＳ１６２２の処理を終了して、この３次元形状データの生成処理ルーチンを終了する。
【０１５６】
なお、３次元形状データの生成処理ルーチンが終了した後に、作業者がポインティング・デバイスや文字入力デバイスからなる入力装置１８によって、表示装置２０のユーザー設定画面１００のプレビュー欄１００ｍを選択すると、３次元形状データの生成処理ルーチンよって生成された３次元形状データの示す３次元形状が画像エリア１００ａに表示される。
【０１５７】
上記したようにして、本発明による３次元形状データの生成方法は、２次元画像データから３次元形状データが生成されるので、３次元スキャナーは必要なくなり、実際に３次元形状を備えた対象物を測定する制約なしに安価に３次元形状データを生成することができる。
【０１５８】
また、本発明による３次元形状データの生成方法は、以下に示す（Ｉ）〜（ＶＩＩ）により、２次元画像データから高品質な３次元形状を示す３次元形状データ、特に、高品質な人間の頭部の顔面側を示す３次元形状データを生成することができる。
【０１５９】
（Ｉ）２次元画像データの示す２次元画像２００の人間の顔面２００ａにおいて設定された右眼の位置と左眼の位置とに基づいて、上記した（１−４）鼻外郭設定処理（図６参照）によって鼻の部位２００ｅの外郭形状が設定される。そして、設定された鼻外郭形状４００に基づいて、鼻形状作成処理（ステップＳ１６１６）により鼻外郭形状４００に対応する３次元メッシュの領域内の全てのメッシュ点のＺ座標値に所定の値が加算される（図２０参照）。その結果、生成された人間の頭部の顔面側の３次元形状において、鼻の盛り上がりが表現される。
【０１６０】
つまり、上記した「従来の技術」の項において記載したようにして、２次元画像データの輝度情報を３次元形状データの高さ情報として変換するときに、人間の頭部の顔面側の３次元形状において鼻の部位が突出せず不自然になるようなことがない。
【０１６１】
（ＩＩ）３次元形状データの生成処理ルーチンの左右輝度バランス処理（ステップＳ１６０４）により、２次元画像データが最適化され、２次元画像の顔面の左側領域の輝度と右側領域の輝度のバランスを補正されるようにしたので、生成される３次元形状の顔面の左側と右側とバランスが崩れるのを防ぐことができる。
【０１６２】
つまり、上記した「従来の技術」の項において記載したように、２次元画像の人間の顔面に所定の方向から光が照射された状態であって、例えば、人間の顔面の右側が左側に比べて明るい領域となっている場合においても、生成された３次元形状データの示す人間の顔面における右側の高さが、左側の高さに比べて高くなってしまい、２次元画像データを取得した際の照明の影響によって人間の顔面の３次元形状が不自然なものとなるようなことがない。
【０１６３】
（ＩＩＩ）３次元形状データの生成処理ルーチンのグレー変換処理（ステップＳ１６０８）において、人間の頭部の顔面側の２次元画像２００（図３参照）に適した比率、具体的には、およそＲ：Ｇ：Ｂ＝７：２：１の比率で、２次元画像データからグレー変換情報を得て、当該グレー変換情報を３次元形状データの高さ情報に変換している。このため、輝度情報を３次元形状データの高さ情報に変換する場合に比べて、２次元画像の顔面と３次元形状の顔面との対応が良くなり、３次元形状の顔面の凹凸を一層自然なものとすることができる。
【０１６４】
（ＩＶ）３次元形状データの生成処理ルーチンのハイライト除去処理（ステップＳ１６１０）において、３次元メッシュの高さ情報の平均値と標準偏差とに基づいて、３次元形状データの高さ情報、即ち、Ｚ座標値に制限がかけられるので、生成された３次元形状データの示す人間の頭部の顔面側の３次元形状において、急峻な凸部が形成されるのを防ぐことができる。
【０１６５】
（Ｖ）上記した（１−６）反転禁止領域設定処理によって白黒反転を行わない範囲が設定される。そして、反転禁止マスクとして設定された白黒反転を行わない範囲を除いた部分を処理対象として、具体的には、人間の頭部の髪２００ｂの部分のみを対象に、白黒反転処理（ステップＳ１６１２）が行われるので、髪の毛など本来盛り上がっている部分が窪んでしまうのを防ぐことができる。
【０１６６】
つまり、上記した「従来の技術」の項において記載したように、２次元画像の人間の顔面における眼の部位が、虹彩の色が黒などで暗い領域となっているときに、人間の顔面全体を対象に白黒の反転処理が行われて、生成された３次元形状データの示す人間の顔面における眼の部位の高さが、眼の部位の周囲に比べて高くなってしまい、人間の顔面の３次元形状において眼の部位が突出してしまって不自然なものとなるようなことがない。
【０１６７】
（ＶＩ）上記した（１−２）３次元化領域設定処理によって、２次元画像２００の人間の顔面２００ａを取り囲むようにして楕円３００が設定される。そして、設定された楕円３００の内部領域たる３次元化領域を対象に、楕円の盛り上げ処理（ステップＳ１６１８）が行われるので、生成される３次元形状において顔面側全体の丸みが表現される。
【０１６８】
（ＶＩＩ）３次元形状データの生成処理ルーチンのフィレット処理（ステップＳ１６２０）において、フィレットメッシュによって３次元メッシュを変形させるので、３次元形状の垂直面の角が丸くなり、人間の顔面にはありえないような垂直面はなくなり、３次元形状の人間の顔面が自然に仕上がる。
【０１６９】
また、本発明による３次元形状データの生成方法によって生成された３次元形状データを、例えば、３次元切削加工装置に用いた場合には、３次元形状データの示す３次元形状が高品質なので、当該３次元形状データに基づいて被加工物から削り出される３次元形状を高品質なものとすることができる。
【０１７０】
なお、上記した実施の形態は、以下の（１）乃至（５）に説明するように変形することができる。
【０１７１】
（１）上記した実施の形態においては、本発明による３次元形状データの生成方法により生成された３次元形状データが３次元形状データ記憶部１６−２に記憶されるようにしたが、これに限られるものではないことは勿論であり、本発明による３次元形状データの生成方法により生成された３次元形状データを、３次元切削加工の駆動装置に出力するようにしたり、外部記憶装置２２に記憶するようにしてもよく、さらに、外部記憶装置２２に記憶された３次元データをＣＡＤシステムなどで用いるようにしてもよい。
【０１７２】
（２）上記した実施の形態においては、カラーの２次元画像データから３次元形状データを生成するようにしたが、これに限られるものではないことは勿論であり、例えば、グレースケールの２次元画像データから３次元形状データを生成するときに、本発明による３次元形状データの生成方法を用いるようにしてもよい。なお、２次元画像データの種類により、ステップＳ１６０８の処理においてグレー変換情報を得ることができな場合には、３次元形状の品質が低下する恐れが生じるが、グレー変換情報に代えて輝度情報を３次元形状データの高さ情報に変換するようにしてもよい。
【０１７３】
（３）本発明による３次元形状データの生成方法においては、本発明による３次元形状データの生成方法により生成された３次元形状データが示す３次元形状は３次元メッシュのみならず、ポリゴンやあるいは曲面に適用することもできる。
【０１７４】
（４）本発明による３次元形状データの生成処理の３次元化準備処理の段階における各処理、あるいは、３次元化実行処理の段階における各種処理はそれぞれ、適宜処理の順序を入れ換えたり、あるいは、除いてもよい。例えば、３次元化実行処理の段階において、３次元形状データの生成処理ルーチンの白黒反転処理（ステップＳ１６１２）と背景削除処理（ステップＳ１６１４）との順序を入れ換えてもよいし、あるいは、背景削除処理（ステップＳ１６１４）を行わなくてもよい。
【０１７５】
（５）上記した実施の形態ならびに上記（１）乃至（４）に示す変形例は、適宜に組み合わせるようにしてもよい。
【０１７６】
【発明の効果】
本発明は、以上説明したように構成されているので、高品質な３次元形状を示す３次元形状データを安価に生成することができるという優れた効果を奏する。
【図面の簡単な説明】
【図１】本発明による３次元形状データの生成方法の実施の形態の一例を実施した３次元形状データ生成装置のブロック図である。
【図２】表示装置の画面上に表示されるユーザー設定画面を示す説明図である。
【図３】本発明による３次元形状データの生成方法に用いられる２次元画像データの示す２次元画像の一例を模式的に示す説明図である。
【図４】本発明による３次元形状データの生成方法の３次元化領域設定処理のときのユーザー設定画面の画像エリアの表示内容を示す説明図であり、（ａ）は点Ｅ１ならびに点Ｅ２が指定された状態を示す説明図であり、（ｂ）は楕円が生成された状態を示す説明図であり、（ｃ）は（ｂ）に示す楕円が変形された状態を示す説明図である。
【図５】本発明による３次元形状データの生成方法の両眼位置設定処理のときのユーザー設定画面の画像エリアの表示内容を示す説明図であり、（ａ）は右眼の位置を示す点ＲＥが表示された状態を示す説明図であり、（ｂ）は左眼の位置を示す点ＬＥが表示された状態を示す説明図である。
【図６】本発明による３次元形状データの生成方法の鼻外郭設定処理のときのユーザー設定画面の画像エリアの表示内容を示す説明図である。
【図７】鼻外郭形状の生成処理ルーチンのフローチャートである。
【図８】鼻外郭形状の生成処理ルーチンの鼻の上端点ＮＵの生成処理を示す説明図である。
【図９】（ａ）（ｂ）は、鼻外郭形状の生成処理ルーチンの鼻の下端点ＮＤの生成処理を示す説明図である。
【図１０】鼻外郭形状の生成処理ルーチンの鼻の右端点ＮＲならびに左端点ＮＬの生成処理を示す説明図である。
【図１１】（ａ）（ｂ）は、鼻外郭形状の生成処理ルーチンの鼻の中心点ＮＣの生成処理を示す説明図である。
【図１２】鼻外郭形状の生成処理ルーチンによって生成された鼻外郭形状を示す説明図である。
【図１３】本発明による３次元形状データの生成方法の鼻外郭設定処理のときのユーザー設定画面の画像エリアの表示内容を示す説明図である。
【図１４】本発明による３次元形状データの生成方法の背景領域設定処理のときのユーザー設定画面の画像エリアの表示内容を示す説明図である。
【図１５】本発明による３次元形状データの生成方法の反転禁止領域設定処理のときのユーザー設定画面の画像エリアの表示内容を示す説明図である。
【図１６】３次元形状データの生成処理ルーチンのフローチャートである。
【図１７】３次元形状データの生成処理ルーチンの鼻形状作成処理を概念的に示す説明図である。
【図１８】３次元形状データの生成処理ルーチンの鼻形状作成処理の内容をエリアＡを中心にして概念的に示す説明図である。
【図１９】３次元形状データの生成処理ルーチンの鼻形状作成処理のときのメッシュ点Ｐの高さｈを求める数式１に対応する内容を示す説明図である。
【図２０】３次元形状データの生成処理ルーチンの鼻形状作成処理後の３次元メッシュの一例を示す説明図である。
【図２１】（ａ）（ｂ）は、３次元形状データの生成処理ルーチンのフィレット処理を示す説明図であり、（ａ）はフィレット処理前の３次元メッシュを示す説明図であり、（ｂ）はフィレット処理後の３次元メッシュを示す説明図である。
【図２２】（ａ）（ｂ）は、３次元形状データの生成処理ルーチンのフィレット処理に用いられるフィレットメッシュを示す説明図である。
【図２３】（ａ）（ｂ）は、３次元形状データの生成処理ルーチンのフィレット処理に用いられるフィレットメッシュを示す説明図である。
【図２４】３次元形状データの生成処理ルーチンのフィレット処理を概念的に示す説明図である。
【図２５】（ａ）（ｂ）は、３次元形状データの生成処理ルーチンのフィレット処理を概念的に示す説明図である。
【図２６】（ａ）（ｂ）は、３次元形状データの生成処理ルーチンのフィレット処理を概念的に示す説明図である。
【符号の説明】
１０ 中央制御装置（ＣＰＵ）
１４ リード・オンリ・メモリ（ＲＯＭ）
１６ ランダム・アクセス・メモリ（ＲＡＭ）
１６−１ ２次元画像データ記憶部
１６−２ ３次元形状データ記憶部
１８ 入力装置
２０ 表示装置
２２ 外部記憶装置
１００ ユーザー設定画面
１００ａ 画像エリア
１００ｂ ２次元画像データ欄
１００ｃ モード選択エリア
１００ｄ 楕円設定モード欄
１００ｅ 右眼設定モード欄
１００ｆ 左眼設定モード欄
１００ｇ 鼻外郭設定モード欄
１００ｈ 領域設定モード欄
１００ｉ 背景マスク欄
１００ｊ 反転禁止マスク欄
１００ｋ ３次元化開始欄
１００ｍ プレビュー欄
２００ ２次元画像
２００ａ 顔面
２００ｂ 髪
２００ｃ 左眼の部位
２００ｄ 右眼の部位
２００ｅ 鼻の部位
３００ 楕円
３００ａ 楕円長軸
４００，４００’鼻外郭形状[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a method for generating three-dimensional shape data, and more particularly, to a method for generating three-dimensional shape data for automatically generating three-dimensional shape data indicating a three-dimensional shape on the face side of a human head, In particular, the present invention relates to a method for generating three-dimensional shape data for generating three-dimensional shape data suitable as three-dimensional shape data used in a three-dimensional shape processing device by numerical control such as a three-dimensional cutting device.
[0002]
[Prior art]
Conventionally, a method of generating three-dimensional shape data indicating a predetermined object having a three-dimensional shape using a three-dimensional scanner is known.
[0003]
More specifically, in a method using a three-dimensional scanner, for example, a human head, particularly a face, as an object is measured by scanning with an optical means of a three-dimensional scanner, and based on the measurement result, Three-dimensional shape data indicating the three-dimensional shape of the human face is generated.
[0004]
However, in the method of generating three-dimensional shape data using such a conventional three-dimensional scanner, there is a restriction that an object such as a human face must be actually measured, and the three-dimensional scanner is expensive. There is a problem that the system for generating the dimensional shape data becomes expensive.
[0005]
There is also known a method of generating 3D shape data from the 2D image data using 2D image data indicating a 2D image of a predetermined object without using a 3D scanner.
[0006]
More specifically, in the method using two-dimensional image data, for example, red (R) component data of each pixel of the two-dimensional image data obtained by photographing a human face with a digital camera, green ( G) component data and blue (B) component data are processed at a ratio of approximately R: G: B = 3: 5: 2 to obtain luminance information indicating the brightness of each pixel. Then, using the luminance information of the two-dimensional image data as the height information of the three-dimensional shape data, three-dimensional shape data indicating the three-dimensional shape of the human face is generated.
[0007]
As a result, the height of the three-dimensional shape indicated by the three-dimensional shape data is increased corresponding to the bright region of the two-dimensional image indicated by the two-dimensional image data (that is, the region having a high luminance information level). The three-dimensional shape has a lower height corresponding to a dark area (that is, an area where the level of luminance information is low).
[0008]
At this time, black and white inversion processing is performed according to the type of the two-dimensional image indicated by the two-dimensional image data. For example, when the 2D image is a human head, if the hair is dark and dark, the height of the corresponding portion of the 3D shape indicated by the generated 3D shape data is low. Since it is unnatural, the two-dimensional image data subjected to the black and white inversion processing and the original two-dimensional image data are synthesized. Then, the luminance information of the synthesized two-dimensional image data composed of data having higher luminance is converted as height information of the three-dimensional shape data.
[0009]
However, in such a conventional method for generating three-dimensional shape data using two-dimensional image data, if the two-dimensional image indicated by the two-dimensional image data is on the face side of a person, particularly a human head, it is generated. There is a problem in that the three-dimensional shape indicated by the three-dimensional shape data thus made does not correspond to the unevenness of the actual human face and becomes unnatural.
[0010]
Specifically, the nose portion of the human face of the two-dimensional image is not particularly bright compared to other areas of the face, so that the luminance information of the two-dimensional image data is used as the height information of the three-dimensional shape data. Even after conversion, the height of the nose portion on the human face indicated by the generated three-dimensional shape data is substantially equal to the periphery of the nose portion, and the nose portion protrudes in the three-dimensional shape of the human face. It was unnatural.
[0011]
In addition, when the eye part on the human face of the two-dimensional image is a dark region with an iris color of black or the like, the generated three-dimensional shape data indicates when black and white inversion processing is performed. The height of the part of the eye on the human face is higher than the surroundings, and the part of the eye protrudes in the three-dimensional shape of the human face, which is unnatural.
[0012]
Further, when the human face of the two-dimensional image is irradiated with light from a predetermined direction and the right side of the human face is a brighter area than the left side, for example, the generated 3 The height on the right side of the human face indicated by the three-dimensional shape data is higher than the height on the left side, and the three-dimensional shape of the human face is unnatural due to the influence of illumination when acquiring the two-dimensional image data. It was something.
[0013]
[Problems to be solved by the invention]
The present invention has been made in view of the problems of the conventional techniques as described above, and an object of the present invention is to generate three-dimensional shape data indicating a high-quality three-dimensional shape at low cost. An object of the present invention is to provide a method for generating three-dimensional shape data that can be made.
[0014]
[Means for Solving the Problems]
  In order to achieve the above object, the invention according to claim 1 of the present invention comprises a specifying means, a generating means,Reversal prohibition area setting means,First addition means, correction means,Smoothing processing means;Gray conversion means, limiting means,Black-and-white reversing means, forming means,Brightness information obtained from two-dimensional image data composed of pixel data corresponding to a predetermined number of pixels, and having a plurality of XYZ coordinate values in an XYZ coordinate system. 3D for generating 3D shape data indicating the 3D shape of a human head using height information indicated by the Z coordinate value of 3D shape data composed of mesh pointsshapeA method for generating three-dimensional shape data in a data generation device, wherein the specifying means,Position of the eye part on the human face indicated by the two-dimensional image dataofSpecifiedReceive inputThe first step and the generating means,A second step of generating a region of the nose portion of the human face indicated by the two-dimensional image data based on the position of the eye portion designated in the first step;The inversion prohibition area setting means receives a designation input of a point other than the hair of the human head indicated by the two-dimensional image data, so that the color similar to the color of the point having received the designation input in the area other than the hair A third step of setting the color area as a region where black and white inversion is not performed;The first adding means isAdding a predetermined luminance value according to the distance from the eye part to the predetermined luminance value of the eye part specified in the first step indicated by the luminance information of the two-dimensional image data; A fourth step of changing the luminance information of the two-dimensional image data to luminance information indicating the luminance value of the addition result;The correction means is,The left eye is the center of the eye part specified in the first step.PredeterminedLeft area2D image data inofAverage brightness value indicated by brightness informationAnd focusing on the right eyePredeterminedRight area2D image data inofA difference between the average luminance value indicated by the luminance information is obtained, and based on the difference, the luminance information of the two-dimensional image data is changed by correcting the balance between the luminance information of the left region and the luminance information of the right region of the face. 5, a sixth step in which the smoothing processing means performs a smoothing process on the two-dimensional image data, a gray conversion means in the fourth step, the fifth step, and the above The red component data, the green component data, and the blue component data constituting the pixel data of the two-dimensional image data processed in the sixth step are weighted and added at a predetermined ratio and gray A seventh step of acquiring a scale value, converting the acquired gray scale value into height information of the three-dimensional shape data and generating the height information, and the limiting means in the seventh step Generation The height indicated by the height information of the three-dimensional shape data generated in the seventh step is limited based on the average value and standard deviation of the height indicated by the height information of the three-dimensional shape data. In the eighth step, the height information is changed in the eighth step with respect to an area other than the area where the black-and-white inversion means does not perform the black-and-white inversion set in the third step. The black-and-white reversal processing is performed on the three-dimensional shape data that has been changed. A ninth step of converting the three-dimensional shape data into the three-dimensional shape data of the region corresponding to the human head hair; and the two-dimensional image data generated by the forming means in the second step. The nose of the human face A tenth step of changing the height information of the three-dimensional shape data as a shape corresponding to the rising of the nose of the human face with the nose tip indicated by the two-dimensional image data as a vertex for the region; and the second addition The means is positioned at the center of the predetermined shape area at the height indicated by the height information of the three-dimensional shape data corresponding to the predetermined shape area specified to include the human head indicated by the two-dimensional image data. An eleventh step of adding a predetermined height so that the height indicated by the corresponding height information is the highest, and changing the height information of the three-dimensional shape data into height information indicating the added height. A twelfth step in which the fillet processing means fillets the three-dimensional shape data;It is made to have.
[0015]
  Therefore, according to the first aspect of the present invention, since the three-dimensional shape data is generated from the two-dimensional image data, the three-dimensional scanner as described in the above-mentioned section “Prior Art” is used. 3D shape data can be generated at low cost without the need to measure objects that actually have a 3D shape.Furthermore, the generated three-dimensional shape data does not make the three-dimensional shape of the human face unnatural due to the influence of illumination when the two-dimensional image data is acquired.
[0016]
  Also, the forming meansFor the region of the nose portion of the human face indicated by the two-dimensional image data generated in the second step, the shape corresponding to the rise of the nose of the human face having the nose tip indicated by the two-dimensional image data as a vertex As the height information of the three-dimensional shape data is changed asIn the generated three-dimensional shape on the face side of the human head, the rising of the nose is expressed, and high-quality three-dimensional shape data indicating the face side of the human head can be generated.
[0018]
  In addition, the black and white reversing means performs black and white reversal processing of the three-dimensional shape data whose height information has been changed in the eighth step for areas other than the areas that are not subjected to black and white reversal set in the third step. The three-dimensional shape data subjected to the black / white reversal processing is compared with the three-dimensional shape data not subjected to the black / white reversal processing, and the three-dimensional shape data having the higher height information is applied to the human head hair. Since it is the 3D shape data of the corresponding area,It is possible to prevent the originally raised portion such as hair from being depressed, and to generate high-quality three-dimensional shape data of the human head.
[0020]
  In addition, the gray conversion unit includes red component data constituting pixel data of the two-dimensional image data processed in the fourth step, the fifth step, and the sixth step; The grayscale value is obtained by weighting and adding the component data and the blue component data at a predetermined ratio, for example, a ratio of approximately R: G: B = 7: 2: 1, and the acquired grayscale value Is converted into the height information of the three-dimensional shape data to generate the height information.The correspondence between the face of the two-dimensional image and the face of the three-dimensional shape is improved, the unevenness of the three-dimensional face can be made more natural, and the three-dimensional showing the face side of the high-quality human head Shape data can be generated.
[0022]
  Further, the restricting means is based on the average value and the standard deviation of the height indicated by the height information of the three-dimensional shape data generated in the seventh step. Because the height information is changed by limiting the height indicated by the height information of the three-dimensional shape data,In the three-dimensional shape on the face side of the human head indicated by the generated three-dimensional shape data, it is possible to prevent a steep convex portion from being formed, and 3 indicating the face side of the high-quality human head. Dimensional shape data can be generated.
[0024]
  Further, the second adding means has the height indicated by the height information of the three-dimensional shape data corresponding to the region of the predetermined shape designated so as to include the human head indicated by the two-dimensional image data. Height information indicating the height obtained by adding the predetermined height so that the height indicated by the height information corresponding to the center of the region of the predetermined shape is the highest, and adding the height information of the three-dimensional shape data To the eleventh step,The roundness of the entire face side is expressed in the generated three-dimensional shape, and high-quality three-dimensional shape data indicating the face side of the human head can be generated.
[0026]
  Also,UpSince the fillet processing means performs fillet processing on the three-dimensional shape data,The corners of the three-dimensional shape of the vertical surface are rounded and curved, and there is no vertical surface that is impossible for human faces. The three-dimensional shape of the human face is naturally finished and the face side of a high-quality human head Can be generated.
[0027]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, an example of an embodiment of a method for generating three-dimensional shape data according to the present invention will be described in detail with reference to the accompanying drawings.
[0028]
FIG. 1 is a block diagram of a three-dimensional shape data generation apparatus that implements an example of an embodiment of a method for generating three-dimensional shape data according to the present invention.
[0029]
In the three-dimensional shape data generation apparatus, the overall operation is controlled by a central processing unit (CPU) 10.
[0030]
The CPU 10 stores a read-only memory (ROM) 14 in which a program for operating the CPU 10 is stored via the bus 12 and two-dimensional image data to be described later under the control of the CPU 10. The three-dimensional shape data according to the present invention is controlled by the image data storage unit 16-1, the three-dimensional shape data storage unit 16-2 that stores the three-dimensional shape data generated by the three-dimensional shape data generation process according to the present invention, and the CPU 10. A random access memory (RAM) 16 in which an area or the like as a working area is set, and an input device 18 including a pointing device such as a mouse and a character input device such as a keyboard A display device 20 composed of a CRT, a liquid crystal display device (LCD), etc .; And a like type of hard disk that stores such data and a configured external storage device 22.
[0031]
Here, the CPU 10 performs a process of executing a program for realizing various processes in the three-dimensional preparation process and the three-dimensional execution process of the three-dimensional shape data generation process according to the present invention described later.
[0032]
Here, as described above, the display device 20 includes, for example, a CRT or a liquid crystal display device 20 (LCD), and various screens are displayed on the display device 20.
[0033]
FIG. 2 schematically shows a user setting screen 100 displayed on the display device 20 when the three-dimensional shape data generation device is activated.
[0034]
In this user setting screen 100, the 2D image data field 100b for selecting and setting the 2D image data used for the 3D shape data generation processing according to the present invention by the file name and the 2D image data field 100b are set. Image area 100a in which the 2D image indicated by the generated 2D image data and the 3D shape indicated by the 3D shape data generated by the 3D shape data generation processing according to the present invention are displayed, and 3D shape data generation According to the present invention, a mode selection area 100c for selecting and setting the operation mode of the apparatus, a background mask column 100i and an inversion prohibition mask column 100j for determining and setting the type of region designated in the region setting mode described later, and Select whether or not to instruct the start of the 3D execution process described later in the 3D shape data generation process A 3D conversion start field 100k to be determined, and a preview for selecting and setting whether or not to display the 3D shape indicated by the 3D shape data generated by the 3D shape data generation processing according to the present invention in the image area 100a The column 100m is displayed.
[0035]
Then, the input device 18 including a pointing device or a character input device inputs a desired instruction of the operator by selecting a desired position on the screen of the display device 20 or inputting a desired numerical value. Is.
[0036]
The contents of various settings made by operating the pointing device or the character input device of the input device 18 are processed by the CPU 10 and stored in a predetermined area of the RAM 16 as information used for the processing for generating the three-dimensional shape data according to the present invention. Is done.
[0037]
More specifically, the input device 18 selects and sets a desired 2D image data file name in the 2D image data field 100b, or selects a desired 2D image or 3D shape displayed in the image area 100a. In the mode selection area 100c, the ellipse setting mode field 100d, the right eye setting mode field 100e, the left eye setting mode field 100f, the nasal outline setting mode field 100g, and the area setting mode field 100h are selected. By selecting and setting a desired operation mode of the three-dimensional shape data generation apparatus, or by selecting the background mask column 100i or the inversion prohibition mask column 100j, the type of region specified in the region setting mode to be described later By selecting the three-dimensional start column 100k, the three-dimensional shape data of the present invention This is for setting an instruction to start a three-dimensionalization execution process, which will be described later, or displaying the three-dimensional shape indicated by the three-dimensional shape data in the image area 100b by selecting the preview field 100m. is there.
[0038]
In the above configuration, the three-dimensional shape data generation processing according to the present invention, specifically,
A generation process of three-dimensional shape data indicating a three-dimensional shape of a human head, particularly, the face side will be described with reference to FIGS.
[0039]
First, in order to facilitate understanding of the processing for generating three-dimensional shape data according to the present invention, the two-dimensional image data used for the processing for generating three-dimensional shape data according to the present invention will be described.
[0040]
The two-dimensional image data indicates a two-dimensional image of a predetermined object having a three-dimensional shape. In this embodiment, two-dimensional image data obtained by photographing the face side of a human head with a digital camera, that is, two-dimensional image data having a color is used.
[0041]
The two-dimensional image data is input from the outside of a three-dimensional shape data generation device such as the external storage device 22 by “(1-1) two-dimensional image data setting process” described later, and the two-dimensional image data is controlled by the CPU 10. It is stored in the data storage unit 16-1.
[0042]
Such two-dimensional image data is composed of pixel data corresponding to a predetermined number of pixels. The pixel data includes red (R) component data, green (G) component data, and blue (B) data, respectively. It consists of component data. Accordingly, luminance information indicating the brightness of each pixel can be obtained by processing at a ratio of approximately R: G: B = 3: 5: 2. That is, each point on the two-dimensional image corresponding to the pixel data constituting the two-dimensional image data (hereinafter simply referred to as “pixel point”) has luminance information indicating the brightness of the pixel point. .
[0043]
Here, FIG. 3 schematically shows an example of the two-dimensional image indicated by the two-dimensional image data. The two-dimensional image data showing the two-dimensional image 200 of the human head as shown in FIG. Are used in this embodiment.
[0044]
The two-dimensional image 200 (see FIG. 3) indicated by the two-dimensional image data is displayed in the image area 100a of the user setting screen 100 (see FIG. 2). In the following description, each part of the human head of the two-dimensional image 200 is represented by “face 200a”, “hair 200b”, “left eye part 200c”, “right eye part 200d”, “ It will be appropriately referred to as “nose portion 200e”.
[0045]
The generation process of the three-dimensional shape data according to the present invention includes two stages of processes, a three-dimensional preparation process and a three-dimensional execution process. Hereinafter, various processes in the two stages will be described in detail.
[0046]
(1) Three-dimensional preparation process
When the three-dimensional shape data generation device is activated, a user setting screen 100 (see FIG. 2) is displayed on the screen of the display device 20. Then, the three-dimensional preparation process is performed by the operator selecting each column of the user setting screen 100 using the input device 18.
[0047]
In this three-dimensional preparation process, the following six processes are sequentially performed. The six processes are a two-dimensional image data setting process, a three-dimensional area setting process, a binocular position setting process, a nose outline setting process, a background area setting process, and an inversion-inhibited area setting process. It will be explained in the following.
[0048]
(1-1) Two-dimensional image data setting process (see FIG. 3)
In this two-dimensional image data setting process, an operator selects and sets two-dimensional image data used for three-dimensional shape data generation processing.
[0049]
Any one of the two-dimensional image data stored in the two-dimensional image data storage unit 16-1 by the operator operating the input device 18 in the two-dimensional image data column 100b of the user setting screen 100 shown in FIG. Alternatively, a desired two-dimensional image data file is input.
[0050]
Thereby, the two-dimensional image data of the file selected in the two-dimensional image data column 100b is set as the two-dimensional image data used for the three-dimensional shape data generation process. In the image area 100a of the user setting screen 100, a two-dimensional image 200 of the human head indicated by the set two-dimensional image data is displayed (see FIG. 3).
[0051]
(1-2) Three-dimensional area setting process (see FIGS. 4A, 4B, and 4C)
In this three-dimensional area setting process, the operator selects and sets a range in which three-dimensional shape data is generated from the two-dimensional image indicated by the two-dimensional image data.
[0052]
Specifically, first, the operator operates the input device 18 to select the ellipse setting mode field 100d in the mode selection area 100c of the user setting screen 100 shown in FIG. As a result, the operation mode of the three-dimensional shape data generation apparatus is changed to the ellipse setting mode.
[0053]
In the ellipse setting mode, the operator operates the pointing device of the input device 18 in the image area 100a of the user setting screen 100 to select one point of the two-dimensional image 200 displayed in the image area 100a. Specify (see point E1 in FIG. 4A). Thereafter, the operator operates the pointing device of the input device 18 to designate one point of the two-dimensional image 200 displayed in the image area 100a (see point E2 in FIG. 4A).
[0054]
As a result, an ellipse 300 having the designated points E1 and E2 as both ends of the ellipse major axis and the ellipse major axis 300a are displayed on the two-dimensional image 200 in the image area 100a (FIG. 4B). reference).
[0055]
Further, the operator operates the pointing device of the input device 18 to designate one point of the two-dimensional image 200 displayed in the image area 100a (see point E3 in FIG. 4B). As a result, the ellipse 300 is deformed so that the designated point E3 is positioned on the circumference of the ellipse 300 (see FIG. 4C).
[0056]
Thus, when the operator designates three points on the two-dimensional image 200 (that is, point E1, point E2, and point E3), an ellipse 300 is automatically generated and set on the two-dimensional image 200. An internal region surrounded by 300 is set as a range in which 3D shape data is generated in the 2D image 200 indicated by the 2D image data, that is, a 3D region.
[0057]
Note that when the operator designates three points on the two-dimensional image 200 so that the ellipse major axis 300a displayed in the image area 100a is parallel to the nose strip on the human face 200a of the two-dimensional image 200, it will be described later. It is possible to make the outline of the nose automatically generated in the nasal outline setting process to be more accurate.
[0058]
(1-3) Binocular position setting process (see FIGS. 5A and 5B)
In this binocular position setting process, the operator selects and sets the position of the right eye and the position of the left eye on the human face 200a of the two-dimensional image 200 indicated by the two-dimensional image data.
[0059]
Specifically, first, the operator operates the input device 18 to select the right eye setting mode field 100e in the mode selection area 100c of the user setting screen 100 shown in FIG. As a result, the operation mode of the three-dimensional shape data generation apparatus is changed to the right eye setting mode.
[0060]
In the right eye setting mode, the operator operates the pointing device of the input device 18 in the image area 100a of the user setting screen 100, and the right of the two-dimensional image 200 displayed in the image area 100a. The center position of the eye part 200d is designated.
[0061]
As a result, the designated center position is set as the position of the right eye. In the image area 100a, a point RE indicating the set position of the right eye is displayed at the center of the right eye portion 200d (see FIG. 5A).
[0062]
Thereafter, the operator operates the input device 18 to select the left eye setting mode field 100f in the mode selection area 100c of the user setting screen 100 shown in FIG. As a result, the operation mode of the three-dimensional shape data generation apparatus is changed to the left eye setting mode.
[0063]
In the left eye setting mode, the operator operates the pointing device of the input device 18 in the image area 100a of the user setting screen 100, and the left of the two-dimensional image 200 displayed in the image area 100a. The center position of the eye part 200c is designated.
[0064]
As a result, the designated center position is set as the position of the left eye. In the image area 100a, a point LE indicating the set position of the left eye is displayed at the center of the left eye portion 200c (see FIG. 5B).
[0065]
(1-4) Nasal outline setting process (see FIG. 6)
In this nasal contour setting process, the contour shape of the nose portion 200e on the human face 200a of the two-dimensional image 200 indicated by the two-dimensional image data is set.
[0066]
Specifically, first, the operator operates the input device 18 to select the nose outline setting mode field 100g in the mode selection area 100c of the user setting screen 100 shown in FIG. As a result, the operation mode of the three-dimensional shape data generation apparatus is changed to the nose outline setting mode.
[0067]
As a result, as shown in FIG. 6, the human face 200a of the two-dimensional image 200 displayed in the image area 100a of the user setting screen 100 has five points (upper end point NU, lower end point ND, left end point NL, A nasal outline 400 comprising a right end point NR and a center point NC) is automatically generated.
[0068]
Of these five points, an inner region surrounded by a quadrangle composed of the upper end point NU, the lower end point ND, the left end point NL, and the right end point NR is a human face 200a of the two-dimensional image 200 indicated by the two-dimensional image data. Is set as the range of the nose portion 200e, that is, the nose region, and the center point NC is set as the tip of the nose.
[0069]
Here, automatic generation of the nose outline 400, that is, automatic generation processing of the upper end point NU, the lower end point ND, the left end point NL, the right end point NR, and the center point NC forming the nose outline 400 will be described with reference to FIGS. 12 will be described in more detail.
[0070]
FIG. 7 shows a flowchart of a nasal contour shape generation processing routine. The flowchart of the nasal contour shape generation processing routine shown in FIG. 7 is started when the operator selects the nasal contour setting mode field 100g on the user setting screen 100 and changes to the nasal contour setting mode as described above. Is to be executed.
[0071]
As shown in FIG. 5B, when the nose outline shape generation processing routine is started, the two-dimensional image 200 displayed in the image area 100a of the user setting screen 100 is displayed as described above (see FIG. 5B). 1-2) The ellipse 300 generated in the three-dimensional area setting process, the point RE indicating the position of the right eye and the position of the left eye set in the above-described (1-3) binocular position setting process LE is displayed.
[0072]
Then, when this nasal contour shape generation processing routine is started, in step S702, the nose upper end point NU is automatically generated. Specifically, a point RE indicating the position of the right eye and a point LE indicating the position of the left eye are obtained as an intermediate point and set as the upper end point NU. That is, as shown in FIG. 8, a point that bisects the straight line L1 connecting the point RE indicating the position of the right eye and the point LE indicating the position of the left eye is calculated, and the calculated point is above the nose. Automatically generated as an end point NU.
[0073]
When the process of step S702 ends, the process proceeds to step S704, and the nose lower end point ND is automatically generated. As shown in FIG. 9A, the ellipse 300 extends in the direction from the end point E1 to the end point E2 of the ellipse major axis 300, starting from the upper end point NU automatically generated in step S702, parallel to the ellipse major axis 300a. A half straight line L2 is obtained.
[0074]
After the half line L2 is calculated, the distance between the right eye position and the left eye position from the upper end point NU on the half line L2, that is, the point RE indicating the position of the right eye and the position of the left eye A point that is at a distance equal to the length of the straight line L1 that connects the point LE indicating is calculated, and the calculated point is automatically generated as the lower end point ND of the nose (see FIG. 9B).
[0075]
Then, following step S704, in step S706, the right end point NR and the left end point NL of the nose are automatically generated (see FIG. 10). A straight line L3 that passes through an intermediate point between the point RE indicating the position of the right eye and the upper end point NU automatically generated in step S702 and connects the upper end point NU and the lower end point ND (that is, part of the half line L2 in step S704). A straight line L4 parallel to the straight line) is obtained. Further, a straight line L5 is obtained which passes through the midpoint between the point LE indicating the position of the left eye and the upper end point NU set in step S704 and is parallel to the straight line L3 connecting the upper end point NU and the lower end point ND.
[0076]
Then, on the straight line L3 connecting the upper end point NU and the lower end point ND, a point at a position 5/6 from the upper end point NU side is calculated, and a point RE indicating the position of the right eye passing through the calculated point A straight line L6 parallel to the straight line L1 connecting the point LE indicating the position of the left eye is obtained.
[0077]
After the straight line L6 is thus calculated, the intersection of the straight line L6 and the straight line L4 is calculated, the calculated point is automatically generated as the right end point NR of the nose, and the intersection of the straight line L6 and the straight line L5 is calculated. The calculated point is automatically generated as the left end point NL of the nose.
[0078]
  Then, when the process of step S706 is completed, the process proceeds to step S708, and the nose center point NC is automatically generated. The left end point NL and the right end point NR set in step S706ofFind the midpoint. That is, as shown in FIG. 11A, a point P that bisects a straight line L6 connecting the left end point NL and the right end point NR is calculated.
[0079]
An intermediate point between the calculated point P and the lower end point ND of the nose, that is, a point that bisects a straight line connecting the calculated point P and the lower end point ND of the nose. Is automatically generated as the center point NC of the nose (see FIG. 11B).
[0080]
Thus, when the center point NC of the nose is automatically generated by the process of step S708, five points of the upper end point NU, the lower end point ND, the left end point NL, the right end point NR, and the center point NC are generated. The side LR connecting the upper end point NU and the right end point NR, the side LDR connecting the right end point NR and the lower end point ND, the side LDL connecting the lower end point ND and the left end point NL, the left end point NL and the upper end point The side LL connecting the NU and the side LC connecting the upper end point NU and the center point NC are automatically generated (see FIG. 12), the process of step S708 is terminated, and the nose outline shape generation process routine is terminated. To do.
[0081]
By the above-described processing routine for the nose outline, the upper end point NU, the lower end point ND, the left end point NL, the right end point NR and the center point NC are automatically generated, and the nose outline 400 (see FIG. 12) is automatically generated. It is displayed on the human face 200a of the two-dimensional image 200 displayed in the image area 100a of the user setting screen 100.
[0082]
The quadrilateral shape composed of the side LR, the side LDR, the side LDL, and the side LL of the nose outline 400 thus automatically generated and displayed is the shape of the nose on the human face 200a of the two-dimensional image 200 displayed in the image area 100a. The center point NC of the nasal contour shape 400 coincides with the nose tip of the nasal region 200e as well as the contour shape of the region 200e.
[0083]
Therefore, the two-dimensional image indicated by the two-dimensional image data is an internal region surrounded by a quadrangle composed of the upper end point NU, the lower end point ND, the left end point NL, and the right end point NR of the nose outline 400 that is automatically generated and displayed. While being set as the range of the nose portion 200e in the human face 200a of the image 200, the center point NC is set as the nose tip.
[0084]
However, depending on the type of the two-dimensional image or the like, the nasal outline 400 that is automatically generated and displayed does not match the nose portion 200e on the human face 200a of the two-dimensional image 200 displayed in the image area 100a. There is also. Only in such a case, the adjustment processing of the nose outline 400 that is automatically generated and displayed is performed.
[0085]
Specifically, as shown in FIG. 6, the nose outline shape 400 that is automatically generated and displayed is the shape of the nose portion 200 e on the human face 200 a of the two-dimensional image 200 displayed in the image area 100 a. If it does not match the outline shape, the operator operates the input device 18 to move the upper end point NU of the nose outline shape 400 so as to match the upper end of the nose portion 200e of the two-dimensional image 200, The lower end point ND of the nose outline 400 is moved so as to coincide with the lower end of the nose part 200e of the two-dimensional image 200, and the left end point NL of the nose outline 400 matches the left end of the nose part 200e of the two-dimensional image 200. The right end point NR of the nose outline 400 is moved so as to coincide with the right end of the nose portion 200e of the two-dimensional image 200, and the center point NC of the nose outline 400 is moved to the nose of the two-dimensional image 200. Nose portion 200e, i.e., moved to match the high position the tallest.
[0086]
The adjustment processing is thus performed, and the upper end point NU of the nose outline 400 ′ (see FIG. 13) that matches the nose portion 200e on the human face 200a of the two-dimensional image 200 displayed in the image area 100a. An internal area surrounded by a quadrangle composed of the lower end point ND, the left end point NL, and the right end point NR is set as the range of the nose portion 200e in the human face 200a of the two-dimensional image 200 indicated by the two-dimensional image data. The center point NC is set as the nose tip.
[0087]
(1-5) Background area setting process (see FIG. 14)
In this background area setting process, the operator sets the range of the background of the human head of the two-dimensional image indicated by the two-dimensional image data.
[0088]
Specifically, first, the operator operates the input device 18 to select the area setting mode column 100h in the mode selection area 100c of the user setting screen 100 shown in FIG. As a result, the operation mode of the three-dimensional shape data generation apparatus is changed to the region setting mode.
[0089]
In the area setting mode, the operator operates the pointing device of the input device 18 in the image area 100a of the user setting screen 100, and the human head of the two-dimensional image displayed in the image area 100a. Specify one point of the background of the part. Further, the area of the same color as the designated point is filled.
[0090]
The background of the human head of the two-dimensional image 200 (see the hatched area shown in FIG. 14) is designated by repeatedly performing such designation of one point and the process of filling the area of the same color with the designated point.
[0091]
Thereafter, the operator operates the input device 18 to select the background mask field 100i on the user setting screen 100. Thereby, the area (hatched area shown in FIG. 14) designated in the image area 100a is set as a background mask indicating the background range of the human head of the two-dimensional image 200 indicated by the two-dimensional image data.
[0092]
(1-6) Inversion prohibited area setting process (see FIG. 15)
In this inversion prohibition region setting process, the operator sets a range in which black and white inversion is not performed in the two-dimensional image indicated by the two-dimensional image data.
[0093]
Specifically, first, the operator operates the input device 18 to select the area setting mode column 100h in the mode selection area 100c of the user setting screen 100 shown in FIG. As a result, the operation mode of the three-dimensional shape data generation apparatus is changed to the region setting mode.
[0094]
In the area setting mode, the operator operates the pointing device of the input device 18 in the image area 100a of the user setting screen 100, and the human 2D image 200 displayed in the image area 100a is displayed. One point other than the hair 200b is designated. Further, the area of the same color as the designated point is filled.
[0095]
By repeatedly specifying such a point and filling the region of the same color with the specified point, a region other than the human head hair 200b of the two-dimensional image 200 (a hatched region shown in FIG. 15) is specified. .
[0096]
Thereafter, the operator operates the input device 18 to select the inversion prohibition mask field 100i on the user setting screen 100. As a result, the area designated in the image area 100a (the hatched area shown in FIG. 15), that is, the area other than the human head hair 200b is subjected to black and white reversal in the two-dimensional image indicated by the two-dimensional image data. It is set as an inversion prohibition mask indicating no range.
[0097]
(2) Three-dimensional execution processing
In the above (1) three-dimensional preparation process, (1-1) two-dimensional image data setting process, (1-2) three-dimensional area setting process, (1-3) binocular position setting process, (1- After the six processes of 4) nasal outline setting process, (1-5) background area setting process and (1-6) inversion prohibition area setting process are sequentially performed and all are finished, the three-dimensionalization execution process is performed.
[0098]
Specifically, after the three-dimensionalization preparation process is completed, when the operator selects the three-dimensionalization start column 100k on the user setting screen 100 of the display device 20 by the input device 18 including a pointing device or a character input device. A three-dimensional shape data generation processing routine (see FIG. 16) is activated and executed.
[0099]
The outline of the three-dimensional shape data generation processing routine shown in FIG. 16 will be described. First, optimization processing of two-dimensional image data used for generation of three-dimensional shape data is performed (steps S1602 to S1606), and then the optimum processing is performed. 3D shape data is generated from the converted 2D image data (step S1608), and the generated 3D shape data is adjusted (steps S1610 to S1622), whereby the above-described 3D preparation processing is performed. Based on the setting in, three-dimensional shape data is generated from the two-dimensional image data.
[0100]
Note that the three-dimensional shape data generated by the three-dimensional shape data generation processing routine is the human face corresponding to the two-dimensional image 200 (see FIG. 3) on the face side of the human head indicated by the two-dimensional image data. This is three-dimensional shape data indicating a three-dimensional shape. The generated three-dimensional shape data is stored in the three-dimensional shape data storage unit 16-2 under the control of the CPU 10.
[0101]
The three-dimensional shape data is a three-dimensional mesh composed of a plurality of mesh points having XYZ coordinate values in the XYZ coordinate system, and each mesh point that is an intersection of the meshes of the three-dimensional mesh has a coordinate value, that is, “X coordinate value” and “Y coordinate value” as position information and “Z coordinate value” as height information are set.
[0102]
In this embodiment, the three-dimensional mesh is initialized by a predetermined initialization process before the three-dimensional shape data generation processing routine is started. As a result, the size of the initialized 3D mesh in the X-axis direction and the Y-axis direction corresponds to the 2D image data, and the X coordinate value and Y coordinate value of each mesh point of the initialized 3D mesh are It has a value corresponding to the pixel data constituting the two-dimensional image data, and the Z coordinate value has a predetermined value (for example, “0”). That is, each initialized mesh point of the three-dimensional mesh is associated with a pixel point on the two-dimensional image indicated by the two-dimensional image data.
[0103]
In the three-dimensional shape data generation processing routine, first, in step S1602, the light effect processing is performed on the two-dimensional image data. This light effect process is a process for preventing the periphery of the eye of the three-dimensional shape from being depressed, as if two lights are irradiated toward both eyes of the two-dimensional image indicated by the two-dimensional image data. To do.
[0104]
Specifically, a straight line L1 (see FIG. 13) connecting a point RE indicating the position of the right eye and a point LE indicating the position of the left eye with the point LE (see FIG. 13) indicating the position of the left eye of the two-dimensional image as the center. In the inner region of the left reference circle whose radius is 1/8 of (see Fig. 8), the minimum luminance value is obtained based on the luminance information of the two-dimensional image data. A straight line L1 (see FIG. 8) connecting the point RE indicating the position of the right eye and the point LE indicating the position of the left eye around the point RE (see FIG. 13) indicating the position of the right eye of the two-dimensional image. In the inner region of the right reference circle having a radius of 1/8 of, the minimum luminance value is obtained based on the luminance information of the two-dimensional image data.
[0105]
Thus, the minimum luminance value around the left eye region 200c of the two-dimensional image 200 (hereinafter, the left eye periphery minimum luminance value) and the minimum luminance value around the right eye region 200d (hereinafter, the right eye periphery minimum). Luminance value), and in the inner region of the left reference circle around the point LE indicating the position of the left eye, the luminance of the cosine curve is added according to the distance from the point LE indicating the position of the left eye, The luminance value obtained by adding the luminance of the cosine curve is set as the ideal luminance value around the left eye. Further, the luminance of the cosine curve is added according to the distance from the point RE indicating the position of the right eye in the inner region of the right reference circle around the point RE indicating the position of the right eye, and the luminance of the cosine curve is added. The obtained luminance value is set as the ideal luminance value around the right eye.
[0106]
Thereafter, the luminance of the inner region of the left reference circle and the luminance of the inner region of the right reference circle are obtained again, and the difference between the left eye peripheral ideal luminance value and the right eye peripheral ideal luminance value is obtained.
[0107]
Then, after restoring the luminance information of the two-dimensional image data, an amplitude cosine curve luminance corresponding to the difference from the ideal luminance value around the left eye is added to each pixel point in the inner region of the left reference circle. Then, the cosine curve luminance having an amplitude corresponding to the difference from the ideal luminance value around the right eye is added to each pixel point in the inner region of the right reference circle.
[0108]
Note that the luminance information of the two-dimensional image data is changed by the light effect process (step S1602), the left / right luminance balance process (step S1604), and the smoothing process (step S1606), which will be described later, to optimize the two-dimensional image data. Is made. At this time, the luminance information of the two-dimensional image data is changed by red (R) component data, green (G) component data, and blue (B) component data constituting the pixel data of the two-dimensional image data. To be reflected in the value of.
[0109]
For example, at the pixel point to which the cosine curve luminance is added by the light effect processing described above, the pixel data of the pixel point is changed according to the amount that the luminance information after the change is higher than the level of the previous luminance information. The values of the red (R) component data, the green (G) component data, and the blue (B) component data are changed.
[0110]
When the process of step S1602 ends, the process proceeds to step S1604, and left / right luminance balance processing is performed on the two-dimensional image data. This left / right luminance balance processing is processing for preventing the balance between the left side and the right side of the three-dimensional face from being lost, and corrects the balance between the luminance of the left region and the right region of the face of the two-dimensional image.
[0111]
Specifically, a straight line L1 (see FIG. 13) connecting a point RE indicating the position of the right eye and a point LE indicating the position of the left eye with the point LE (see FIG. 13) indicating the position of the left eye of the two-dimensional image as the center. In the inner region of the left reference circle whose radius is 1/2 of (see Fig. 8), an average luminance value is obtained based on the luminance information of the two-dimensional image data. A straight line L1 (see FIG. 8) connecting the point RE indicating the position of the right eye and the point LE indicating the position of the left eye around the point RE (see FIG. 13) indicating the position of the right eye of the two-dimensional image. In the inner region of the right reference circle having a radius of ½, a mean luminance value is obtained based on the luminance information of the two-dimensional image data.
[0112]
Thus, the average value of luminance around the left eye portion 200c of the two-dimensional image 200 (hereinafter, left eye surrounding luminance average value) and the average value of luminance around the right eye portion 200d (hereinafter, right eye surrounding luminance). Then, a difference between the left eye surrounding luminance average value and the right eye surrounding luminance average value is obtained, and a gradient of inclination corresponding to the difference is added to each pixel point of the two-dimensional image.
[0113]
Following the processing in step S1604, in step S1606, smoothing processing is performed on the two-dimensional image data. This smoothing process is a process for smoothing the surface of a three-dimensional shape, for example, the surface of the skin, and performs noise removal by Gaussian distribution on the two-dimensional image data.
[0114]
Specifically, attention is paid to a predetermined pixel point on the two-dimensional image indicated by the two-dimensional image data, and a certain range of luminance around the pixel point is obtained. When there are pixel points with extremely different luminances, the original two-dimensional image data is updated by multiplying the coefficient by the Gaussian distribution except the pixel points with extremely different luminances. Such processing is repeated for all pixel points on the two-dimensional image indicated by the two-dimensional image data.
[0115]
  Step S1606When the process is completed, the process advances to step S1608 to perform a gray conversion process. In this gray conversion process, the red (R) component data, the green (G) component data, and the blue (B) that constitute the pixel data of the two-dimensional image data optimized by the processes in steps S1602 to S1606. This is a process in which the data of the above component is subjected to gray conversion at a predetermined ratio, and a value obtained by performing gray conversion (hereinafter, gray conversion information) is used as height information of the three-dimensional shape data. That is, three-dimensional shape data is generated from the two-dimensional image data by the processing in step S1608, and the Z coordinate value of the mesh point of the initialized three-dimensional mesh is set corresponding to the two-dimensional shape.
[0116]
Specifically, first, attention is paid to a predetermined pixel point on the two-dimensional image indicated by the two-dimensional image data, and red (R) component data and green (G) component constituting the pixel data of this pixel point. Data, and blue (B) component data. At this time, if the luminance information of the two-dimensional image data has been changed by the optimization processing in steps S1602 to S1606, the red (R) component data, green ( G) component data and blue (B) component data are obtained.
[0117]
Then, gray conversion is performed at a ratio of approximately R: G: B = 7: 2: 1 to obtain gray conversion information of the predetermined pixel point. The obtained gray conversion information is set as the height information of the three-dimensional shape data, that is, the Z coordinate value of the mesh point of the three-dimensional mesh corresponding to the predetermined pixel point on the two-dimensional image.
[0118]
Further, the same processing is repeated for all pixel points on the two-dimensional image indicated by the two-dimensional image data.
[0119]
Note that the gray conversion information obtained from the two-dimensional image data in the process of step S1608 is different from the luminance information obtained from the two-dimensional image data. That is, both gray conversion information and luminance information can be obtained from optimized two-dimensional image data, but the luminance information is processed at a ratio of approximately R: G: B = 3: 5: 2. On the other hand, the gray conversion information is obtained by processing at a ratio of approximately R: G: B = 7: 2: 1.
[0120]
The ratio of R: G: B = 7: 2: 1 for obtaining this gray conversion information is a ratio suitable for the two-dimensional image 200 (see FIG. 3) of the human head. That is, by using the gray conversion information obtained at such a ratio, the correspondence between the face of the two-dimensional image and the face of the three-dimensional shape is improved compared to the case where the luminance information is the height information of the three-dimensional shape data. The unevenness of the three-dimensional face can be made more natural.
[0121]
When the process of step S1608 ends, the process proceeds to step S1610 to perform highlight removal processing on the three-dimensional shape data. This highlight removal process is a process for preventing the formation of steep convex portions in the three-dimensional shape, and an abnormal Z coordinate based on the average value and standard deviation of the height information of the three-dimensional mesh. Remove the value.
[0122]
Specifically, the height information of the three-dimensional shape data, that is, the average value and standard deviation (σ) of the Z coordinate values of the mesh points of the three-dimensional mesh set corresponding to the two-dimensional image data in step S1608. Is calculated.
[0123]
Thereafter, the range of 1σ from the average value is extended to the entire three-dimensional mesh, and the Z coordinate value deviating from the limit value is limited so as to be within the limit value.
[0124]
When the process of step S1610 ends, the process proceeds to step S1612 to perform black and white reversal processing on the three-dimensional shape data. This black-and-white reversal process is a process for preventing an originally raised portion such as hair (that is, dark hair in a two-dimensional image, such as hair) in a three-dimensional shape. The region with a low value is inverted and synthesized with the original three-dimensional shape data.
[0125]
Specifically, the Z coordinate value that the mesh point of the three-dimensional mesh has, that is, the height information of the three-dimensional shape data corresponds to the luminance information of the two-dimensional image data by the processing of step S1608 to step S1610. The following processing is performed.
[0126]
Black and white inversion processing is performed based on a preset luminance (that is, Z coordinate value). Then, the three-dimensional shape data subjected to the black and white reversal processing is compared with the original three-dimensional shape data, and synthesized so as to be composed of data with higher luminance (that is, Z coordinate value).
[0127]
This black and white inversion processing is a portion of the two-dimensional image indicated by the two-dimensional image data excluding the range in which black and white inversion set as the inversion prohibition mask in the above (1-6) inversion prohibition area setting processing is not performed. The region of the three-dimensional mesh corresponding to is targeted. Therefore, in this embodiment, the black and white inversion processing is performed only on the 3D mesh area corresponding to the human head hair 200b, which is a part other than the hatching area shown in FIG. No black-and-white reversal processing is performed on the entire image, that is, the entire three-dimensional mesh.
[0128]
When the process of step S1612 ends, the process proceeds to step S1614, and the background deletion process is performed on the three-dimensional shape data. This background deletion process is a process for making a human object stand out in a three-dimensional shape, and deletes an unnecessary background part.
[0129]
Specifically, in the above-described (1-5) background region setting process, the height information in this region, that is, in this region, is targeted for the region of the three-dimensional mesh corresponding to the portion set as the background mask. The Z coordinate value of the mesh point is set to “0”. Therefore, in this embodiment, the height information of the hatched area shown in FIG. 14, that is, the three-dimensional mesh corresponding to the background of the two-dimensional image is “0”.
[0130]
When the process of step S1614 ends, the process proceeds to step S1616 to perform a nose shape creation process on the three-dimensional shape data. This nose shape creation process is a process for expressing the rising of the nose in a three-dimensional shape, and the nose outline shape 400 (or figure) set by the above-described (1-4) nose outline setting process shown in FIG. 13 is used.
[0131]
Specifically, a region of a three-dimensional mesh corresponding to an internal region of a two-dimensional image surrounded by a quadrangle composed of an upper end point NU, a lower end point ND, a left end point NL, and a right end point NR of the nose outline 400 is set as a processing target. To do.
[0132]
First, in the nose outline 400, the angles and distances of the center point NC and the upper end point NU, the center point NC and the lower end point ND, the center point NC and the left end point NL, and the center point NC and the right end point NR are obtained.
[0133]
Based on the obtained angles and distances between the center point NC, the upper end point NU, the lower end point ND, the left end point NL and the right end point NR, predetermined mesh points of the three-dimensional mesh in the region of the nose outline 400 Pay attention to P, and determine the region to which this mesh point P belongs. That is, the area of the three-dimensional mesh to which the mesh point P belongs includes an area A corresponding to a triangular shape having apexes of the center point NC, the upper end point NU, and the right end point NR, and the center point NC and the right end point NR. An area B corresponding to a triangular shape having three vertices with the lower end point ND, an area C corresponding to a triangular shape having three points including the central point NC, the lower end point ND, and the left end point NL, and a central point It is one of the four areas of an area D corresponding to a triangular shape having three points of NC, the left end point NL, and the upper end point NU.
[0134]
After the region to which the mesh point P belongs is obtained, a straight line R connecting the center point NC and the mesh point P is extended, and an intersection point Q between the straight line R and the opposite side of the center point NC is obtained. At this time, the side connecting two points other than the center point NC among the three vertices of the region to which the mesh point P belongs is set as the opposite side. Specifically, as shown in FIG. 18, when the mesh point P belongs to the area A, the straight line R connecting the center point NC and the mesh point P is extended, and the opposite side of the straight line R and the center point NC is extended. An intersection point Q with the side LR is obtained.
[0135]
After the intersection point Q is obtained in this way, according to the distance L1 between the center point NC and the intersection point Q and the distance L2 between the center point NC and the mesh point P, the following equation 1 The height h is obtained (see FIG. 19). Note that 1/10 of the maximum Z coordinate value of the mesh point of the three-dimensional mesh is H.
[0136]
[Expression 1]

... Formula 1
The height h calculated by Equation 1 (ie, the Z coordinate value) is added to the Z coordinate value of the mesh point P.
[0137]
Further, the same processing is repeated for all mesh points in the three-dimensional mesh region corresponding to the nose outline 400. As a result, the region of the three-dimensional mesh corresponding to the inner region of the two-dimensional image surrounded by the quadrangle composed of the upper end point NU, the lower end point ND, the left end point NL, and the right end point NR of the nose outline 400 is raised. The corresponding shape is obtained (see FIG. 20).
[0138]
When the process of step S1616 ends, the process proceeds to step S1618, and a process for raising a three-dimensional ellipse is performed. This ellipse raising process is a process for expressing the roundness of the entire face in a three-dimensional shape. The ellipse 300 set by the above-described (1-2) three-dimensional area setting process (see FIG. 4C). ) Is used.
[0139]
Specifically, a region of a three-dimensional mesh corresponding to the inner region of the two-dimensional image surrounded by the ellipse 300 is set as a processing target.
[0140]
Then, the Z coordinate values of all the mesh points in the region of the three-dimensional mesh corresponding to the ellipse 300 are obtained assuming an ellipse sphere with the center of the ellipse 300 (a point on the ellipse major axis 300a) as the top. The height information, that is, the Z coordinate value is added.
[0141]
When the process of step S1618 ends, the process proceeds to step S1620, and fillet processing is performed on the three-dimensional shape data. This fillet process is a process for naturally finishing a three-dimensional human face, and rounds the corners of a vertical plane that cannot be found on a human face (see FIGS. 21A and 21B).
[0142]
Specifically, the center T of the funnel-like fillet mesh (see FIG. 22A) is made to coincide with each other for each mesh point of the three-dimensional mesh of the three-dimensional shape data. If the height information (Z coordinate value) of the mesh point of the three-dimensional mesh is higher than the height information of the fillet mesh point of the corresponding fillet mesh, the Z coordinate value of the mesh point of the three-dimensional mesh is The 3D mesh is deformed according to the height information of the fillet mesh.
[0143]
At the start of the fillet processing, the three-dimensional mesh generated by the processing from step S1602 to step S1618 is copied to prepare two three-dimensional meshes, a conversion source three-dimensional mesh and a conversion destination three-dimensional mesh. To do. That is, the height information of the conversion source 3D mesh is copied to the conversion destination 3D mesh, and the conversion source 3D mesh and the conversion destination 3D mesh are the same before the start of the fillet processing.
[0144]
22 to 26, the mesh points of the three-dimensional mesh and the mesh points of the fillet mesh are illustrated in a band shape for easy explanation.
[0145]
The fillet mesh used for the fillet processing has a square bottom surface having the same size in the X-axis direction and the size in the Y-axis direction (see FIG. 22B). When the radius of the fillet mesh is R, the size L in the X-axis direction and the Y-axis direction of the fillet mesh is R × 2 + 1.
[0146]
For each fillet mesh point of such a fillet mesh, a distance r from the center T of the fillet mesh is obtained (see FIGS. 23A and 23B). For a fillet mesh point whose calculated distance r is shorter than the radius R of the fillet mesh, a Z coordinate value Lr calculated by the following equation 2 is set as height information of the fillet mesh point.
[0147]
[Expression 2]

... Formula 2
On the other hand, for the fillet mesh point other than the fillet mesh point whose calculated distance r is shorter than the radius R of the fillet mesh, the Z coordinate value of the maximum value Lmax (for example, 255000) is used as the height information of the fillet mesh point. Set.
[0148]
A fillet mesh composed of fillet mesh points having Z coordinate values set in this way is a funnel-like body as shown in FIG. Then, the center T of the fillet mesh is made to coincide with a predetermined mesh point of the conversion source three-dimensional mesh to obtain the height information baseL of the mesh point of the three-dimensional mesh (see FIG. 24). This height information baseL is equal to the Z coordinate value of the mesh point of the conversion source three-dimensional mesh with which the center T of the fillet mesh coincides.
[0149]
Then, for all fillet mesh points of the fillet mesh, the Z coordinate value mL of the fillet mesh point and the height srcL of the mesh point of the conversion source three-dimensional mesh corresponding to the fillet mesh point of the Z coordinate value mL are obtained ( FIG. 25 (a)).
[0150]
Then, the obtained transformation source three-dimensional mesh height srcL is compared with the transformation source three-dimensional mesh height information baseL, and as a result of the comparison processing, the height srcL is larger than the height information baseL (higher ), The fillet mesh height targetL corresponding to the mesh point of the conversion source 3D mesh to be processed is set to
targetL = (srcL−baseL) × mL / Lmax + baseL (see FIG. 28B).
[0151]
After that, the height target L of the fillet mesh at the mesh point of the conversion source 3D mesh to be processed is compared with the height information of the conversion destination 3D mesh corresponding to the mesh point of the conversion source 3D mesh to be processed If the height targetL is smaller (lower) than the height information of the destination 3D mesh, the height information of the destination 3D mesh, that is, the Z coordinate value of the mesh point of the destination 3D mesh Is replaced with the height targetL (see FIG. 26A).
[0152]
Then, by performing such processing for all fillet mesh points of the fillet mesh, the corners of the vertical plane of the conversion destination three-dimensional mesh become round fillet surfaces (see FIG. 26B).
[0153]
Following the processing of step S1620, in step S1622, isolated region removal processing is performed on the three-dimensional shape data. This isolated region removal processing is processing for preventing the presence of a region isolated in a three-dimensional shape, and deletes an extra mesh point of the three-dimensional shape mesh.
[0154]
For a predetermined mesh point of the mesh point of the three-dimensional mesh of the three-dimensional shape data, the mesh points located in the eight directions around the predetermined mesh point are targeted, and effective data around the mesh point, that is, the predetermined mesh point When the number of mesh points whose Z coordinate value is not “0” is one or less among the eight mesh points positioned around, the Z coordinate value of the mesh point is set to “0”. Such a process is repeated for all mesh points of the three-dimensional mesh.
[0155]
When the isolated region removal processing is thus performed for all the mesh points of the three-dimensional mesh, the processing in step S1622 is terminated, and the three-dimensional shape data generation processing routine is terminated.
[0156]
When the operator selects the preview field 100m on the user setting screen 100 of the display device 20 by the input device 18 including a pointing device or a character input device after the three-dimensional shape data generation processing routine is finished, the three-dimensional shape data is displayed. The three-dimensional shape indicated by the three-dimensional shape data generated by the shape data generation processing routine is displayed in the image area 100a.
[0157]
As described above, the 3D shape data generation method according to the present invention generates 3D shape data from 2D image data, so that a 3D scanner is not necessary, and an object that actually has a 3D shape. It is possible to generate three-dimensional shape data at a low cost without the restriction of measuring.
[0158]
In addition, the method for generating three-dimensional shape data according to the present invention uses the following (I) to (VII) to display three-dimensional shape data indicating a high-quality three-dimensional shape from two-dimensional image data, in particular, a high-quality human being. It is possible to generate three-dimensional shape data indicating the face side of the head.
[0159]
(I) Based on the position of the right eye and the position of the left eye set on the human face 200a of the two-dimensional image 200 indicated by the two-dimensional image data, the above-described (1-4) nasal outline setting process (FIG. 6). The contour shape of the nose portion 200e is set. Based on the set nose outline 400, a predetermined value is added to the Z coordinate values of all mesh points in the region of the three-dimensional mesh corresponding to the nose outline 400 by the nose outline creation process (step S1616). (See FIG. 20). As a result, the rising of the nose is expressed in the three-dimensional shape on the face side of the generated human head.
[0160]
That is, as described in the above-mentioned section “Prior Art”, when the luminance information of the two-dimensional image data is converted as the height information of the three-dimensional shape data, The shape of the nose does not protrude and it does not become unnatural.
[0161]
(II) The two-dimensional image data is optimized by the left-right luminance balance processing (step S1604) of the three-dimensional shape data generation processing routine, and the balance between the luminance of the left region and the right region of the face of the two-dimensional image is corrected. As a result, the balance between the left side and the right side of the generated three-dimensional face can be prevented from being lost.
[0162]
That is, as described in the above-mentioned section “Prior Art”, the human face of the two-dimensional image is irradiated with light from a predetermined direction, and the right side of the human face is compared with the left side, for example. Even when the area is bright, the height on the right side of the human face indicated by the generated three-dimensional shape data is higher than the height on the left side, and two-dimensional image data is acquired. The three-dimensional shape of the human face does not become unnatural due to the effect of lighting.
[0163]
(III) In the gray conversion process (step S1608) of the three-dimensional shape data generation process routine, a ratio suitable for the two-dimensional image 200 (see FIG. 3) on the face side of the human head, specifically about R : G: B = 7: 2: 1 The gray conversion information is obtained from the two-dimensional image data, and the gray conversion information is converted into the height information of the three-dimensional shape data. For this reason, the correspondence between the face of the two-dimensional image and the face of the three-dimensional shape is improved compared to the case where the luminance information is converted into the height information of the three-dimensional shape data. Can be.
[0164]
(IV) In the highlight removal processing (step S1610) of the three-dimensional shape data generation processing routine, based on the average value and standard deviation of the height information of the three-dimensional mesh, Since the Z coordinate value is limited, it is possible to prevent a steep convex portion from being formed in the three-dimensional shape on the face side of the human head indicated by the generated three-dimensional shape data.
[0165]
(V) A range in which black and white inversion is not performed is set by the above (1-6) inversion prohibition area setting process. Then, the black and white reversal process is performed on the part excluding the range where black and white reversal that is set as a reversal prohibition mask is not performed, specifically, only the part of the human head hair 200b (step S1612). Therefore, it is possible to prevent the originally raised portion such as the hair from being depressed.
[0166]
That is, as described in the section of “Prior Art” above, when the eye part of the human face of the two-dimensional image is a dark area such as a black iris color, the entire human face As a result of the black and white reversal processing being performed on the subject, the height of the eye part in the human face indicated by the generated three-dimensional shape data is higher than that around the eye part. In the three-dimensional shape, the part of the eye does not protrude and become unnatural.
[0167]
(VI) The ellipse 300 is set so as to surround the human face 200a of the two-dimensional image 200 by the above-described (1-2) three-dimensional area setting process. Then, since the ellipse raising process (step S1618) is performed on the three-dimensional area that is the inner area of the set ellipse 300, the roundness of the entire face side is expressed in the generated three-dimensional shape.
[0168]
(VII) In the fillet processing (step S1620) of the three-dimensional shape data generation processing routine, the three-dimensional mesh is deformed by the fillet mesh, so that the corners of the vertical surface of the three-dimensional shape are rounded, so that it is impossible for a human face. The vertical surface disappears, and the three-dimensional human face is naturally finished.
[0169]
In addition, when the 3D shape data generated by the 3D shape data generation method according to the present invention is used in, for example, a 3D cutting apparatus, the 3D shape indicated by the 3D shape data is high quality. The three-dimensional shape cut out from the workpiece based on the three-dimensional shape data can be made high quality.
[0170]
The embodiment described above can be modified as described in the following (1) to (5).
[0171]
(1) In the above embodiment, the 3D shape data generated by the 3D shape data generation method according to the present invention is stored in the 3D shape data storage unit 16-2. Of course, the present invention is not limited, and the 3D shape data generated by the 3D shape data generation method according to the present invention may be output to a drive device for 3D cutting, or may be stored in the external storage device 22. Alternatively, the three-dimensional data stored in the external storage device 22 may be used in a CAD system or the like.
[0172]
(2) In the above-described embodiment, the three-dimensional shape data is generated from the color two-dimensional image data. However, the present invention is not limited to this. When generating three-dimensional shape data from image data, the method for generating three-dimensional shape data according to the present invention may be used. If the gray conversion information cannot be obtained in the process of step S1608 depending on the type of the two-dimensional image data, the quality of the three-dimensional shape may be deteriorated. However, the luminance information is used instead of the gray conversion information. You may make it convert into the height information of three-dimensional shape data.
[0173]
(3) In the three-dimensional shape data generation method according to the present invention, the three-dimensional shape indicated by the three-dimensional shape data generated by the three-dimensional shape data generation method according to the present invention is not only a three-dimensional mesh but also a polygon or It can also be applied to curved surfaces.
[0174]
(4) Each process in the stage of the three-dimensional preparation process of the process of generating the three-dimensional shape data according to the present invention, or various processes in the stage of the three-dimensionalization execution process, respectively, May be omitted. For example, at the stage of the three-dimensionalization execution process, the order of the black-and-white reversal process (step S1612) and the background deletion process (step S1614) of the three-dimensional shape data generation process routine may be interchanged, or the background deletion process (Step S1614) may not be performed.
[0175]
(5) You may make it combine the above-mentioned embodiment and the modification shown in said (1) thru | or (4) suitably.
[0176]
【The invention's effect】
Since the present invention is configured as described above, it has an excellent effect that three-dimensional shape data indicating a high-quality three-dimensional shape can be generated at low cost.
[Brief description of the drawings]
FIG. 1 is a block diagram of a three-dimensional shape data generation apparatus that implements an example of an embodiment of a method for generating three-dimensional shape data according to the present invention.
FIG. 2 is an explanatory diagram showing a user setting screen displayed on the screen of the display device.
FIG. 3 is an explanatory diagram schematically showing an example of a two-dimensional image indicated by two-dimensional image data used in a method for generating three-dimensional shape data according to the present invention.
FIG. 4 is an explanatory diagram showing display contents of an image area of a user setting screen during a three-dimensional area setting process of a method for generating three-dimensional shape data according to the present invention, where (a) shows points E1 and E2; It is explanatory drawing which shows the designated state, (b) is explanatory drawing which shows the state by which the ellipse was produced | generated, (c) is explanatory drawing which shows the state by which the ellipse shown in (b) was deform | transformed.
FIG. 5 is an explanatory diagram showing display contents of an image area of a user setting screen during a binocular position setting process of the method for generating three-dimensional shape data according to the present invention, where (a) shows the position of the right eye It is explanatory drawing which shows the state where RE was displayed, (b) is explanatory drawing which shows the state where the point LE which shows the position of the left eye was displayed.
FIG. 6 is an explanatory diagram showing display contents of an image area of a user setting screen during a nasal contour setting process of the method for generating three-dimensional shape data according to the present invention.
FIG. 7 is a flowchart of a nasal outline shape generation processing routine.
FIG. 8 is an explanatory diagram showing generation processing of the upper end point NU of the nose in the generation processing routine for the nose outline.
FIGS. 9A and 9B are explanatory diagrams showing generation processing of the lower end point ND of the nose in the generation processing routine for the nasal outline. FIG.
FIG. 10 is an explanatory diagram showing generation processing of a right end point NR and a left end point NL of a nose outline shape generation processing routine;
FIGS. 11A and 11B are explanatory diagrams showing generation processing of a nose center point NC in a generation processing routine for a nasal contour. FIG.
FIG. 12 is an explanatory diagram showing a nasal contour shape generated by a nasal contour shape generation processing routine;
FIG. 13 is an explanatory diagram showing display contents of an image area of a user setting screen during a nasal contour setting process of the method for generating three-dimensional shape data according to the present invention.
FIG. 14 is an explanatory diagram showing display contents of an image area of a user setting screen at the time of setting a background area in the method for generating three-dimensional shape data according to the present invention.
FIG. 15 is an explanatory diagram showing display contents of an image area of a user setting screen at the time of inversion prohibited area setting processing of the method for generating three-dimensional shape data according to the present invention.
FIG. 16 is a flowchart of a three-dimensional shape data generation processing routine.
FIG. 17 is an explanatory diagram conceptually showing a nose shape creation process of a three-dimensional shape data generation process routine.
FIG. 18 is an explanatory diagram conceptually showing the contents of a nose shape creation process in a three-dimensional shape data generation process routine centered on area A.
FIG. 19 is an explanatory diagram showing the contents corresponding to Equation 1 for obtaining the height h of the mesh point P in the nose shape creation process of the three-dimensional shape data generation process routine.
FIG. 20 is an explanatory diagram illustrating an example of a three-dimensional mesh after a nose shape creation process of a three-dimensional shape data generation process routine;
FIGS. 21A and 21B are explanatory diagrams showing fillet processing of a three-dimensional shape data generation processing routine, and FIG. 21A is an explanatory diagram showing a three-dimensional mesh before fillet processing; ) Is an explanatory view showing a three-dimensional mesh after fillet processing.
FIGS. 22A and 22B are explanatory diagrams showing a fillet mesh used for a fillet process in a three-dimensional shape data generation process routine; FIGS.
FIGS. 23A and 23B are explanatory diagrams showing a fillet mesh used for a fillet process in a three-dimensional shape data generation process routine; FIGS.
FIG. 24 is an explanatory diagram conceptually showing a fillet process of a three-dimensional shape data generation process routine.
FIGS. 25A and 25B are explanatory diagrams conceptually showing fillet processing of a three-dimensional shape data generation processing routine. FIGS.
FIGS. 26A and 26B are explanatory diagrams conceptually showing fillet processing of a three-dimensional shape data generation processing routine. FIGS.
[Explanation of symbols]
10 Central control unit (CPU)
14 Read-only memory (ROM)
16 Random access memory (RAM)
16-1 Two-dimensional image data storage unit
16-2 3D shape data storage
18 Input device
20 Display device
22 External storage device
100 User setting screen
100a Image area
100b Two-dimensional image data column
100c mode selection area
100d Ellipse setting mode field
100e Right eye setting mode field
100f Left eye setting mode field
100g Nose outline setting mode field
100h area setting mode field
100i Background mask field
100j Inversion prohibition mask field
100k 3D start column
100m preview field
200 2D image
200a face
200b hair
200c Left eye region
200d Right eye region
200e Nose area
300 ellipse
300a Ellipse long axis
400, 400 'nose outline shape

Claims (1)

Designation means;
Generating means;
Reversal prohibition area setting means,
First addition means;
Correction means;
Smoothing processing means;
Gray conversion means,
Limiting means,
Black and white reversal means;
Forming means;
A second adding means;
Fillet processing means,
Luminance information obtained from two-dimensional image data composed of pixel data corresponding to a predetermined number of pixels is represented by Z coordinate values of three-dimensional shape data composed of a plurality of mesh points having XYZ coordinate values in the XYZ coordinate system. A method for generating three-dimensional shape data in a three-dimensional shape data generation device that generates three-dimensional shape data indicating a three-dimensional shape of a human head using the displayed height information,
It said designating means is a first step of receiving an input designating a position in the region of the eye in human facial indicated by the two-dimensional image data,
Said generating means, a second step of generating said first based on the location of the site of the designated eye in step, regions of the region of the nose of a human face indicated by the two-dimensional image data,
The inversion prohibition area setting means receives a designation input of a point other than the hair of the human head indicated by the two-dimensional image data, so that the color similar to the color of the point having received the designation input in the area other than the hair A third step of setting the color area as a region where black and white inversion is not performed;
The first adding means responds to the predetermined peripheral luminance value of the eye part specified in the first step indicated by the luminance information of the two-dimensional image data according to the distance from the eye part. A fourth step of adding a predetermined luminance value and changing the luminance information of the two-dimensional image data to luminance information indicating the luminance value of the addition result;
It said correction means, and centered on the mean value and the right eye luminance indicated by the first brightness information of the two-dimensional image data in a predetermined left region around the left eye of the sites specified eye step The difference between the luminance value indicated by the luminance information of the two-dimensional image data in the predetermined right region is obtained, and the balance between the luminance information of the left region of the face and the luminance information of the right region is corrected based on the difference. A fifth step of changing luminance information of the two-dimensional image data;
A sixth step in which the smoothing processing means performs a smoothing process on the two-dimensional image data;
The gray conversion means includes red component data and green component data constituting the pixel data of the two-dimensional image data processed in the fourth step, the fifth step, and the sixth step. The data and the blue component data are weighted and added at a predetermined ratio to obtain a gray scale value, and the obtained gray scale value is converted into height information of the three-dimensional shape data to obtain the height information. A seventh step of generating;
The three-dimensional generated in the seventh step based on the average value and standard deviation of the height indicated by the height information of the three-dimensional shape data generated in the seventh step. An eighth step of changing the height information by limiting the height indicated by the height information of the shape data;
The black-and-white reversing means performs black-and-white reversal processing of the three-dimensional shape data whose height information has been changed in the eighth step for regions other than the region that is not subjected to black-and-white reversal set in the third step, The three-dimensional shape data that has undergone the black-and-white reversal processing is compared with the three-dimensional shape data that has not undergone the black-and-white reversal processing, and the three-dimensional shape data with higher height information corresponds to the hair of the human head. A ninth step for setting the three-dimensional shape data of the region;
For the region of the nose portion of the human face indicated by the two-dimensional image data generated in the second step, the forming means has a nose of the human face whose apex is the nose tip indicated by the two-dimensional image data. A tenth step of changing the height information of the three-dimensional shape data as a shape corresponding to the swell;
The second adding means has the predetermined shape at a height indicated by height information of the three-dimensional shape data corresponding to an area of the predetermined shape specified so as to include the human head indicated by the two-dimensional image data. A predetermined height is added so that the height indicated by the height information corresponding to the center of the region becomes the highest, and the height information of the three-dimensional shape data is changed to height information indicating the added height. Eleventh steps to:
The method for generating three-dimensional shape data, wherein the fillet processing means includes a twelfth step of performing fillet processing on the three-dimensional shape data.

Images