JP4444440B2 - Stereo image measuring device - Google Patents

Description

【００２８】
本発明では、一例として、基準データブロックを左画像からテンプレート画像とし、右画像の探索領域内の捜索データブロックを単位として探索を行い、左画像の基準データブロック（テンプレート）に対して右画像の一致部分である捜索データブロックを探索する。一致部分は、相互相関係数が最大になる（１に近くなる）点とする。
【００２９】
以上の相関処理により、左右画像の対応点が求まるので、ステレオ法の原理により、各測定点における三次元座標値を計算する。ここで、図７に、ステレオ法についての説明図を示す。
【００３０】
（ステレオ法）
簡単のために、同じカメラを２台使用し、それぞれの光軸は平行でカメラレンズの主点からＣＣＤ面までの距離ａが等しく、ＣＣＤは光軸に直角に置かれているものとする。
２つの光軸間距離（基線長）をＬとする。
【００３１】
物体上の点Ｐ_１（ｘ_１、ｙ_１）、Ｐ_２（ｘ_２、ｙ_２）の座標の間には、以下のような関係がある。
ｘ_１＝ａｘ／ｚ
ｙ_１＝ｙ_２＝ａｙ／ｚ −−−（５）
ｘ_２−ｘ_１＝ａＬ／ｚ
但し、全体の座標系（ｘ、ｙ、ｚ）の原点をカメラ１のレンズ主点にとるものとする。カメラ１による撮影画像を左画像、カメラ２による画像を右画像としたとき、画像相関処理により、左画像ｘ_１の位置に対するｘ_２の位置が求まる（類似度の一番高い点）。従って、▲３▼式よりｚを求め、これを用いて▲１▼式、▲２▼式よりｘ、ｙが求められる。
【００３２】
再び、計測についてのフローチャートに戻り説明する。ステップＳ１１０で、求められた左右画像の相関結果を表示部４０に表示する。この場合、捜索データブロックを探索した結果、一番大きな相関係数（類似度が高い点）をそのデータブロックでの類似点として、それら相関結果を画像上に表示する。
【００３３】
図８に、相関結果の表示についての説明図を示す。例えば、図８（ａ）のＥ１領域や図８（ｂ）に示されるように、相関係数の低かったエリア又は点があれば、それらは強調して表示される。表示方法は、実際の相関係数値を色別等に表示してもよいし、しきい値を数段階設けて表示してもよい。ステップＳ１２０で、表示された結果が満足行くものであれば終了する。
【００３４】
つぎに、追加測定フローについて説明する。ステップS１３０では、表示された結果が満足いかなければ、ステップS１５０に進む。ステップＳ１５０では、測定に要するエリア、例えば、相関係数の低いエリアについて測量機１により追加計測を行なう。ここで、マニュアル計測であれば、ステップＳ８０へ移る。
【００３５】
Ｃ．半自動計測
半自動計測は、自動視準型トータルステーションを使って行なう。図９に、半自動計測についてのフローチャートを示す。
【００３６】
ステップＳ５１０では、例えば、図８に示されているように、相関係数の低いエリアが表示されているＥ１エリアをポインティングデバイス（ノートパソコンであれば、カーソルで、ペンコンピュータであればペン）で指し示す。ステップＳ５２０では、指示された点のｘ、ｙ座標の位置情報をパソコンから測量機１へ転送し、その位置を計測するように測量機１をモーター駆動して、計測する。計測指示は、パソコンから測量機１へ自動送出される。あるいは、計測指示は、人間が測量機の位置を確認して計測指示を出してもよい。仮に、計測しようとした個所が計測不能な点であれば、エリア内で計測可能な別の個所を探し測定する。これら画像を利用した半自動計測作業は、マニュアル計測と比較すると、測量機を視準しながら計測する、という手順を省略できるので、相当の作業効率化が達成される。すなわち、測量機で視準できる像は、対象物の局所領域しか映っていないため 、追加計測したい個所に視準するのは大変困難かつ時間のかかる作業となる。しかしながら、画像上をクリックするだけの作業とすれば、誰にでもできる簡単な作業となる。
【００３７】
次にステップＳ８０へ移る。ステップS８０で、追加計測した測定点を加え、三角形を作成する。図１０に、測定点の追加についての説明図を示す。すなわち、図１０で示された、追加点Ｃ７により、もう一度測定点の接続をし直す。この場合、図示のように追加点に対し更に詳細な三角形をつくってもよいし、前の接続点にかかわらずもう一度全部三角形を作り直してもよい。
【００３８】
また、ステップ１５０で、追加計測を行なわなかったり、選択可能な点が無かった場合等は、三角形を組み替える。図１１に、三角形の組み替えについての説明図を示す。例えば、図１１（ａ）のように三角形を作成していた場合、図１１（ｂ）のように組み替えることができる。この他に、ステップS１５０で線状に計測した点を追加選択すれば、その分細かく三角形を作成することになり、より信頼性、密度ともに高いものとなる。
【００３９】
以下上述と同様にステップＳ９１以降の処理を実行し、ステップS１００で、新しい領域について、画像相関処理を行なう。この場合、更に詳細な三角形を作成した場合は、新たに追加された三角形領域のみ画像相関処理を行なう。ステップS１１０で、相関結果を表示部に表示する。ステップS１２０で、結果が良好であれば、終了する。ステップS１３０で、表示された結果が満足できない場合は、再び追加測定に進む。ステップS１５０で、さらに追加測定を行いたい領域に対して、モニタを見ながら追加計測する。以下手順は、上記ステップＳ８０以降と同様であり、満足出来るまで繰り返す。
【００４０】
仮にいくら追加計測を行なっても改善されない場合又は追加計測をしない場合は、ステップＳ１４０へ進む。ステップS１４０で、例えば図１０のＥ１０の領域がどうしても良い結果が得られないときは、Ｃ１、Ｃ２、Ｃ７の計測点からなる平面Ｅ１０の平面方程式から標高値を内挿する。例えば、Ｅ１０領域内のＣ８を求めたいとき、Ｃ８の座標値を（ｘ_８、ｙ_８、ｚ_８）とすれば、Ｃ８の標高値ｚ_８は、
ｚ_８＝−（ａｘ_８＋ｂｙ_８＋ｄ）／ｃ −−−（６）
により算出する。（係数ａ、ｂ、ｃ、ｄは、Ｃ１、Ｃ２、Ｃ７の３次元座標値より算出する）。
【００４１】
Ｄ．自動測定
測量機に自動視準型トータルステーションを利用することによって、自動計測を行なうこともできる。
以下に自動測定について説明する。
【００４２】
ステップＳ１５０では、追加測定を以下の手順で行なうことにより、計測の自動化が行える。Ｓ１００の画像相関処理を行なった後、相関係数値にしきい値を設け、しきい値以下の相関係数が含まれるエリアについて追加計測を行なうようにする。しきい値に関しては、はじめから例えば０．５以下というように固定にしてもよいし、対象物によって決めても良い。計測エリアは、しきい値以下の相関係数が含まれる三角形領域とする。例えば、図８（ａ）ではＥ１の領域とする。測量機による追加計測点は、相関係数の低い三角形領域の三角形の重心位置としたり、あるいは、そのエリア内で相関係数の最も低い点としても良いし、低相関係数エリアの領域の中心（重心）としてもよい。以下に三角形の重心位置及び相関係数の低い領域の重心の求め方について説明する。
【００４３】
まず、三角形の重心は、例えば、図８（ａ）の三角形Ｅ１（Ｃ１，Ｃ２，Ｃ５）内の相関係数が低かった場合、その三角形の重心Ｃ７（図１０参照）を次の計測点として以下の要領で求める。Ｃ１、Ｃ２、Ｃ５の座標値をＣ１（Ｘ_１、Ｙ_１、Ｚ_１）、Ｃ２（Ｘ_２、Ｙ_２、Ｚ_２）、Ｃ５（Ｘ_５、Ｙ_５、Ｚ_５）、とした場合、
Ｘ＝（Ｘ_１＋Ｘ_２＋Ｘ_５）／３ −−−（７）
Ｙ＝（Ｙ_１＋Ｙ_２＋Ｙ_５）／３ −−−（８）
を次の計測選択点Ｃ７のＸＹ座標とする。
【００４４】
あるいは、図８（ｂ）で示される相関係数の低いエリアの分布からその重心を求めてもよい（例、モーメント法）。モーメント法では、次式により求められたｘ_ｇ、ｙ_ｇを追加計測選択点のＸ、Ｙ座標とする。追加計測選択点は１点でなくとも、複数点あってもよい。
ｘ_ｇ＝｛Σｘ＊｛１−ｃｏｒ（ｘ、ｙ）｝｝／Σ｛１−ｃｏｒ（ｘ、ｙ）｝
−−−（９）
ｙ_ｇ＝｛Σｙ＊｛１−ｃｏｒ（ｘ、ｙ）｝｝／Σ｛１−ｃｏｒ（ｘ、ｙ）｝
−−−（１０）
（ｘ_ｇ、ｙ_ｇ）：重心位置の座標、ｃｏｒ（ｘ、ｙ）：（ｘ、ｙ）座標上の相関値
【００４５】
以上より、追加計測点を決定したら、その座標値に測量機を制御して測量機を駆動、計測させる。仮にこれらの点が計測不能、困難な点であったら、その近傍をサーチし計測させる。他のステップはコンピュータ処理となるため、ステップＳ８０〜Ｓ１５０までの一連の作業が自動化される。
【００４６】
Ｅ．グラフィック表示による修正
図１２に、グラフィック表示についてのフローチャートを示す。
次に、相関処理の結果からグラフィック表示を行い確認、修正する方法について説明する。即ち、図２中ステップＳ１１０を以下の処理とすることで実現することができる。
【００４７】
まず、ステップS１０２で、求められた各点の三次元座標を接続して三角形を作成する。ステップS１０６で、三次元座標をもとに、グラフィック表示画像を作成する。必要に応じ、三次元座標の各点やそれらを接続したもの、等高線、あるいは三角形上に面や画像を貼った鳥瞰図を作成する。三角形上に画像を貼った図は、正射投影画像でも良い。必要なければ（視覚的に判断できれば）、これら処理を施さずとも（例えば三角形を接続しただけのものでも）良い。三次元座標が視覚的にあらわせるものなら何でもよい。これらをステレオモニタあるいはパソコンのモニタ等に適切に判断可能なグラフィック表示画像を表示する。ステップS１０６では、作成された画像を表示部４０に表示する。表示方法としては、三角形を重ね合せ表示する、あるいは相関係数を重ね合せ表示する等、状況により判断しやすいものを表示すればよい。
【００４８】
また、ステップＳ１０２〜Ｓ１０６で作成・表示された鳥瞰画像等において、間違っていたり、不自然と思われる領域について追加計測を行なうことができる。図１３に、グラフィック表示についての説明図を示す。例えば、図１３（ａ）に示される三角形を接続したものを表示するだけでも、実際の状況と違っていれば、その表示から悪い個所が判断可能となる。例えば、得られた三次元座標値が、本来、図１３（ａ）のＦ１であるはずなのに、間違えて図１３（ｂ）のＦ１’に移動してしまった場合、明らかに誤りが確認できる。更に図１３（ｂ）に面や画像を貼り付けて表示すれば、実際と違うことが一層明確に強調される。更に、相関係数値と重ね合せ表示すれば、仮に相関係数の低いエリアと悪い領域が一致していれば、より一層修正エリアが明確になる。この場合、図１３（ｂ）のＦ１周辺のエリアをマニュアルあるいは半自動で追加計測すればよい。
【００４９】
こうして、マニュアル計測であれば、ステップＳ８０へ移る。一方、自動視準型トータルステーションを使えば、半自動計測が可能となり、ステップＳ１１０へ移る。半自動計測では、上述したように、例えば、図１３に示されているように、間違っている、あるいは不自然と思われるＦ１エリアをノートパソコンであれば、カーソルで、ペンコンピュータであればペンで指し示すことで、以降の処理を実行する。
【００５０】
Ｆ．探索領域設定方法
次に、探索領域設定部２０による、探索領域、基準データブロック、及び捜索データブロックの設定及びスキャンについて説明する。探索領域の設定方法としては次のように例示され、順に説明する。
１．探索領域の設定：包含四角形
２．探索領域の大きさから各データブロックを設定する
３．基準点からの距離に応じて、各データブロックの位置、移動ステップを設定する
４．基準点からの距離に応じて、各データブロックの大きさを設定する
５．基準点の相関値より各データブロックの大きさを設定する
【００５１】
１．探索領域の設定：包含四角形
図１４に、探索領域の設定についての説明図（１）を示す。
基本的に左右画像の探索領域は、ステップＳ３０により相互標定が行われているので、縦視差がほぼ除去され、図中左右画像の各点ＣＬ１、ＣＲ１は、エピポーララインＰ１上に、ＣＬ２、ＣＲ２はＰ２上に、ＣＬ３、ＣＲ３はＰ３上にある。図に示されるように、測量機により計測した点から作成した三角形を左画像上でＣＬ１、ＣＬ２、ＣＬ３、右画像上でＣＲ１、ＣＲ２、ＣＲ３としたとき、探索領域を各三角形を包含する四角形領域とする。ここで例えば、領域内の左画像Ｔを基準データブロック（テンプレート）として、この画像に対応する位置を右画像から探索したい場合について説明する。この場合、捜索データブロックを含む右画像を捜索領域Ｓとして縦視差が除去されていればエピポーララインＰ３上で探索を行なう。もし、縦視差の除去が不完全であれば、対応するエピポーラライン周辺も探索することとなる。これら処理を、各基準点を含む包含四角形領域内の各ラインで行なう。図のように設定すると、隣接する三角形と重複領域ができるが、その分基準点以外の三角形領域を間違いなく確実に探索できる。
【００５２】
ここで、図１５に探索領域についての説明図を示す。図１４の三角形の内側の領域を探索領域としても良いが、基準点以外の対応点はその三角形領域内にあるかどうか不確実である。従って、図１５に示すように、三角形の領域を含む領域を探索領域と設定すれば効率的となる。
【００５３】
図１６に、探索領域の設定についての説明図（２）を示す。
図１６に示されるように、各基準データブロック（テンプレート）Ｔ１、Ｔ２に対応する捜索データブロックの位置を探索する際、ラインＰ３上の捜索領域にオーバーラップした各テンプレートに対するサーチエリアＳ１、Ｓ２を設けてもよい。ここでは、一例として、基準データブロックＴ１、Ｔ２の中心から所定範囲のエリアをサーチエリアＳ１、Ｓ２としたものである。こうすれば、明らかに各基準点から近い方のエリアが効率的に探索でき、オーバーラップさせることで信頼性が高められる。サーチエリアは各テンプレートに対し複数設定してよい。更に、基準点の右画像の対応点ＣＲ１、ＣＲ２、ＣＲ３の近傍のサーチエリアは小さく、離れるにしたがって大きく設定してもよい（なお、適宜最低値又は最高値を設定してもよい）。また、探索領域内のデータブロックに対する捜索領域（サーチエリア）の大きさは、後述する各データブロックの位置、移動ステップ、大きさによって適宜決めればよい。こうすれば、三角形を含む範囲を効率良く探索することが可能となる。
【００５４】
２．探索領域の大きさから各データブロックを設定する
図１７に、探索領域の大きさの所定についての説明図を示す。
図１７（ａ）に示すように、計測点によって探索領域である各三角形の大きさが異なるため、それぞれの三角形領域にあった大きさのデータブロックとするものである。例えば、前述したように探索領域として設定した包含四角形（図１４参照）を基準に各データブロックを決定する。この外接四角形に対し、１／ｋ の大きさをデータブロックとする。係数ｋは、あらかじめ固定としてもよいし、求めたい精度と領域の大きさにより適宜決定してもよい。他の例として、図１７（ｂ）に示されるように、三角形の内接円の半径ｒを求め、そのｒあるいは、ｒの１／２、１／３等をデータブロックサイズとしてもよい。ｒは次式によって求められる。
ｒ＝√｛（ｓ−ｏ）（ｓ−ｐ）（ｓ−ｑ）／ｓ｝ −−−（１１）
但し、ｓ＝１／２（ｏ＋ｐ＋ｑ） −−−（１２）
ｏ、ｐ、ｑは三角形の３辺のそれぞれの長さであり、これらは、３点の計測座標により計算する。
データブロックサイズの求めかたはこれにこだわらず、各三角形の大きさに応じたものであれば良い。
【００５５】
３．基準点からの距離に応じて、各データブロックの位置、移動ステップを設定する
図１８に、探索領域の設定についての説明図（３）を示す。
【００５６】
３．１ 基準データブロックに対する捜索データブロックの位置
図１８（ａ）に示されているように、例えば、左画像上のＴ１をテンプレート（基準データブロック）として、それと同一画像を右画像上から探索したい場合、Ｔ１の位置を基準点ＣＬ１、ＣＬ３からの距離ａ、ｂに応じて右画像からエピポーララインＰ３上でＣＲ１、ＣＲ３からａ、ｂに配分した点Ｓ１をサーチエリアの中心として、ライン上を所定の範囲内で探索する。こうすることで、Ｔ１の対応点として推定されるＳ１領域近傍の探索を行なえるので、効率良く短時間で処理が行なえるようになる。
【００５７】
３．２ 探索領域内の位置及び移動ステップ
例えば、図１８（ｂ）に示されるように、３．１で設定されたサーチエリアの中心から、両側に離れるほど移動ステップを粗くしていくことにより、探索がさらに効率良く成される。すなわち推定される領域の中心近くは密に、離れるほど粗くしていくことにより、より効率的な探索がなされる。粗くしていく比率は、単純に係数を設定してもよいし、三角形の重心位置（３．２．１参照）あるいは基準点２点と近い方の比からの距離に比例して変えていってもよい（３．２．２参照）。
【００５８】
３．２．１ 三角形重心法
三角形の重心位置から各データブロックの位置及び移動ステップを決める方法について説明する。図１９に、三角形の重心とデータブロックについての説明図を示す。
図１９（ａ）に示されているように、三角形の重心位置Ｇは各基準点より等距離となるが、重心以外の点Ｔと基準点までの距離は、重心から基準点までの距離より短くなる距離がある。これより、重心位置をデータブロックの最大のサイズとして、各基準点に近づく程ステップを小さくしていく。Ｃ１（ｘ１、ｙ１）、Ｃ２（ｘ２、ｙ２）、Ｃ３（ｘ３、ｙ３）とし、Ｔ（ｘ、ｙ）としたとき、各点のＴからの距離は、以下のようになる。
Ｔ−Ｃ１：Ｌ１＝√｛（ｘ−ｘ１）^２＋（ｙ−ｙ１）^２｝ −−−（１３）
Ｔ−Ｃ２：Ｌ２＝√｛（ｘ−ｘ２）^２＋（ｙ−ｙ２）^２｝ −−−（１４）
Ｔ−Ｃ３：Ｌ３＝√｛（ｘ−ｘ３）^２＋（ｙ−ｙ３）^２｝ −−−（１５）
また、各点の重心位置Ｇからの距離は、
Ｇ−Ｃ１＝Ｇ−Ｃ２＝Ｇ−Ｃ３＝Ｌｇ
＝√｛（ｘｇ−ｘ１）^２＋（ｙｇ−ｙ１）^２｝ −−−（１６）
式（１３）〜（１５）で求められた、距離Ｌ１、Ｌ２、Ｌ３の中から最小となるものを求め、その距離と重心までの距離の比を求め、移動ステップを決定する。例えば、重心位置Ｇのステップサイズを最大としてＳＧＴ、最小のステップサイズをＳＭＩＮとすれば、テンプレート位置から最小距離Ｌ（図１９（ａ）ではＬ２）の点のステップサイズは、
ステップサイズ＝Ｌ／Ｌｇ×（ＳＧＴ−ＳＭＩＮ）＋ＳＭＩＮ −−−（１７）
とする。
【００５９】
３．２．２ 近い法の基準点２点の比からの距離法：
各基準点からのＴまでの距離（近い方の２点）に応じて比例配分し、探索する距離に応じてステップサイズを可変とする。例えば、式（１３）〜（１５）の距離の小さい方２つの距離の比より決定する。図１９（ａ）のように、Ｌ２＜Ｌ３＜Ｌ１ のときは、Ｌ２＝Ｌ３ となる点を最大ステップ巾ＳＭＡＸとして、
ステップ＝Ｌ２／Ｌ３×ＳＭＡＸ −−−（１８）
とすればよい。
【００６０】
尚、探索領域内の位置及び移動ステップの決め方はこの方法にはこだわるものでない。
４．基準点からの距離に応じて、各データブロック、捜索領域の大きさを設定する
４．１ 各基準点からデータブロックとする画像位置の距離を求め、その距離の比に応じてデータブロックサイズを可変にする。図１９（ｂ）に示されているように、三角形の重心位置Ｇは各基準点より等距離となるが、重心以外の点Ｔと基準点までの距離は、重心から基準点までの距離より短くなる距離が存在する。これより、重心位置のデータブロックを最大のサイズとして、各基準点に近づく程エリアを小さくしていく。Ｃ１（ｘ１、ｙ１）、Ｃ２（ｘ２、ｙ２）、Ｃ３（ｘ３、ｙ３）とし、Ｔ（ｘ、ｙ）としたとき、各点のＴからの距離は、式（１３）〜（１５）より同様に求められる。また、各点の重心位置Ｇからの距離は、式（１６）より求められる。そして、距離Ｌ１、Ｌ２、Ｌ３の中から最小となるものをみつけ、その距離と重心までの距離の比を求め、データブロックサイズを決定する。
例えば、重心位置ＧのデータブロックサイズをＧＴ、データブロック位置から最小距離Ｌ（図１９（ｂ）ではＬ２）の点のデータブロックサイズをＣＴ（図１９（ｂ）ではＣ２のデータブロック）とすると、
データブロックサイズ＝Ｌ／Ｌｇ×（ＧＴ−ＣＴ）＋ＣＴ −−−（１９）
とする。
【００６１】
また、データブロックの縦横の巾は、縦視差が標定処理によってすでにほぼ除去されていれば、縦方向は可変とせず一定、横方向だけでも良い。このようにすることで、基準点の近傍は、対応点がそばにあるという前提でデータブロックを小さく、遠く離れるにしたがってそばにあるかどうか不確実となるのでデータブロックを大きくして効率良く確実に対応点を得ることができる。
【００６２】
尚、データブロックサイズの決め方はこの方法にはこだわるものでない。このようにすれば、効率的に速く探索が可能となる。
【００６３】
５．基準点の相関値より各データブロックの大きさを設定する
図２０に、データブロックの大きさについての説明図を示す。
各三角形の頂点（基準点）は、計測された点であるので、すでに対応づけが成されている。従って、この各点における各種テンプレート（基準データブロック）サイズに対する相関係数を求め（図２０（ａ）に示すように）、一番高い相関値を示したテンプレートをテンプレート画像とする。この求められたテンプレートサイズの画像で各基準点に近いエリア内を探索するようにする。各基準点で求まったテンプレートサイズを、その探索する位置に応じて変える。
【００６４】
例えば、一例として、各基準点で求められたサイズと各基準点からの距離（近い方の２点）に応じて比例配分し、探索する距離に応じてサイズを可変とする（図２０（ｂ）参照）。Ｃ１（ｘ１、ｙ１）、Ｃ２（ｘ２、ｙ２）、Ｃ３（ｘ３、ｙ３）とし、テンプレートＴ（ｘ、ｙ）としたとき、求めるテンプレートサイズは、式（１３）〜（１５） より求まる。従って求められた小さい方２つの距離の比より決定する。例えば、図２０（ｂ）のように、Ｌ２＜Ｌ３＜Ｌ１ のときは、ＣＬ２とＣＬ３のテンプレートサイズをＬ２：Ｌ３の比より分割、Ｔのサイズとする。
【００６５】
尚、データブロックの縦横の巾は、縦視差が標定処理によってすでにほぼ除去されていれば、縦方向は可変とせず一定、横方向だけでも良い。
【００６６】
本発明のように、基準点から生成された三角形を基準に画像相関処理を行なうことで、疎密探索画像相関法のような異なる解像度の画像を多数用意する必要が無くなり、結果として高速かつ信頼性の高い画像相関処理を行なうことが出来るようになる。加えて、測量機による併用測定ができるので、計測が信頼性高く、確実なものとなり、従来では時間がかかり、計測が行なえなかった場所も、速く、容易に計測可能となる。
【００６７】
【発明の効果】
本発明によると、以上のように、ノンプリズムＴＳと画像相関処理を併用することで、立ち入ることの出来ないような箇所又は危険個所等のように従来測定が不可能であった領域についても、非接触で計測することを可能とせしめ、各方法で単独で測定する場合よりも全体の計測を一段と高速化、効率化、高信頼化させることができる。
【図面の簡単な説明】
【図１】本発明の測定装置の全体ブロック図。
【図２】オンライン計測についてのフローチャート。
【図３】ステレオ画像の説明図。
【図４】計測点による三角形の作成についての説明図。
【図５】入力画像とテンプレート画像についての説明図。
【図６】３×３画素における例を示す図。
【図７】ステレオ法についての説明図。
【図８】相関結果の表示についての説明図。
【図９】半自動計測についてのフローチャート。
【図１０】測定点の追加についての説明図。
【図１１】三角形の組み替えについての説明図。
【図１２】グラフィック表示についてのフローチャート。
【図１３】グラフィック表示についての説明図。
【図１４】探索領域の設定についての説明図（１）。
【図１５】探索領域についての説明図。
【図１６】探索領域の設定についての説明図（２）。
【図１７】探索領域の大きさの所定についての説明図。
【図１８】探索領域の設定についての説明図（３）。
【図１９】三角形の重心とデータブロックについての説明図。
【図２０】データブロックの大きさについての説明図。
【符号の説明】
１ 測量機
２ 測定装置
３ カメラ
１０ 基準点捜索部
２０ 探索領域設定部
３０ 演算部
４０ 表示部
５０ 測定部
６０ バス[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a stereo image measurement device, and more particularly to a measurement device that measures a three-dimensional image from a stereo image.
[0002]
[Prior art]
Conventionally, when measuring a relatively large object as a three-dimensional image, the number of measurement points is increased by scanning the measurement points with a surveying instrument such as a total station (TS) or by moving the GPS. A three-dimensional image was represented by a set of many small faces surrounded by. As another method, three-dimensional measurement is performed by performing image correlation processing (stereo matching) or the like from a stereo image and representing the three-dimensional image as a set of many surface shapes.
[0003]
[Problems to be solved by the invention]
In the case of the former prior art, particularly when using a total station (TS), there is a method of acquiring three-dimensional data by automatically scanning a measurement point using a surveying instrument with an automatic collimation function (non-prism TS). . The non-prism TS is a total station that does not require a reflecting mirror such as a prism. However, since this method measures while driving the TS with a motor, the measurement time is enormous. For example, assuming that one point measurement takes 1 second, only about 200 × 200 points are measured, and about 11 hours are required. In addition, there are problems such as that measurement is impossible where the distance measuring light does not return, and accuracy decreases as the distance measuring beam becomes farther.
[0004]
Further, when the GPS is moved as in the latter prior art, since the measurement point coordinates are measured while moving the platform on which the GPS is mounted, it still takes much time and effort. Further, since the GPS has to be moved to the measurement target point, measurement in a dangerous place cannot be performed.
[0005]
In addition, in GPS, when taking a measurement from a stereo image, it is necessary to provide a reference point (orientation point) on the measurement object, and the taken stereo image must be located. It is not known whether measurement is possible, and stereo matching (image correlation processing) has problems such as an increase in calculation time and inability to measure portions without features. In particular, in order to reliably perform image correlation processing, a so-called image correlation method based on coarse to fine search is used. In this method, correlation processing is not performed on images with high resolution from the beginning, but correlation processing is performed step by step from images with low resolution to images with higher resolution, thereby reducing local errors, This is a technique for increasing the reliability (see Mikio Takagi, Yoshihisa Shimoda, Image Analysis Handbook p709). However, with this method, image correlation processing must be performed for each image having a different resolution, which takes a long time for calculation.
[0006]
In addition, when measuring with a stereo camera with a fixed base line, there is a merit that a reference point is not required, but the angle of view (measurement range) is limited, and if you want to increase the measurement range, connection for image connection There was a problem that a point (reference point) was required.
[0007]
In the present invention, in view of the above points, by using non-prism TS and image correlation processing together, even for regions that could not be measured conventionally, such as places that can not be entered or dangerous places, The purpose is to enable non-contact measurement, and to make the entire measurement much faster, more efficient, and more reliable than when measuring each method alone.
[0008]
[Means for Solving the Problems]
In the present invention, in order to solve the above-described problems, for example, there are the following features, and these processes enable measurement with higher speed and higher reliability than in the past.
1. Take a stereo image of the area you want to measure
2. Non-prism TS measures 6 or more reference (orientation) points in the area included in both stereo images.
3. Measure (orientate) the reference (orientation) points measured by TS on the stereo image.
4). Determine the area for image correlation processing from the measurement points
5). Perform image correlation processing on regions
An image created from the three-dimensional coordinate value obtained from the correlation coefficient of 6.5 or the result of the correlation process is displayed on the display.
[0009]
Furthermore, the present invention has the following features. By adding these processes, it is possible to perform measurement with higher speed and higher reliability. Furthermore, when it is considered that the reliability of measurement is insufficient, the following processes 7 to 10 may be repeated until satisfied.
7). If an area with a low correlation coefficient or an image created from measurement data is unsatisfactory, additional measurement is performed at that location.
8). Re-determine the area for image correlation processing from measurement points
9. Perform image correlation processing on regions
10. An image created from the correlation result is displayed on the display.
[0010]
In the present invention, an area that cannot be measured by the non-prism TS and cannot be obtained by the image correlation process can be calculated by interpolating the altitude value from the information of the measurement area of the non-prism TS. The advantages of these processes are as follows.
-Since the three-dimensional measurement value of the non-prism TS can be used as the initial value of stereo matching, it is possible to omit the coarse-to-fine image correlation processing and the calculation time is shortened.
・ By measuring the combination of stereo image and non-prism TS, the overall measurement time is much faster than the measurement of a single TS or stereo image.
-It is possible to measure with a non-prism TS a place where it is difficult to measure with an image (ex. A place without a feature) or with a non-prism TS. In this way, it is possible to interpolate the difficult measurement points of both.
-Where the image correlation is poor, interpolation is performed by measurement of the non-prism TS, and the measurement value is set as an initial value, so that the measurement area can be made denser and appropriate, and reliability is improved.
・ Because it becomes possible to measure at the site, it is possible to measure while checking the error location, so the reliability of the measurement is improved and the failure is eliminated.
[0011]
According to the solution of the present invention,
For a stereo image containing at least three measurement points for which position data is known, at least a part of the measurement points is defined as a segment point, and at least three segments among a plurality of defined segment points A setting unit for setting a search area based on points;
Based on the search region set by the setting unit, a calculation unit that performs correlation processing on images of corresponding search regions on each stereo image;
From the correlation result by the calculation unit, a measurement unit that measures the coordinates of a point at an arbitrary position;
With
According to the result of the correlation process, the calculation unit The region with a low correlation coefficient Measurement area where new measurement points are required And the position of the measurement area A stereo image measuring apparatus is provided for generating information on the above.
[0012]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[0013]
A. hardware
FIG. 1 shows an overall block diagram of the measuring apparatus of the present invention.
This system includes a surveying instrument 1, a measuring device 2, and a camera 3. The measurement device 2 includes a reference point search unit 10, a search area setting unit 20, a calculation unit 30, a display unit 40, a measurement unit 50, and a bus 60. The reference point search unit 10, the search area setting unit 20, the calculation unit 30, the display unit 40, and the measurement unit 50 are mounted on, for example, a personal computer and are connected to each other by a bus 60.
[0014]
The surveying instrument 1 measures the reference points at the site. This is not necessary when obtaining and measuring a stereo image including reference points in advance. The camera 3 acquires images, and for example, a digital camera or a film camera can be used. Even if the camera 3 is not provided, an image in which the reference points are copied may be obtained and analyzed in advance.
[0015]
The reference point search unit 10 associates a reference point that has been previously measured by the surveying instrument 1 with an image. The search area setting unit 20 sets a search area when performing image correlation processing, a reference data block, and each block of a search data block from the reference points associated by the reference point search unit 10. The calculation unit 30 performs orientation calculation and image correlation processing (stereo matching) for the search region set by the search region setting unit 20. The display unit 40 is a stereo monitor that enables stereoscopic viewing, a personal computer monitor, or the like. If a stereoscopic monitor is used, more detailed and accurate three-dimensional measurement and confirmation of measurement results can be performed. The display unit 40 graphically displays captured stereo images, correlation coefficient values obtained from the results of correlation processing, points and contour lines created from three-dimensional coordinates obtained from the results of correlation processing, bird's eye views, ortho images, etc. To do. The method of performing the graphic display by the display unit 40 for confirmation or additional measurement can be used for, for example, when it is desired to visually determine the measurement result appropriately or for an object whose correlation coefficient is not reliable. . Graphic display can be performed in real time.
[0016]
When the correlation result is bad as a result of the correlation process, the measurement unit 50 performs additional measurement. As the additional measurement by the measurement unit 50, there is a method of displaying a correlation coefficient and performing additional measurement of a portion with little correlation. In addition to this, create points, contour lines, wireframe models, surface models with surfaces, bird's-eye views and ortho images with images attached, based on the 3D coordinates of each point, and create graphics. There is a method in which additional measurement is performed for each defect point of the displayed image. In addition, the correlation coefficient is displayed on the screen, and the method for confirming or correcting the image is to display an area with a low correlation coefficient, and to perform manual or semi-automatic measurement correction based on the screen, and the correlation coefficient. There is a method of automatically measuring an area below a predetermined threshold by using.
[0017]
The advantage of the present invention is that it can be easily displayed, confirmed, and corrected in real time without using a stereo monitor, etc., and the overall measurement time is shortened, and the reliability is high and reliable. 3D measurement is possible.
[0018]
Next, a detailed explanation will be given of the case where manual / semi-automatic measurement and automatic measurement are performed based on the correlation result.
[0019]
B. Basic measurement flow
FIG. 2 shows a flowchart for online measurement. Online measurement is, for example, measuring an image at a measurement site and displaying a three-dimensional image.
[0020]
In step S10, first, a stereo image of an area to be measured at the site is first taken. FIG. 3 is an explanatory diagram of a stereo image. As shown in this figure, two overlapping stereo images (left image L and right image R) are taken. Next, in step S20, the surveying instrument 1 measures three or more reference (orientation) points in an area included in both the left and right images taken in stereo. The reference point may be 3 or more, but if it is 6 or more, the orientation process (see step S40 described later) is performed more stably, and the subsequent analysis becomes highly reliable. Therefore, here, as an example, description will be made with six points. That is, the surveying instrument 1 measures 6 points as reference (orientation) points C1 to C6 in the region where the left and right images in FIG. 3 overlap. Next, the stereo image data and the reference point coordinate value measured by the surveying instrument are transferred to a place where a stereo image measuring apparatus such as an office is located. For the transfer, it is possible to use a memory medium for storing an image or to transmit the image through a telephone line or the like.
[0021]
Next, in step S40, a reference (orientation) point measured by the surveying instrument 1 is measured (associated) on the left and right images displayed on the display unit 40. That is, in this example, the points C1 to C6 measured by the surveying instrument 1 are each positioned on the left image L and the right image R.
[0022]
In step S80, the measurement points measured in step S40 are connected to each other to create a triangle. FIG. 4 is an explanatory diagram for creating a triangle from measurement points. In this example, the points are connected from measurement points C1 to C6 to form a triangle. In this case, the points may be connected to form a quadrangle instead of a triangle. However, since the triangle can be further classified in the region (a triangle can form two planes with four points; It is possible to improve accuracy and reliability. There is an irregular triangular network (TIN) as a method of interpolating three-dimensional coordinates from such random points. TIN generates a mesh having a triangular unit. For details on TIN, see Masao Iri, Takeshi Koshizuka: Computational Geometry and Geographic Information Processing, pp 127, Franz Aurenhammer, Atsuyoshi Sugihara: Voronoi Diagram, Introduction to One Basic Geometric Data Structure, ACM Computing Surveys, Vol. 23, pp 345-405 ".
[0023]
In step S91, the reference point search unit 10 detects data relating to the reference data block (template) from one image, for example, the left image. For example, this data includes a triangle formed by three dividing points and a quadrangle including the triangle. In step S92, the position and size of the template are determined based on the distance from the template division point and the like. In step S93, the search area setting unit 20 determines a search area for performing image correlation processing from the template of the left image measured on the image. In step S94, the search area determined for the other image, eg, the right image, is scanned using the template for the left image. The setting and scanning of the template and search area will be described later.
[0024]
In step S100, the calculation unit 30 performs image correlation processing between the template and the search area for each divided area. That is, the corresponding point of the right image corresponding to the left image is obtained by obtaining the one having a large correlation (or the corresponding point of the left image corresponding to the right image is obtained), and the three-dimensional coordinates thereof are calculated. Note that image correlation processing (stereo matching) includes a residual sequential test method (SSDA method) and a cross-correlation coefficient method.
[0025]
Here, a cross-correlation coefficient method suitable for automatic measurement will be described.
(Method using cross-correlation coefficient)
FIG. 5 is an explanatory diagram for the input image and the template image. N as shown ₁ × N ₁ A template image of a pixel ₁ × M ₁ Search range in pixel input image (M ₁ -N ₁ +1) ² The upper left position of the template image is calculated so that the correlation coefficient r in the following equation is maximized, and it is considered that the template image has been searched.
[0026]
Here, FIG. 6 is a diagram illustrating an example of 3 × 3 pixels. In the case of this example, a comparison image I having the same value or a large correlation value as the template image T of the left image is searched from the right image on the same line (epipo line). That is, the above equation is calculated corresponding to each pixel. The pixels are shifted one by one or a predetermined number to obtain one having a high correlation value.
[0027]
[Expression 1]

[0028]
In the present invention, as an example, the reference data block is used as a template image from the left image, the search is performed in units of search data blocks in the search area of the right image, and the right image is compared with the reference data block (template) of the left image. Search the search data block which is the matching part. The coincident portion is a point where the cross-correlation coefficient is maximized (close to 1).
[0029]
Since the corresponding points of the left and right images are obtained by the above correlation processing, the three-dimensional coordinate value at each measurement point is calculated by the principle of the stereo method. Here, FIG. 7 shows an explanatory diagram of the stereo method.
[0030]
(Stereo method)
For simplicity, it is assumed that two identical cameras are used, each optical axis is parallel, the distance a from the principal point of the camera lens to the CCD surface is equal, and the CCD is placed at right angles to the optical axis.
Let L be the distance between the two optical axes (base line length).
[0031]
Point P on the object ₁ (X ₁ , Y ₁ ), P ₂ (X ₂ , Y ₂ ) Coordinates are as follows.
x ₁ = Ax / z
y ₁ = Y ₂ = Ay / z --- (5)
x ₂ -X ₁ = AL / z
However, the origin of the entire coordinate system (x, y, z) is taken as the lens principal point of the camera 1. When the image taken by the camera 1 is a left image and the image taken by the camera 2 is a right image, the left image x is obtained by image correlation processing. ₁ X for the position of ₂ The position of is found (the point with the highest similarity). Therefore, z is obtained from the equation (3), and x and y are obtained from the equations (1) and (2) using this.
[0032]
Returning to the flowchart about the measurement, the description will be continued. In step S110, the obtained correlation result of the left and right images is displayed on the display unit 40. In this case, as a result of searching the search data block, the correlation result (the point having the highest degree of similarity) is set as the similarity point in the data block, and the correlation result is displayed on the image.
[0033]
FIG. 8 is an explanatory diagram for displaying the correlation result. For example, if there is an area or a point having a low correlation coefficient as shown in the E1 region of FIG. 8A or FIG. 8B, they are displayed with emphasis. As the display method, the actual correlation coefficient value may be displayed for each color or the like, or a threshold value may be provided in several stages. If the displayed result is satisfactory in step S120, the process ends.
[0034]
Next, an additional measurement flow will be described. In step S130, if the displayed result is not satisfactory, the process proceeds to step S150. In step S150, the surveying instrument 1 performs additional measurement on an area required for measurement, for example, an area with a low correlation coefficient. Here, if it is manual measurement, it will move to step S80.
[0035]
C. Semi-automatic measurement
Semi-automatic measurement is performed using an automatic collimation type total station. FIG. 9 shows a flowchart for semi-automatic measurement.
[0036]
In step S510, for example, as shown in FIG. 8, the E1 area where an area with a low correlation coefficient is displayed is displayed with a pointing device (a cursor for a notebook computer or a pen for a pen computer). Point to. In step S520, the x and y coordinate position information of the instructed point is transferred from the personal computer to the surveying instrument 1, and the surveying instrument 1 is driven by a motor so as to measure the position. The measurement instruction is automatically sent from the personal computer to the surveying instrument 1. Alternatively, the measurement instruction may be issued by a person confirming the position of the surveying instrument. If the point to be measured cannot be measured, another point that can be measured in the area is searched for and measured. Compared with manual measurement, the semi-automatic measurement work using these images can omit the procedure of performing measurement while collimating the surveying instrument, so that considerable work efficiency can be achieved. That is, an image that can be collimated by the surveying instrument shows only a local region of the object, and it is very difficult and time-consuming to collimate at a place where additional measurement is desired. However, if you simply click on the image, it is a simple task that anyone can do.
[0037]
Next, the process proceeds to step S80. In step S80, additional measurement points are added to create a triangle. FIG. 10 is an explanatory diagram for adding measurement points. That is, the measurement point is connected again by the additional point C7 shown in FIG. In this case, more detailed triangles may be created for the additional points as shown, or all triangles may be recreated once again regardless of the previous connection points.
[0038]
If no additional measurement is performed or there is no selectable point in step 150, the triangle is rearranged. FIG. 11 is an explanatory diagram for rearranging triangles. For example, when a triangle is created as shown in FIG. 11A, it can be rearranged as shown in FIG. In addition to this, if the points measured linearly in step S150 are additionally selected, triangles are created accordingly, and both reliability and density are higher.
[0039]
Thereafter, the processing after step S91 is executed in the same manner as described above, and image correlation processing is performed for a new region in step S100. In this case, when a more detailed triangle is created, image correlation processing is performed only on the newly added triangle region. In step S110, the correlation result is displayed on the display unit. If the result is good in step S120, the process ends. If the displayed result is not satisfactory in step S130, the process proceeds to additional measurement again. In step S150, an additional measurement is performed for an area where additional measurement is desired while viewing the monitor. The following procedure is the same as that after step S80 and is repeated until satisfactory.
[0040]
If it is not improved even if additional measurement is performed, or if additional measurement is not performed, the process proceeds to step S140. In step S140, for example, if the region E10 in FIG. 10 does not give a good result, the altitude value is interpolated from the plane equation of the plane E10 composed of the measurement points C1, C2, and C7. For example, when C8 in the E10 region is to be obtained, the coordinate value of C8 is set to (x ₈ , Y ₈ , Z ₈ ), The elevation value z of C8 ₈ Is
z ₈ =-(Ax ₈ + By ₈ + D) / c --- (6)
Calculated by (The coefficients a, b, c, and d are calculated from the three-dimensional coordinate values of C1, C2, and C7).
[0041]
D. Automatic measurement
Automatic measurement can also be performed by using an automatic collimation type total station for the surveying instrument.
The automatic measurement will be described below.
[0042]
In step S150, the measurement can be automated by performing the additional measurement according to the following procedure. After performing the image correlation process of S100, a threshold value is set for the correlation coefficient value, and additional measurement is performed for an area including a correlation coefficient equal to or lower than the threshold value. The threshold value may be fixed, for example, 0.5 or less from the beginning, or may be determined depending on the object. The measurement area is a triangular area including a correlation coefficient equal to or less than a threshold value. For example, in FIG. 8A, the region is E1. The additional measurement point by the surveying instrument may be the center of gravity of the triangle in the triangular area with a low correlation coefficient, or may be the point with the lowest correlation coefficient in the area, or the center of the area in the low correlation coefficient area. (Centroid) may be used. The method for obtaining the center of gravity of the triangle and the center of gravity of the region having a low correlation coefficient will be described below.
[0043]
First, for example, when the correlation coefficient in the triangle E1 (C1, C2, C5) in FIG. 8A is low, the center of gravity of the triangle C7 (see FIG. 10) is used as the next measurement point. The following procedure is used. The coordinate values of C1, C2, and C5 are set to C1 (X ₁ , Y ₁ , Z ₁ ), C2 (X ₂ , Y ₂ , Z ₂ ), C5 (X ₅ , Y ₅ , Z ₅ ), And
X = (X ₁ + X ₂ + X ₅ ) / 3 --- (7)
Y = (Y ₁ + Y ₂ + Y ₅ ) / 3 --- (8)
Is the XY coordinate of the next measurement selection point C7.
[0044]
Alternatively, the center of gravity may be obtained from the distribution of the area having a low correlation coefficient shown in FIG. 8B (eg, moment method). In the method of moments, x _g , Y _g Are the X and Y coordinates of the additional measurement selection point. There may be a plurality of additional measurement selection points instead of one.
x _g = {Σx * {1-cor (x, y)}} / Σ {1-cor (x, y)}
--- (9)
y _g = {Σy * {1-cor (x, y)}} / Σ {1-cor (x, y)}
--- (10)
(X _g , Y _g ): Coordinates of the center of gravity position, cor (x, y): correlation value on (x, y) coordinates
[0045]
As described above, when the additional measurement point is determined, the surveying instrument is controlled to the coordinate value to drive and measure the surveying instrument. If these points cannot be measured and are difficult, the vicinity is searched and measured. Since the other steps are computer processing, a series of operations from steps S80 to S150 are automated.
[0046]
E. Modification by graphic display
FIG. 12 shows a flowchart for graphic display.
Next, a method for confirming and correcting the graphic display from the correlation processing result will be described. That is, it is realizable by performing step S110 in FIG. 2 as the following processes.
[0047]
First, in step S102, a triangle is created by connecting the obtained three-dimensional coordinates of each point. In step S106, a graphic display image is created based on the three-dimensional coordinates. If necessary, create a bird's-eye view with 3D coordinate points and their connections, contour lines, or a surface or image on a triangle. The figure in which the image is pasted on the triangle may be an orthographic projection image. If it is not necessary (as long as it can be visually judged), it is not necessary to perform these processes (for example, only by connecting triangles). Anything can be used as long as the three-dimensional coordinates can be visualized. These are displayed on a stereo display or a monitor of a personal computer as a graphic display image that can be appropriately determined. In step S106, the created image is displayed on the display unit 40. As a display method, a method that is easy to judge depending on the situation, such as displaying triangles in a superimposed manner or displaying correlation coefficients in a superimposed manner, may be displayed.
[0048]
Further, in the bird's-eye view image created and displayed in steps S102 to S106, additional measurement can be performed for an area that is wrong or seems unnatural. FIG. 13 is an explanatory diagram for graphic display. For example, even if only the connected triangles shown in FIG. 13A are displayed, if the actual situation is different, it is possible to determine a bad place from the display. For example, when the obtained three-dimensional coordinate value is supposed to be F1 in FIG. 13 (a), but mistakenly moves to F1 ′ in FIG. 13 (b), an error can be clearly confirmed. Further, if a surface or image is pasted and displayed in FIG. 13B, the fact that it is different from the actual one is more clearly emphasized. Furthermore, if the correlation coefficient value is superimposed and displayed, if the area with a low correlation coefficient matches the bad area, the correction area becomes clearer. In this case, the area around F1 in FIG. 13B may be additionally measured manually or semi-automatically.
[0049]
Thus, if it is manual measurement, it will move to step S80. On the other hand, if the automatic collimation type total station is used, semi-automatic measurement is possible, and the process proceeds to step S110. In the semi-automatic measurement, as described above, for example, as shown in FIG. 13, the F1 area that is wrong or seems unnatural is displayed with a cursor if it is a notebook computer, and with a pen if it is a pen computer. By pointing, the subsequent processing is executed.
[0050]
F. Search area setting method
Next, setting and scanning of the search area, the reference data block, and the search data block by the search area setting unit 20 will be described. The search area setting method is exemplified as follows and will be described in order.
1. Search area setting: Inclusion rectangle
2. Set each data block from the size of the search area
3. Set the position and movement step of each data block according to the distance from the reference point
4). Set the size of each data block according to the distance from the reference point
5). Set the size of each data block from the correlation value of the reference point
[0051]
1. Search area setting: Inclusion rectangle
FIG. 14 is an explanatory diagram (1) for setting the search area.
Basically, since the relative orientation is performed in step S30 in the search area of the left and right images, the vertical parallax is substantially removed, and the points CL1 and CR1 of the left and right images in the figure are CL2, CR2 on the epipolar line P1. Is on P2, and CL3 and CR3 are on P3. As shown in the figure, when the triangles created from the points measured by the surveying instrument are CL1, CL2, CL3 on the left image and CR1, CR2, CR3 on the right image, the search area is a quadrangle that includes each triangle. This is an area. Here, for example, a case will be described in which the left image T in the region is used as a reference data block (template) and a position corresponding to this image is to be searched from the right image. In this case, if the vertical parallax is removed with the right image including the search data block as the search area S, the search is performed on the epipolar line P3. If the removal of vertical parallax is incomplete, the vicinity of the corresponding epipolar line is also searched. These processes are performed for each line in the included square area including each reference point. If the setting is made as shown in the figure, an overlapping area is formed with an adjacent triangle, but a triangular area other than the reference point can be definitely searched.
[0052]
Here, FIG. 15 shows an explanatory diagram of the search area. Although the area inside the triangle of FIG. 14 may be used as the search area, it is uncertain whether corresponding points other than the reference point are within the triangle area. Therefore, as shown in FIG. 15, it is efficient to set a region including a triangular region as a search region.
[0053]
FIG. 16 is an explanatory diagram (2) for setting a search area.
As shown in FIG. 16, when searching for the position of the search data block corresponding to each reference data block (template) T1, T2, search areas S1, S2 for each template overlapping the search area on the line P3 are displayed. It may be provided. Here, as an example, search areas S1 and S2 are areas within a predetermined range from the centers of the reference data blocks T1 and T2. In this way, obviously, the area closer to each reference point can be efficiently searched, and the reliability can be improved by overlapping. A plurality of search areas may be set for each template. Further, the search area in the vicinity of the corresponding points CR1, CR2, and CR3 of the right image of the reference point may be small and may be set larger as the distance increases (a minimum value or a maximum value may be set as appropriate). Further, the size of the search area (search area) for the data block in the search area may be appropriately determined according to the position, movement step, and size of each data block described later. This makes it possible to efficiently search for a range including a triangle.
[0054]
2. Set each data block from the size of the search area
FIG. 17 is an explanatory diagram for determining the size of the search area.
As shown in FIG. 17A, since the size of each triangle that is a search area differs depending on the measurement point, the data block has a size suitable for each triangle area. For example, each data block is determined based on the inclusion rectangle (see FIG. 14) set as the search area as described above. The size of 1 / k is a data block for the circumscribed rectangle. The coefficient k may be fixed in advance, or may be determined as appropriate depending on the accuracy desired and the size of the area. As another example, as shown in FIG. 17B, the radius r of the inscribed circle of the triangle may be obtained, and r or 1/2, 1/3, etc. of r may be used as the data block size. r is obtained by the following equation.
r = √ {(so) (s−p) (s−q) / s} −−− (11)
However, s = 1/2 (o + p + q) --- (12)
o, p, and q are the lengths of each of the three sides of the triangle, and these are calculated from the three measurement coordinates.
The method for obtaining the data block size is not limited to this, and any data block size may be used as long as it corresponds to the size of each triangle.
[0055]
3. Set the position and movement step of each data block according to the distance from the reference point
FIG. 18 is an explanatory diagram (3) for setting the search area.
[0056]
3.1 Location of search data block relative to reference data block
As shown in FIG. 18A, for example, when T1 on the left image is used as a template (reference data block) and the same image is to be searched from the right image, the position of T1 is set to the reference points CL1 and CL3. In accordance with the distances a and b from the right image, the search is performed within a predetermined range on the line with the point S1 allocated to CR1 and CR3 to a and b as the center of the search area on the epipolar line P3 from the right image. By doing so, a search in the vicinity of the S1 region estimated as the corresponding point of T1 can be performed, so that the processing can be performed efficiently and in a short time.
[0057]
3.2 Position in the search area and moving step
For example, as shown in FIG. 18 (b), the search is performed more efficiently by making the movement step rougher toward the both sides from the center of the search area set in 3.1. That is, the search is performed more efficiently by making the area near the center of the estimated area denser and rougher as the distance increases. The ratio of roughening may be set simply by a coefficient, or may be changed in proportion to the distance from the center of gravity of the triangle (see 3.2.1) or the ratio closer to the two reference points. (See 3.2.2).
[0058]
3.2.1 Triangular center of gravity method
A method for determining the position and movement step of each data block from the center of gravity of the triangle will be described. FIG. 19 is an explanatory diagram of the center of gravity of the triangle and the data block.
As shown in FIG. 19A, the center of gravity G of the triangle is equidistant from each reference point, but the distance from the point T other than the center of gravity to the reference point is more than the distance from the center of gravity to the reference point. There is a shorter distance. As a result, the position of the center of gravity is set as the maximum size of the data block, and the step is made smaller as it approaches each reference point. When C1 (x1, y1), C2 (x2, y2), C3 (x3, y3) and T (x, y) are set, the distance from T of each point is as follows.
T-C1: L1 = √ {(x−x1) ² + (Y-y1) ² } --- (13)
T-C2: L2 = √ {(x−x2) ² + (Y−y2) ² } --- (14)
T-C3: L3 = √ {(x−x3) ² + (Y−y3) ² } --- (15)
The distance from the center of gravity G of each point is
G-C1 = G-C2 = G-C3 = Lg
= √ {(xg−x1) ² + (Yg-y1) ² } --- (16)
The smallest one of the distances L1, L2, and L3 obtained by the equations (13) to (15) is obtained, the ratio of the distance to the center of gravity is obtained, and the movement step is determined. For example, if the step size at the center of gravity position G is the maximum, SGT and the minimum step size is SMIN, the step size of the point at the minimum distance L (L2 in FIG. 19A) from the template position is
Step size = L / Lg × (SGT−SMIN) + SMIN −−− (17)
And
[0059]
3.2.2 Distance method from the ratio of the two reference points of the close method:
Proportional distribution is performed according to the distance from each reference point to T (two closest points), and the step size is variable according to the distance to be searched. For example, it is determined from the ratio of the smaller two distances in the equations (13) to (15). As shown in FIG. 19A, when L2 <L3 <L1, the point where L2 = L3 is set as the maximum step width SMAX.
Step = L2 / L3 × SMAX (18)
And it is sufficient.
[0060]
It should be noted that the method for determining the position in the search area and the movement step is not particular to this method.
4). Set the size of each data block and search area according to the distance from the reference point
4.1 Obtain the distance of the image position to be a data block from each reference point, and make the data block size variable according to the distance ratio. As shown in FIG. 19B, the center of gravity G of the triangle is equidistant from each reference point, but the distance from the point T other than the center of gravity to the reference point is more than the distance from the center of gravity to the reference point. There is a shorter distance. As a result, the data block at the center of gravity is set to the maximum size, and the area is reduced as it approaches each reference point. When C1 (x1, y1), C2 (x2, y2), C3 (x3, y3) and T (x, y), the distance from T of each point is obtained from the equations (13) to (15). The same is required. Moreover, the distance from the gravity center position G of each point is calculated | required from Formula (16). Then, the smallest one of the distances L1, L2, and L3 is found, the ratio of the distance to the center of gravity is obtained, and the data block size is determined.
For example, if the data block size at the center of gravity G is GT, and the data block size at the point of the minimum distance L (L2 in FIG. 19B) from the data block position is CT (C2 data block in FIG. 19B). ,
Data block size = L / Lg × (GT−CT) + CT −−− (19)
And
[0061]
Further, the vertical and horizontal widths of the data block may be constant or only in the horizontal direction without changing the vertical direction if the vertical parallax has already been substantially removed by the orientation processing. In this way, the data block is small in the vicinity of the reference point on the assumption that the corresponding point is near, and it becomes uncertain whether it is near as you move farther away. A corresponding point can be obtained.
[0062]
Note that the method of determining the data block size is not particular to this method. This makes it possible to search efficiently and quickly.
[0063]
5). Set the size of each data block from the correlation value of the reference point
FIG. 20 is an explanatory diagram for the size of the data block.
Since the vertex (reference point) of each triangle is a measured point, it has already been associated. Accordingly, correlation coefficients for various template (reference data block) sizes at these points are obtained (as shown in FIG. 20A), and the template showing the highest correlation value is used as the template image. An area close to each reference point is searched for in the obtained template size image. The template size obtained at each reference point is changed according to the search position.
[0064]
For example, as an example, proportional distribution is performed according to the size obtained at each reference point and the distance from each reference point (two closest points), and the size is variable according to the distance to be searched (FIG. 20B). )reference). When C1 (x1, y1), C2 (x2, y2), and C3 (x3, y3) are used as a template T (x, y), the template size to be obtained is obtained from equations (13) to (15). Therefore, it is determined from the ratio of the two smaller distances obtained. For example, as shown in FIG. 20B, when L2 <L3 <L1, the template size of CL2 and CL3 is divided by the ratio of L2: L3, and the size of T is set.
[0065]
It should be noted that the vertical and horizontal widths of the data block may be constant or only in the horizontal direction without changing the vertical direction if vertical parallax has already been substantially eliminated by the orientation processing.
[0066]
By performing image correlation processing based on triangles generated from reference points as in the present invention, it is not necessary to prepare many images with different resolutions as in the sparse search image correlation method, resulting in high speed and reliability. High image correlation processing can be performed. In addition, since the combined measurement by the surveying instrument can be performed, the measurement is reliable and reliable, and it is possible to quickly and easily measure a place where the measurement has conventionally been time-consuming and could not be performed.
[0067]
【The invention's effect】
According to the present invention, as described above, by using the non-prism TS and image correlation processing together, even for areas that could not be measured conventionally, such as places that cannot be entered or dangerous places, It is possible to perform non-contact measurement, and the entire measurement can be further increased in speed, efficiency, and reliability as compared with the case where each method is used alone.
[Brief description of the drawings]
FIG. 1 is an overall block diagram of a measuring apparatus according to the present invention.
FIG. 2 is a flowchart for online measurement.
FIG. 3 is an explanatory diagram of a stereo image.
FIG. 4 is a diagram for explaining the creation of a triangle based on measurement points.
FIG. 5 is an explanatory diagram of an input image and a template image.
FIG. 6 is a diagram showing an example of 3 × 3 pixels.
FIG. 7 is an explanatory diagram of a stereo method.
FIG. 8 is an explanatory diagram for displaying a correlation result.
FIG. 9 is a flowchart for semi-automatic measurement.
FIG. 10 is an explanatory diagram regarding addition of measurement points.
FIG. 11 is an explanatory diagram for rearranging triangles.
FIG. 12 is a flowchart for graphic display.
FIG. 13 is an explanatory diagram of graphic display.
FIG. 14 is an explanatory diagram (1) for setting a search area.
FIG. 15 is an explanatory diagram of a search area.
FIG. 16 is an explanatory diagram (2) for setting a search area;
FIG. 17 is an explanatory diagram regarding a predetermined size of a search area.
FIG. 18 is an explanatory diagram (3) for setting a search area.
FIG. 19 is an explanatory diagram of a centroid of a triangle and a data block.
FIG. 20 is an explanatory diagram regarding the size of a data block.
[Explanation of symbols]
1 surveying instrument
2 Measuring equipment
3 Camera
10 Reference point search section
20 Search area setting section
30 Calculation unit
40 display section
50 Measuring unit
60 bus

Claims (11)

For a stereo image containing at least three measurement points for which position data is known, at least a part of the measurement points is defined as a segment point, and at least three segments among a plurality of defined segment points A setting unit for setting a search area based on points;
Based on the search region set by the setting unit, a calculation unit that performs correlation processing on images of corresponding search regions on each stereo image;
A measurement unit that measures the coordinates of a point at an arbitrary position from the correlation result by the calculation unit,
The arithmetic unit uses a region having a low correlation coefficient as a measurement region that requires a new measurement point in accordance with a result of the correlation process, and creates a position information of the measurement region. . The stereo image measurement device according to claim 1,
Furthermore, it has a display unit for displaying a stereo image,
The display unit performs a predetermined display in a region where additional measurement is required according to information on the position of the measurement region created by the calculation unit. The stereo image measurement device according to claim 2,
The display unit is configured to graphically display an area that requires additional measurement on a stereo image,
A stereo system configured to receive position data when the position data of a new measurement point in the area is measured by an external surveying device based on the graphic display displayed on the display unit. Image measuring device. The stereo image measuring device according to claim 1 or 2,
The measurement unit outputs the information of the measurement area created by the calculation unit to the surveying instrument with an automatic collimation function, causes the surveying instrument to measure the position of a new measurement point in the area indicated by the area data, and A stereo image measuring apparatus configured to receive measured position data. The stereo image measuring device according to claim 1,
The setting unit sets a new search region on the stereo image after selecting a measurement point in a region requiring detailed division as a new division point according to the measurement region information created by the calculation unit. And
A stereo image measuring apparatus, wherein the arithmetic unit is configured to perform a correlation process on images of the new search areas. The stereo image measuring device according to claim 1,
The setting unit sets an inclusion quadrangle including a triangle formed by three adjacent points among the obtained segment points or measurement points as a search area in each stereo image measurement device. The stereo image measuring device according to claim 1,
The setting unit provides a reference data block in one search area of the stereo image and a search data block in the other search area of the stereo image, and the position or movement of each data block according to the distance from the dividing point. A stereo image measuring device configured to set a step. The stereo image measuring device according to claim 1,
The setting unit provides a reference data block in one search area of the stereo image and a search data block in the other search area of the stereo image, and sets the size of each data block according to the distance from the dividing point. A stereo image measuring device configured to be set. The stereo image measuring device according to claim 1,
The setting unit provides a reference data block in one search area of the stereo image, a search data block in the search area of the stereo image, sets a plurality of data blocks in the vicinity of the dividing points, and obtains a correlation result. And a stereo image measuring apparatus configured to determine the size of each data block according to the correlation result. The stereo image measuring device according to claim 1,
The setting unit provides a reference data block in one search area of the stereo image and a search data block in the other search area of the stereo image, and determines the size of each data block according to the size of the search area. A stereo image measuring device configured to determine a stereo image measuring device. The stereo image measuring device according to any one of claims 1 to 10,
The setting unit sets a data block smaller than the area based on the set search area,
The calculation unit uses a block corresponding to a data block of one stereo image as a template,
Scan the other stereo image in the same vertical position as the template,
A stereo image measuring apparatus, wherein a data block corresponding to the template is searched based on the calculated correlation value.

Images