JP4750197B2 - Endoscope device - Google Patents

Description

  The present invention relates to an endoscope apparatus that enables objective determination of a cross-sectional shape at a desired cutting position based on image signals obtained from a plurality of viewpoints.

  In recent years, endoscopes have been widely used in the medical field and the industrial field. In an observation image with a normal endoscope, the object is generally planar, and it is difficult to recognize irregularities and the like.

  For this reason, Patent Document 1 discloses a measurement endoscope apparatus that obtains images from a plurality of viewpoints and performs three-dimensional measurement. Further, Patent Document 2 displays a distance from the reference plane to the measurement point by designating the reference plane and the measurement point, and can be used to objectively recognize the height or depth of the unevenness of the object. A mirror device is shown.

  Conventional measuring endoscope apparatuses can measure three-dimensional coordinates of a point designated on a reference image, a distance between two points, or a depth from a set plane.

Japanese Patent Publication No. 8-12332 Japanese Patent No. 2778739

  However, in order to know the three-dimensional shape of the observation target, a large number of points are specified, and the shape is estimated based on the three-dimensional coordinates obtained from each point. There was a problem that it was difficult to do.

  There is also a method of creating a 3D model by obtaining 3D information of the entire imaging range from image information. However, in order to create a 3D model, it takes a lot of time for computation processing by a computer, which is not practical. There was a problem.

  Therefore, for example, when measuring the depth of the corroded portion inside the pipe, it takes a lot of inspection time to identify the deepest place.

  The present invention has been made in view of the above circumstances, and provides an endoscope apparatus capable of easily specifying a cut surface desired to be measured and easily grasping a cross-sectional shape of an observation target on the specified cut surface. The purpose is to provide.

An endoscope apparatus according to a first aspect of the present invention is an imaging unit that images an observation target from a plurality of viewpoints, and refers to one of the images at the plurality of viewpoints obtained by the imaging unit as a reference image and the remaining images. An endoscope apparatus having an arithmetic processing unit for performing stereo measurement as an image,
The arithmetic processing unit includes an image display unit that displays the reference image on a screen, and a cutting reference line designation unit that designates a cutting reference line that specifies a cutting position of the observation target on the image displayed on the screen. A point on the reference image corresponding to the point on the cutting reference line as a point of interest, a corresponding point search means for searching for a corresponding point on the reference image corresponding to the point of interest, and a point of interest on the reference image The cutting point, which is a three-dimensional coordinate of a point on a space mapped to each point of interest on the cutting reference line from the position of the point and the position of the corresponding point on the reference image obtained by the corresponding point searching means Cross-section information calculation means for obtaining cross-section information of the observation object at a position, and cross-section information output means for outputting cross-section information based on a value obtained by the cross-section information calculation means, the cross-section information output means , The standard of the screen The cross-section information based on the cutting reference line displayed on the image, to be displayed on said image display means as a cross-sectional shape outline view on the screen, the image display means, the reference image and the cutting reference line displays one that is divided into two parts in the previous SL screen and characterized by displaying the cross-sectional shape outline view to the other on the two split the screen.
The endoscope apparatus according to the second aspect of the present invention has an imaging unit that images an observation target from a plurality of viewpoints, and refers to one of the images at the plurality of viewpoints obtained by the imaging unit as a reference image and the remaining images. An endoscope apparatus having an arithmetic processing unit for performing stereo measurement as an image,
The arithmetic processing unit includes an image display unit that displays the reference image on a screen, and a cutting reference line designation unit that designates a cutting reference line that specifies a cutting position of the observation target on the image displayed on the screen. A point on the reference image corresponding to the point on the cutting reference line as a point of interest, a corresponding point search means for searching for a corresponding point on the reference image corresponding to the point of interest, and a point of interest on the reference image The cutting point, which is a three-dimensional coordinate of a point on a space mapped to each point of interest on the cutting reference line from the position of the point and the position of the corresponding point on the reference image obtained by the corresponding point searching means Cross-section information calculation means for obtaining cross-section information of the observation object at a position, and cross-section information output means for outputting cross-section information based on a value obtained by the cross-section information calculation means, the cross-section information output means , The standard of the screen The cross-sectional information based on the cutting reference line displayed on the image is displayed on the screen as a cross-sectional shape outline on the screen, and the image display means displays the cross-sectional shape outline on the screen. It is displayed in the window displayed above.
An endoscope apparatus according to a third aspect of the present invention refers to an imaging unit that images an observation target from a plurality of viewpoints, one of the images at the plurality of viewpoints obtained by the imaging unit, and a reference to the remaining images An endoscope apparatus having an arithmetic processing unit for performing stereo measurement as an image,
The arithmetic processing unit includes an image display unit that displays the reference image on a screen, and a cutting reference line designation unit that designates a cutting reference line that specifies a cutting position of the observation target on the image displayed on the screen. A point on the reference image corresponding to the point on the cutting reference line as a point of interest, a corresponding point search means for searching for a corresponding point on the reference image corresponding to the point of interest, and a point of interest on the reference image The cutting point, which is a three-dimensional coordinate of a point on a space mapped to each point of interest on the cutting reference line from the position of the point and the position of the corresponding point on the reference image obtained by the corresponding point searching means Cross-section information calculation means for obtaining cross-section information of the observation object at the position, and the cutting reference line displayed on the reference image on the screen, which outputs cross-section information based on the value obtained by the cross-section information calculation means The cross-sectional information based on State anda section information output means for displaying on the image display means as a cross-sectional shape outline view on the screen, the image display means, which can be designated the cutting reference line by the cutting reference line specifying means from the transition to features and Turkey in a state of displaying the cross-sectional shape outline view.

  According to this configuration, by specifying a desired cut surface in the displayed reference image, the cross-section information of the desired cut surface of the observer is displayed on the screen.

  ADVANTAGE OF THE INVENTION According to this invention, the endoscopic apparatus which can perform designation | designated of the cut surface which desires a measurement easily, and can grasp | ascertain easily the cross-sectional shape of the observation object in the designated cut surface is realizable.

1 to 15 relate to an embodiment of the present invention, and FIG. 1 is an explanatory diagram showing a schematic configuration of an endoscope apparatus capable of performing measurement. The figure explaining the measuring device provided with the image pick-up part and arithmetic processing part which were opened in the endoscope insertion part front-end | tip part The figure which shows the state which observes the observation object part with an endoscope Flow chart explaining stereo measurement The figure which shows an example of the endoscopic image and cutting | disconnection reference line which are displayed on the screen of a monitor by the reference | standard image display means The figure explaining a cut surface and a section outline The figure explaining an example of other reference lines The figure which shows one example of the cross-section outline which shows cross-section information The figure explaining the example of a display of the cross-sectional outline displayed with respect to the standard image The figure explaining the camera coordinate system when showing cross-section information The figure which shows the other example of the cross-section outline which shows cross-section information The figure which shows another example of the cross-section outline which shows cross-section information The figure which shows the other example of the cross-section outline which shows cross-section information Flowchart explaining corresponding point search algorithm The figure explaining the monitor screen shown after corresponding point search 16 to 22 are diagrams for explaining a device for reducing the ambiguity of association between a plurality of images, and FIG. 16 is a diagram for explaining an image displayed on a monitor screen. Flowchart explaining a method for increasing the accuracy of cross-section information using the order on the reference line A flowchart for explaining a method for improving the accuracy of cross-section information using depth information of surrounding points Flowchart explaining a method for increasing the accuracy of cross-section information by using inverse corresponding points Flowchart explaining a method for improving the accuracy of cross-section information using the magnitude of deviation from the epipolar line Flow chart explaining a method for increasing the accuracy of cross-section information using differential values Diagram showing the shape of a small area set around the point of interest The figure explaining an example of the reference line obtained by designating one point The figure explaining the other example of the reference line obtained by designating one point

Embodiments of the present invention will be described below with reference to the drawings.
As shown in FIG. 1, an endoscope apparatus 1 for performing measurement according to this embodiment includes an electronic endoscope having an imaging unit that forms an observation image of a site to be observed on an imaging element to be described later. Mirror 2 (hereinafter referred to as an endoscope) and arithmetic processing for performing image processing on an image signal of an observation image obtained by the endoscope 2 and performing various measurements on the basis of the image processed image data It is mainly comprised by the measuring device 3 which has a part.

  The measuring device 3 has a built-in light source device (not shown) for supplying illumination light to the observation target, so that an endoscopic image of the observation target portion is displayed on the screen of the monitor 30. It has become.

  The endoscope 2 has an elongated insertion portion 20, and an operation portion 21 that also serves as a gripping portion is disposed at the proximal end portion of the insertion portion 20. A universal cord 23 having a connector 22 detachably attached to the measuring device 3 at the base end extends from the side of the operation unit 21.

  The insertion portion 20 includes a distal end portion 24 incorporating an imaging optical system, which will be described later, in order from the distal end side, a bending portion 25 formed by bending a plurality of bending pieces so as to be rotatable, and an elongated flexible member. And the flexible tube portion 26 formed in the above. Reference numeral 27 denotes an operation knob 27 for bending the bending portion 25.

  As shown in FIG. 2, the distal end portion 24 includes a pair of objective lens systems 41 and 42 and an image sensor 43 that photoelectrically converts an optical image formed on the imaging surface through the objective lens systems 41 and 42 into an image signal. A configured imaging unit 40 is arranged.

  The objective lens systems 41 and 42 are configured to be able to image an observation target region from a plurality of viewpoints. Each optical image that has passed through the objective lens systems 41 and 42 on the imaging surface of the image sensor 43 is provided. Forms an image. That is, the endoscope 2 of the present embodiment is a stereoscopic endoscope that obtains a stereo image having a so-called parallax.

  As shown in FIGS. 1 and 2, the measuring device 3 is a camera control unit (hereinafter abbreviated as CCU) 31 that converts an image signal photoelectrically converted by the image sensor 43 into a video signal for display on the monitor 30. A video capture circuit 32 that converts the video signal generated by the CCU 31 into a digital image signal, and an arithmetic processing unit that performs an operation for measurement based on the digital image signal converted by the video capture circuit 32. The computer mainly includes a host computer 33 and a console 34 that operates the measuring device 3 via the host computer 33.

  As shown in FIG. 3, the illumination light generated by the light source device of the measuring device 3 is transmitted through the connector 22, the universal cable 23, and a light guide fiber (not shown) that passes through the insertion portion 20, and the illumination window 44. To the observation target region 4. As a result, an endoscopic image of the observation target region 4 is displayed on the monitor 30 of the measuring device 3.

  In the present embodiment, the console 34 is separated from the measuring device main body 35 and the monitor 30 is integrated with the measuring device main body 35. Consists of separate bodies.

With reference to FIG. 4 thru | or FIG. 24, the operation | movement and effect | action at the time of performing a stereo measurement using the endoscope apparatus 1 comprised as mentioned above are demonstrated.
For example, the insertion portion 20 of the endoscope 2 is inserted into a pipe tube, and observation of a corroded portion or a scratch is started.

  Then, first, as shown in step S101 of the flowchart of FIG. 4, an observation image of the observation target portion 4 illuminated by the illumination light is formed on the imaging surface of the imaging element 43 through the objective lens systems 41 and 42, respectively. The image signals of the respective observation images that have been imaged and photoelectrically converted on the image sensor 43 are measured via signal lines and connectors 22 (not shown) that pass through the insertion unit 20, the operation unit 21, and the universal cable 23. 3 CCUs 31.

  The image signal transmitted to the CCU 31 is generated into a video signal by the CCU 31, and then transmitted to the video capture circuit 32 to be converted into a digital image signal. As shown in step S102, the digital video signal is converted into a host computer. On the other hand, as shown in step S103, a video signal is transmitted to the monitor 30 to display an endoscopic observation image on the screen of the monitor 30 as shown in step S103.

  If a corroded part is found during monitor observation, the corroded part is measured. At this time, first, the observer operates the console 34 to switch to the stereo measurement mode. Then, as shown in FIG. 5, on the screen of the monitor 30 by the image display means in the host computer 33, for example, one objective lens among the endoscopic images of the observation images captured by the objective lens systems 41 and 42, respectively. An image captured by the system 41 is used as a standard image, and an image captured by the other objective lens system 42 is used as a reference image. The correction standard image 50 and the corrected reference image 51, which are respective measurement correction images, are divided on the monitor 30. Is displayed.

  It should be noted that the images 50 and 51 are based on the distortion correction coefficient of the imaging unit 40 obtained in advance from the digital image signal captured by the host computer 33 by the corrected image generation means as shown in step S104. It is generated as a measurement correction image subjected to distortion correction. Then, the stereo measurement such as the corresponding point search described below is performed on the correction reference image 50 and the correction reference image 51 which are the correction images for measurement.

  In order to perform the measurement, first, as shown in step S105, two points A for obtaining a cutting reference line for specifying a cutting position of a portion for which section information for measurement is to be obtained on the reference image 50 shown in FIG. B is designated through the cutting reference line designation means. This work is performed by the observer moving the pointer such as an arrow or a cursor displayed on the screen using the console 34 or the like.

  Then, when the observer designates the points A and B, a straight line shown by a dotted line connecting the points A and B connects the cutting reference line (hereinafter abbreviated as a reference line) 53 or the points A and B. A line segment indicated by a solid line becomes a cutting reference line segment (hereinafter abbreviated as a reference line segment) 54. A cut surface determined by the cutting reference line 53 will be described with reference to FIG. The image of the observation object 4 is originally connected as an inverted image to the opposite side of the observation object across the optical center. However, for easy understanding, the correction reference image 50 is an upright image between the observation object 4 and the optical center L in the drawing. In between.

  In addition to the reference line 53 specified as described above, as shown in FIG. 7, an angle (in the figure, a middle point) between the point A and the point B is orthogonal (in the figure, orthogonal). A straight line indicated by an alternate long and short dash line may be set as the auxiliary line 55 which is another reference line.

  The cutting plane 56 intended by the observer is a plane indicated by a two-dot chain line orthogonal to the correction reference image 50 in the drawing. The cut surface 56 is defined as a plane including the reference line 53 and the optical center L of the optical system that captures the reference image.

  The points A1 and B1 shown in the figure are the mapping points of the points A and B on the correction reference image 50 onto the surface of the observation target part 4. That is, the cross-sectional outline 57 that is a display target in the present embodiment is defined as a shared line between the cut surface 56 and the surface when the observation target site 4 is viewed from the line-of-sight direction.

  Accordingly, the optical axis of the reference image optical system is projected as a point on the correction reference image 50, and that point is the image center O of the correction reference image 50. Reference numeral 58 denotes an optical axis, which is represented as a one-dot chain line connecting the image center O and the optical center L in FIG.

  As shown in step S106, all pixels on the reference line 53 (or reference line segment 54) on the correction reference image 50, which is a correction image for measurement, are set as attention points in order to obtain cross-sectional information on the cut surface 56. Thus, in step S107, corresponding point search means searches for corresponding points that are pixels on the corrected reference image 51 corresponding to the attention point, and renders a corresponding point group 59 described later. The corresponding point search algorithm will be described later with reference to a flowchart of FIG. 14 and FIG.

  If a corresponding point is found in the corresponding point search in step S107, the process proceeds to step S108, and the difference between the position of the target point on the correction reference image 50 and the position of the corresponding point on the correction reference image 51, In other words, the parallax for each attention point is obtained, the process proceeds to step S109, and the obtained parallax and the baseline length that is the distance between the optical centers of each optical system obtained in advance and the focal length of each optical system, Based on the optical data such as the coordinates of the mapping point on the correction image for measurement of the optical axis of each optical system, the three-dimensional coordinates of the point on the space where the point of interest is mapped are calculated by the cross-section information calculation means.

  If the three-dimensional coordinates of the points corresponding to all the pixels on the reference line 53 are obtained in step S109, the process proceeds to step S110, and the cross-sectional information of the cut surface 56 based on the three-dimensional coordinates of each point is as follows. The monitor 30 is configured as desired by the observer from among the four patterns shown, and the process proceeds to step S111, where the cross-section information that allows the cross-section information output means to easily grasp the shape of the cut surface to be observed is displayed as a cross-section outline diagram 30 Is displayed on the screen. As a method of displaying the cross-sectional outline diagram on the screen of the monitor 30, only the cross-sectional outline diagram is displayed instead of the corrected reference image 51 in the screen, or another window is displayed on the screen. In this case, there is a method of displaying in this window or a method of displaying the correction reference image 50 in a superimposed manner.

Here, the cross-section information and the corresponding point search algorithm will be described.
First, cross-sectional information will be described. There are the following four display methods for the cross-section information.

(1) Display the section information directly on the cut surface:
The cross-sectional information is displayed as the cross-sectional shape outline 57a shown in FIG. 8, where the vertical axis is an orthogonal projection of the reference line segment 54 or the reference line 53 onto the cut surface 56, and the horizontal axis is the cut surface of the optical axis 58. In this case, a cross-sectional outline diagram is linearly obtained.

  In order to display the cross-sectional outline 57a on the screen of the monitor 30 during measurement, another window is opened on the monitor and the cross-sectional outline 57a is displayed in the window, as shown in FIG. As described above, the reference image 50 displayed on the monitor 30 may be superimposed on the cross-sectional outline 57a. In the case where the images are superimposed and displayed, an arrow C or the like serving as a mark indicating the line-of-sight direction is displayed as a clue to know the unevenness.

(2) Projecting and displaying cross-sectional information on a plane in space:
As shown in FIG. 10, the optical center L of the optical system that captures the reference image 50 is the origin, and toward the observation target surface 52, the horizontal rightward direction is the x axis, the vertical upward direction is the y axis, and the depth direction is z. The camera coordinate system taken on the axis is formed and projected onto the (xz) plane shown in FIG. 11A or the (yz) plane shown in FIG. indicate. In this display, since the same projection method as that used in normal drawings can be selected, there is an advantage that an observer familiar with the projection method in the drawings can easily grasp the shape. An arbitrary coordinate axis and an arbitrary projection plane may be set depending on the application.

(3) Display cross-sectional information in pseudo three-dimensional:
As shown in FIG. 12, a viewpoint different from the viewpoint from the endoscope 2 is newly set in the real space, and a sectional shape outline 57d viewed from the set different viewpoint is displayed. The cylindrical shape shown in the figure shows the position of the endoscope 2 in space for reference. It is also possible to simultaneously display an image showing the position of the viewpoint in three dimensions on a separate screen. In this display method, the three-dimensional shape of the observation target can be intuitively grasped by sequentially moving the viewpoint and following the change in appearance.

(4) Display by extracting depth information from the reference line in the real space As shown in FIGS. 8 and 13, the reference line segment 54 or the point D on the reference line 54 is mapped onto the surface of the observation object 4. The distance between the point D 1 and the point D 2 mapped on the straight line A 1 B 1 connecting the points A 1 and B 1, which are the mapping points of the points A and B to the real space, is a reference line segment 54 on the corrected reference image 50. On the other hand, it is shown in the vertical direction. By setting the point D to be all the points on the reference line segment 54 or the reference line 53, the cross-sectional shape outline 57e is displayed. In this display method as well, the arrow C indicating the direction of the line of sight is displayed as a clue to know the unevenness as in the description in (1). As a clue, the cross-sectional outline 57e may be set to be colored according to the length of the line segment D1 D2. With this display method, the result can be displayed compactly even in an image having a very large depth difference.

Next, the corresponding point search algorithm will be described.
All search for corresponding points is performed on the correction image for measurement. The search for corresponding points is known as template matching or window matching, and uses an algorithm for searching for corresponding points with the help of information about points around the point of interest.

  A straight line in space projected as a point on the correction reference image 50 is projected as a straight line on the correction reference image 51. This straight line is called an epipolar line. Corresponding points on the corrected reference image 51 with respect to the attention point on the correction standard image 50 theoretically exist only on the epipoline. Therefore, the search for the corresponding points may be performed in the range of the epipolar line and several pixels above and below it with the error in mind. Therefore, the search procedure is as shown in the flowchart of FIG.

  First, when the search for corresponding points shown in step S107 is started, a region P1 is set around the point of interest as shown in step S201. In step S202, an epipolar line corresponding to the point of interest is calculated. In step S203, points within a range of several pixels above and below the epipolar line are extracted as candidate points of corresponding points.

  Next, the process proceeds to step S204, where a region P2 having the same size as the region P1 set as the target point is set around the candidate point, and as shown in step S205, the region P1 of the target point and each corresponding point The degree of correspondence is obtained by calculating normalized cross-correlation of density values with the region P2 around the candidate point or the sum of squares of differences, etc., and the degree of correspondence of all candidate points is calculated as shown in steps S206 and S207. Save the results for all candidate points. When the correspondence levels for all candidate points have been stored, the coordinates are stored with the candidate point having the highest correspondence level as the corresponding point for the target point, as shown in steps S208 and S209. If the corresponding points for all the points of interest are obtained as shown in step S210, the corresponding point search is terminated and the process proceeds to step S108.

  By performing this corresponding point search, as shown in FIG. 14, each corresponding point corresponding to each target point of the reference line segment 54 of the corrected reference image 50 is drawn, and by searching all the corresponding points, the corresponding points are determined. A corresponding point group 59 that is a set is displayed on the corrected reference image 51.

  In this way, by converting to the stereo measurement mode during observation and specifying two points on the reference image displayed on the monitor screen, a portion where the observer wants to easily observe is located. Thus, the cutting reference line can be formed.

  Further, the cross-sectional information of the cutting reference line designated by the observer is displayed on the monitor screen as a cross-sectional outline drawing desired by the observer, so that the cross-sectional shape can be easily grasped.

  As a result, if a corroded part is discovered during observation, the observation part can be specified efficiently, and the cross-sectional shape of the specified observation part can be intuitively grasped in a short time. To.

  Note that by limiting the search range of corresponding points not to the cutting reference line but to the cutting reference line segment, it is possible to further shorten the time until the result is obtained.

  In the present embodiment, the corresponding point search is performed for all the points of interest on the reference line 53 or the reference line segment 54. However, the correspondence between the reference image 50 and the reference image 51 is essentially ambiguous. There is. For this reason, for example, an accurate point is not obtained at a point where there is no density difference in the surrounding area, a point in the occlusion area that is visible in the reference image but not in the reference image, or vice versa, or a point adjacent to the occlusion area. Matching becomes difficult.

  As a result, there is a possibility that an incorrect point is set as a corresponding point. By setting the incorrect point as a corresponding point, noise is included in the position information of the displayed corresponding point group 59. This will cause the target shape to be misjudged.

Therefore, the accuracy of the cross-sectional information obtained can be improved by eliminating the points that are likely to be mistaken for correspondence or by increasing the accuracy of correspondence by a plurality of ideas described below.
Hereinafter, a device for reducing the ambiguity of association between a plurality of images will be described.

(1) A method for improving the accuracy of the cross-sectional information using the order on the reference line will be described with reference to the flowcharts of FIGS.
As shown in the flowcharts of FIGS. 16 and 17, as shown in step S301, the arrangement order of all the points of interest arranged on the reference line 53 is obtained. This order is inherent even when projected to an axis parallel to the imaging plane, which is perpendicular to the base line that connects the optical center of the optical system that captures the reference image and the optical center of the optical system that captures the reference image. It does not change. This is the same value as when projected onto the axis 62 on the reference image 51 that is orthogonal to the epipolar line 61 corresponding to the image center O of the standard image 50.

  Therefore, as shown in step S302, the projection points on all the corresponding points corresponding to all the attention points are calculated on the axis 62, while the process proceeds to step S303, and the arrangement order of the projection points on the axis 62 is obtained.

  Then, as shown in steps S304, S305, and S306, it is checked whether or not the arrangement order of the points of interest and the arrangement order of the projection points of the corresponding points are inconsistent. While the points are excluded from the attention points, the points having the same arrangement order are adopted as the attention points with respect to the corresponding points. That is, for example, for the attention points arranged in the order of a, b, c, d, e on the reference line, the projection points of the corresponding points are arranged on the axis 62 in the order of a, b, d, c, e. In this case, the attention points corresponding to the points d and c are excluded from the occurrence of contradiction in the order of arrangement.

  This process is performed between step S106 and step S107 in the flowchart shown in FIG. 4, and the processes after step S107 are calculated only for the points that have been adopted.

(2) A method for improving the accuracy of the cross-section information using the depth information of surrounding points will be described with reference to the flowchart of FIG.
First, the attention point extraction range shown in step S106 is expanded to pixels adjacent to the attention point. That is, as shown in step S401 in FIG. 18, a total of nine points including the attention point on the reference line 53 and the surrounding eight points are associated.

  Then, as shown in steps S402 and S403, among the nine depth values, the average of the three values that are the central values (the value excluding the maximum three values and the minimum three values) is used as the depth value of the target point. To process. This process becomes a kind of low-pass filter. This process is performed between step S109 and step S110 in the flowchart shown in FIG.

(3) A method for improving the accuracy of the cross-sectional information by using the inverse corresponding point will be described with reference to the flowchart of FIG.
In step S107, after a corresponding point of a certain point of interest in the correction reference image 50 is taken in the corrected reference image 51, the corresponding point is taken as shown in step S501 of FIG. Accordingly, the inverse corresponding point from the corrected reference image 51 to the corrected reference image 50 is taken.

  Then, as shown in steps S502, S503, S504, and S505, it is confirmed whether or not the reverse corresponding point corresponds to the original attention point, and a point that does not have a corresponding point is regarded as a point in which the association has failed. A process of excluding the point of interest as the point of interest is performed. This processing is performed between step S107 and step S108 shown in FIG. 4, and the processing after step S108 is calculated only for the adopted points.

(4) A method for improving the accuracy of the cross-sectional information using the magnitude of deviation from the epipolar line will be described with reference to the flowchart of FIG.
The deviation from the epipolar line of the corresponding point on the corrected reference image 51 determined by the correlation result should have a certain tendency. By utilizing this fact, as shown in step S601 of FIG. 20, the deviation amounts for all corresponding points are obtained. Then, the deviation amount obtained as shown in step S602 is statistically processed, and as shown in step S603, the tendency of deviation is obtained from the processing result and a threshold value for allowing the deviation amount is determined.

  Thereafter, as shown in steps S604, S605, S606, and S607, those that deviate from the deviation amount threshold value are excluded from the attention points as points that have failed to be associated, and points that are less than the deviation amount threshold value are adopted as attention points. Perform the process. This processing is performed between step S107 and step S108 shown in FIG. 4, and the processing after step S108 is calculated only for the adopted points.

(5) A method for improving the accuracy of the cross-sectional information using the differential value will be described with reference to the flowchart of FIG.
The association is easy to take when there is a density change around the point of interest, and difficult to take when there is no density change. Therefore, the ambiguity of the association can be reduced by selecting only the places where the density changes as the attention points.
For this reason, as shown in step S701 of FIG. 21, the value of the density change is obtained by applying a differential filter at the point of interest.

  Then, as shown in steps S702, S703, S704, and S705, it is determined whether or not the value of the density change is equal to or greater than the threshold value, and the presence or absence of the density change is detected. Sometimes adopted as a point of interest. That is, a search for corresponding points is performed only for pixels that are determined to have a density change. This processing is performed between step S106 and step S107 shown in FIG. 4, and the processing after step 107 is calculated only for the points adopted.

(6) A method of increasing the accuracy of the cross-sectional information by adjusting the shape of the region set around the attention point will be described.
When the attention point is adjacent to the occlusion area, the association is ambiguous because the occlusion area enters the search area set around the attention point. Therefore, a search is performed by using a small region in which the attention point is located at the edge of the search region, and the result of the region having the highest correspondence is set as the correspondence of the candidate point. An example of the small region is a region as shown on the right side of the boundary shown by the dotted line in FIG. 22, and a black circle mark in the drawing indicates the attention point. And the area | region shown on the left side of a boundary line is an area | region normally set.

  By obtaining the cross-sectional information by combining the six methods described above alone or in combination with any two or more, the accuracy of the obtained cross-sectional information becomes high.

  Further, instead of obtaining the cross-section information by designating two points as described above, by designating only one point on the correction reference screen 50A as a single reference point 71 as shown in FIGS. The cross-sectional information may be obtained using a straight line passing through the reference point 71 as a reference line.

The following can be set as the reference line.
(1) A straight line 72 shown in FIG. 23 that passes through the designated reference point 71 and is parallel to the horizontal direction of the screen of the monitor 30.
(2) A straight line 73 shown in FIG. 23 that passes through the designated reference point 71 and is parallel to the vertical direction of the screen of the monitor 30.
(3) Straight lines 72 and 73 shown in FIG. 23 that pass through the designated reference point 71 and are parallel to the horizontal and vertical directions of the monitor 30.
(4) A straight line 74 shown in FIG. 24 that passes through the designated reference point 71 and intersects the two straight lines 72 and 73 at an arbitrary angle.

  As described above, the reference line is determined by designating one point, whereby the operation can be further simplified, and the location where the cross-sectional shape is desired can be designated directly.

  In the present embodiment, an example in which two images are used has been described. However, even if there are three or more images, a cross-sectional display can be performed in the same manner.

  Further, as the second embodiment, the image displayed when switching to the stereo measurement mode is used as a reference image and a reference image before correction instead of the correction reference image 50 and the correction reference image 51, and a reference line is designated. You may make it perform on a reference | standard image. As a result, the process of generating the correction image for measurement can be simplified, and the time required for switching to the stereo measurement mode can be greatly reduced. Other operations and effects are the same as those in the first embodiment.

  The present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the spirit of the invention.

DESCRIPTION OF SYMBOLS 1 ... Endoscope apparatus 30 ... Monitor 35 ... Measuring device main body 40 ... Imaging part 41, 42 ... Objective-lens system 50 ... Base image 51 ... Reference image 53 ... Cutting reference line 57a ... Cross-sectional shape outline

Claims (3)

An imaging unit that images an observation target from a plurality of viewpoints, and an arithmetic processing unit for performing stereo measurement using one of the images at the plurality of viewpoints obtained by the imaging unit as a reference image and the remaining images as reference images An endoscopic device,
The arithmetic processing unit includes:
Image display means for displaying the reference image on a screen;
On the image displayed on the screen, a cutting reference line specifying means for specifying a cutting reference line for specifying the cutting position of the observation target;
A corresponding point search means for searching for a corresponding point on the reference image corresponding to the point of interest, with a point on the reference image corresponding to the point on the cutting reference line as a point of interest;
A three-dimensional space point mapped to each point of interest on the cutting reference line from the position of the point of interest on the reference image and the position of the corresponding point on the reference image obtained by the corresponding point search means Cross-section information calculating means for obtaining cross-section information of the observation object at the cutting position, which is coordinates;
Cross-section information output means for outputting cross-section information based on the value obtained by the cross-section information calculation means;
Comprising
The cross-section information output means causes the image display means to display the cross-section information based on the cutting reference line displayed on the reference image of the screen as a cross-sectional shape outline diagram on the screen,
The image display means, said reference image and displays the cutting reference line segment to one which is bisected by the previous SL screen, for displaying the cross-sectional shape outline view to the other on the two split the screen An endoscope apparatus characterized by the above. An imaging unit that images an observation target from a plurality of viewpoints, and an arithmetic processing unit for performing stereo measurement using one of the images at the plurality of viewpoints obtained by the imaging unit as a reference image and the remaining images as reference images An endoscopic device,
The arithmetic processing unit includes:
Image display means for displaying the reference image on a screen;
On the image displayed on the screen, a cutting reference line specifying means for specifying a cutting reference line for specifying the cutting position of the observation target;
A corresponding point search means for searching for a corresponding point on the reference image corresponding to the point of interest, with a point on the reference image corresponding to the point on the cutting reference line as a point of interest;
A three-dimensional space point mapped to each point of interest on the cutting reference line from the position of the point of interest on the reference image and the position of the corresponding point on the reference image obtained by the corresponding point search means Cross-section information calculating means for obtaining cross-section information of the observation object at the cutting position, which is coordinates;
Cross-section information output means for outputting cross-section information based on the value obtained by the cross-section information calculation means;
Comprising
The cross-section information output means causes the image display means to display the cross-section information based on the cutting reference line displayed on the reference image of the screen as a cross-sectional shape outline diagram on the screen,
The endoscope apparatus characterized in that the image display means displays the sectional shape outline diagram in a window displayed on the screen. An imaging unit that images an observation target from a plurality of viewpoints, and an arithmetic processing unit for performing stereo measurement using one of the images at the plurality of viewpoints obtained by the imaging unit as a reference image and the remaining images as reference images An endoscopic device,
The arithmetic processing unit includes:
Image display means for displaying the reference image on a screen;
On the image displayed on the screen, a cutting reference line specifying means for specifying a cutting reference line for specifying the cutting position of the observation target;
A corresponding point search means for searching for a corresponding point on the reference image corresponding to the point of interest, with a point on the reference image corresponding to the point on the cutting reference line as a point of interest;
A three-dimensional space point mapped to each point of interest on the cutting reference line from the position of the point of interest on the reference image and the position of the corresponding point on the reference image obtained by the corresponding point search means Cross-section information calculating means for obtaining cross-section information of the observation object at the cutting position, which is coordinates;
Cross-section information is output based on the value obtained by the cross-section information calculation means, and the cross-section information based on the cutting reference line displayed on the reference image of the screen is displayed on the screen as a cross-sectional shape outline diagram. Section information output means to be displayed on the image display means as
Comprising
The image display means, from the state to be designated the cutting reference line by the cutting reference line designating means, to migrate to a state of displaying the cross-sectional shape outline view
The endoscope apparatus according to claim and this.

Images