[go: up one dir, main page]

US20230003506A1 - Dimension measurement method and dimension measurement device - Google Patents

Dimension measurement method and dimension measurement device Download PDF

Info

Publication number
US20230003506A1
US20230003506A1 US17/942,548 US202217942548A US2023003506A1 US 20230003506 A1 US20230003506 A1 US 20230003506A1 US 202217942548 A US202217942548 A US 202217942548A US 2023003506 A1 US2023003506 A1 US 2023003506A1
Authority
US
United States
Prior art keywords
line segment
dimension
dimension measurement
lines
user
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
US17/942,548
Other languages
English (en)
Inventor
Masaki Fukuda
Toru Matsunobu
Satoshi Yoshikawa
Kensho Teranishi
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Panasonic Intellectual Property Management Co Ltd
Original Assignee
Panasonic Intellectual Property Management Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Panasonic Intellectual Property Management Co Ltd filed Critical Panasonic Intellectual Property Management Co Ltd
Assigned to PANASONIC INTELLECTUAL PROPERTY MANAGEMENT CO., LTD. reassignment PANASONIC INTELLECTUAL PROPERTY MANAGEMENT CO., LTD. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: TERANISHI, Kensho, YOSHIKAWA, SATOSHI, FUKUDA, MASAKI, MATSUNOBU, TORU
Publication of US20230003506A1 publication Critical patent/US20230003506A1/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
    • G01B11/00—Measuring arrangements characterised by the use of optical techniques
    • G01B11/02—Measuring arrangements characterised by the use of optical techniques for measuring length, width or thickness
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C11/00—Photogrammetry or videogrammetry, e.g. stereogrammetry; Photographic surveying
    • G01C11/02—Picture taking arrangements specially adapted for photogrammetry or photographic surveying, e.g. controlling overlapping of pictures
    • G01C11/025—Picture taking arrangements specially adapted for photogrammetry or photographic surveying, e.g. controlling overlapping of pictures by scanning the object
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C11/00—Photogrammetry or videogrammetry, e.g. stereogrammetry; Photographic surveying
    • G01C11/04—Interpretation of pictures
    • G01C11/30—Interpretation of pictures by triangulation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/12—Edge-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/13—Edge detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G06T7/55—Depth or shape recovery from multiple images
    • G06T7/564—Depth or shape recovery from multiple images from contours
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/761—Proximity, similarity or dissimilarity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/64—Three-dimensional objects
    • G06V20/647—Three-dimensional objects by matching two-dimensional images to three-dimensional objects
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/63—Control of cameras or camera modules by using electronic viewfinders
    • H04N23/633—Control of cameras or camera modules by using electronic viewfinders for displaying additional information relating to control or operation of the camera
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/64—Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2210/00—Indexing scheme for image generation or computer graphics
    • G06T2210/04—Architectural design, interior design

Definitions

  • the present disclosure relates to a dimension measurement method and a dimension measurement device.
  • Patent Literature (PTL) 1 discloses a method of calculating the shape of a structure by performing point cloud measurement using a multi-viewpoint image measurement method based on a plurality of images.
  • the present disclosure has as an object to provide a dimension measurement method or a dimension measuring device that can reduce the processing amount.
  • a dimension measurement method includes: extracting a plurality of lines from a plurality of images generated by shooting a target area from a plurality of viewpoints, and generating a line segment model which is a three-dimensional model of the target area that is expressed using the plurality of lines; calculating a dimension of a particular part inside the target area, using the line segment model; and outputting the dimension calculated.
  • the present disclosure is capable of providing a dimension measurement method or a dimension measuring device that is capable of reducing the processing amount.
  • FIG. 1 is a diagram illustrating an example of dimension measurement according to an embodiment.
  • FIG. 2 is a diagram illustrating an example of dimension measurement according to an embodiment.
  • FIG. 3 is a block diagram of a dimension measurement device according to an embodiment.
  • FIG. 4 is a block diagram of an imaging unit according to an embodiment.
  • FIG. 5 is a block diagram of a controller according to an embodiment.
  • FIG. 6 is a block diagram of a dimension measurer according to an embodiment.
  • FIG. 7 is a sequence diagram of a dimension measurement process according to an embodiment.
  • FIG. 8 is a diagram illustrating an example of image shooting according to an embodiment.
  • FIG. 9 is a flowchart of a line segment reconstruction process according to an embodiment.
  • FIG. 10 is a schematic diagram for describing line segment reconstruction according to an embodiment.
  • FIG. 11 is a schematic diagram for describing line segment reconstruction according to an embodiment.
  • FIG. 12 is a schematic diagram for describing line segment reconstruction according to an embodiment.
  • FIG. 13 is a diagram illustrating an example of a screen display according to an embodiment.
  • FIG. 14 is a diagram illustrating an example of a screen display according to an embodiment.
  • FIG. 15 is a diagram illustrating an example of a screen display according to an embodiment.
  • FIG. 16 is a diagram illustrating an example of a screen display according to an embodiment.
  • FIG. 17 is a diagram illustrating an example of a screen display according to an embodiment.
  • FIG. 18 is a flowchart of a dimension measurement process according to an embodiment.
  • FIG. 19 is a diagram illustrating an example of a screen display according to an embodiment.
  • FIG. 20 is a diagram illustrating an example of a screen display according to an embodiment.
  • FIG. 21 is a flowchart of a dimension measurement process according to an embodiment.
  • a dimension measurement method includes: generating, using a plurality of images generated by shooting a target area from a plurality of viewpoints, a line segment model which is a three-dimensional model of the target area that is expressed using lines; calculating a dimension of a particular part inside the target area, using the line segment model; and outputting the dimension calculated.
  • the dimension measurement method is capable of reducing the processing amount by calculating the dimension using a line segment model. For example, compared to a case where the shape of the target area is represented by a point cloud, by representing only an edge of the target area using a line, the processing amount can be reduced.
  • a dimension measurement method which uses a three-dimensional model in which, aside from edges of the target area, a plane of the target area is also represented by a point cloud, in order to perform measurement for an edge of the target area, it is necessary to extract only the point cloud representing the edge from the three-dimensional model.
  • the line segment model only includes the point cloud representing edges of the target area, and thus it is not necessary to extract the point cloud representing the edge. Furthermore, the data amount of the line segment model according to an aspect of the present disclosure is less than the data amount of a three-dimensional model in which a plane of the target area is also expressed by a point cloud.
  • the dimension measurement method may further include displaying a user interface including the line segment model, and the particular part may be determined based on lines which are included in the line segment model and specified by a user via the user interface.
  • the user by having the user select a plurality of lines from a line segment model, the dimension of the particular part which is based on the selected plurality of lines is measured. Accordingly, the user can easily measure the dimension of the desired part.
  • the dimension may be a distance between two lines selected by the user via the user interface.
  • the user by having the user select two lines from the line segment model, the distance between the selected two lines is measured. Accordingly, the user can easily measure the dimension of the desired part.
  • the dimension may be a distance between a line and a plane that are selected by the user via the user interface.
  • a dimension between the line and the plane which is based on the selected plurality of lines is measured. Accordingly, the user can easily measure the dimension of the desired part.
  • the plane may be defined by two lines selected by the user via the user interface.
  • the user can easily select a desired plane.
  • the dimension measurement method may further include presenting, to the user, a message prompting the user to shoot an entirety of a target object included in the target area, when the entirety of a target object is not included in an image that has been shot.
  • the dimension measurement method is able to generate a line segment model using an image generated by shooting the entirety of the target object, and thus is able to improve the accuracy of the line segment model. Therefore, the dimension measurement method can improve dimension measurement accuracy.
  • the lines may be a line segment.
  • the dimension measurement method is able to generate a line segment model using a line segment which has both ends, without having to use the portion that is cut off in the image, and thus is able to improve the accuracy of the line segment model.
  • a dimension measurement device includes: a processor and a memory.
  • the processor uses the memory to generate, using a plurality of images generated by shooting a target area, a line segment model which is a three-dimensional model of the target area that is expressed using lines; calculates a dimension of a particular part inside the target area, using the line segment model; and outputs the dimension calculated.
  • the dimension measurement device is capable of reducing the processing amount by calculating the dimension using a line segment model.
  • a dimension measurement device includes: an input interface, a processor, and an output interface. Images shot from a plurality of viewpoints of a target area are inputted to the input interface.
  • the processor generates a three-dimensional model that represents the target area using lines, based on the images.
  • the processor calculates the dimensions of a particular part of the three-dimensional model. The calculated dimensions are outputted from an output interface.
  • the present embodiment will describe a device and a method that can automatically measure dimensions of a target from video shot using a camera, in a mobile terminal, for example.
  • FIG. 1 and FIG. 2 are diagrams illustrating examples of a dimension measurement.
  • FIG. 1 illustrates an example of a kitchen in which, for example, height L 1 of a kitchen counter and distance L 2 from the floor to an electric outlet are to be measured.
  • each of L 1 and L 2 is defined as a distance between a plane and a line.
  • L 1 is the distance between the floor (a plane) and one side of the counter (a line).
  • FIG. 2 illustrates an example of a front door area in which, for example, door undercut L 3 (i.e., the distance between the floor and the bottom side of the door), tile width L 4 , and door gap L 5 are to be measured.
  • L 3 is a distance between a line and a plane
  • L 4 and L 5 are distances between a line and a line. In this manner, in the present embodiment, distances between a line and a line or a line and a plane are measured.
  • FIG. 3 is a block diagram of dimension measurement device 100 according to the present embodiment.
  • Dimension measurement device 100 includes imaging unit 200 , controller 300 , dimension measurer 400 , and user interface 500 .
  • this dimension measurement device 100 is included in a tablet terminal or a mobile terminal such as a smart phone, etc.
  • Imaging unit 200 shoots images (a moving image or a still image). Controller 300 controls imaging unit 200 , dimension measurer 400 , and user interface 500 .
  • Dimension measurer 400 generates a line segment model which is a three-dimensional model represented by line segments, by performing line segment reconstruction using an image shot by imaging unit 200 .
  • a line segment model is a model obtained by removing shapes (for example, planes) other than line segments from a three-dimensional model.
  • dimension measurer 400 measures, using a line segment model, a dimension (distance) of a part specified by a user.
  • a three-dimensional model is a representation of a shot measurement target made on a computer.
  • the three-dimensional model for example, has position information of respective three-dimensional parts on the measurement target.
  • the line segment model may be a three-dimensional model represented using lines.
  • a line refers to a line segment which has two ends, a half-line which has only one end, a line which has no ends, or a combination thereof.
  • having no ends means that the ends are cut off in the image, for example.
  • User interface 500 receives input from a user. Furthermore, user interface 500 presents information to the user.
  • user interface 500 is a display and a touch panel. It should be noted that, user interface 500 is not limited to the above examples, and may be any user interface.
  • user interface 500 may include a microphone, a speaker, and so on.
  • FIG. 4 is a block diagram illustrating the configuration of imaging unit 200 .
  • Imaging unit 200 includes camera 201 and platform 202 .
  • Camera 201 includes storage 211 , controller 212 , optical system 213 , and image sensor 214 .
  • Storage 211 stores images shot by image sensor 214 .
  • Controller 212 controls storage 211 , optical system 213 , and image sensor 214 .
  • Optical system 213 includes a lens, and so on, for allowing light to enter image sensor 214 .
  • Image sensor 214 shoots an image.
  • Platform 202 controls the shooting direction of camera 201 .
  • FIG. 5 is a block diagram illustrating the configuration of controller 300 .
  • Controller 300 includes imaging controller 301 , UI controller 302 , dimension measurement controller 303 , and storage 304 .
  • Image controller 301 controls imaging unit 200 .
  • UI controller 302 controls user interface 500 .
  • Dimension measurement controller 303 controls dimension measurer 400 .
  • Storage 304 stores images shot by imaging unit 200 , a line segment model generated by dimension measurer 400 , and so on.
  • FIG. 6 is a block diagram illustrating the configuration of dimension measurer 400 .
  • Dimension measurer 400 includes image obtainer 401 , preprocessor 402 , line segment reconstructor 403 , plane estimator 404 , and measurer 405 .
  • Image obtainer 401 obtains images shot by imaging unit 200 .
  • Preprocessor 402 performs preprocessing on the obtained images.
  • Line segment reconstructor 403 generates a line segment model by performing line segment reconstruction using the pre-processed images.
  • Plane estimator 404 estimates a plane included in the line segment model.
  • Measurer 405 measures, using a line segment model, the distance between line segments or the distance between a line segment and a plane.
  • FIG. 7 is a sequence diagram of a dimension measurement process by dimension measurement device 100 .
  • This example is an example of a case of measuring the distance between a line segment and a plane.
  • the user performs a shooting start instruction via the user interface (S 11 ).
  • the start instruction is performed by the selection of a menu or the activation of an application, or the like, on a screen.
  • controller 300 When controller 300 receives the start instruction, controller 300 sends a shooting instruction to imaging unit 200 .
  • Imaging unit 200 shoots images (still images) in accordance with the shooting instruction (S 12 ).
  • the images (still images) obtained are two or more images of the same target object (or target space) shot from different viewpoints.
  • FIG. 8 is a diagram illustrating an example of the shooting of the images.
  • the user shoots images of the target object (for example, the kitchen) from different positions using a single imaging device (for example, a tablet terminal).
  • imaging unit 200 does not necessarily have to be included in dimension measurement device 100 , and may be included in a terminal different from the terminal in which dimension measurement device 100 is included. In this case, the images shot by imaging unit 200 are sent to dimension measurement device 100 via any communication means such as wireless communication, etc.
  • the images may be a plurality of images shot using a plurality of fixed cameras. Furthermore, the images may be images from two viewpoints shot from one position using a stereo camera. Furthermore, the images may be a plurality of frames included in a moving image shot by a single camera while moving. Furthermore, the images may be a combination of the above.
  • the shot images are sent to dimension measurer 400 via controller 300 .
  • Dimension measurer 400 generates a line segment model by performing line segment reconstruction using the images (S 13 ).
  • FIG. 9 is a flowchart of the line segment reconstruction process (S 13 ). Furthermore, FIG. 10 to FIG. 12 are schematic diagrams for describing the line segment reconstruction.
  • dimension measurer 400 detects the line segments included in the respective images (S 31 ). Next, dimension measurer 400 calculates the feature amount of each line segment (S 32 ). Next, as illustrated in FIG. 11 , dimension measurer 400 performs matching of line segments between images, using the calculated feature amounts (S 33 ). Specifically, dimension measurer 400 detects corresponding line segments which are line segments that correspond to each other (i.e., are the same) between images. Next, as illustrated in FIG. 12 , dimension measurer 400 estimates the camera parameters (three-dimensional positions and orientations) of the respective images and the three-dimensional position of the line segment, by geometrical calculation using the relationship between the corresponding lines (S 34 ).
  • dimension measurer 400 may generate, using a plurality of images, a three-dimensional model represented by point cloud data (point cloud), and generate a line segment model by detecting line segments included in the generated three-dimensional model.
  • the line segment model generated by line segment reconstruction (S 13 ) is sent to controller 300 .
  • Controller 300 generates, using the line segment model, a line segment model image of the line segment model as seen from a predetermined direction, and sends the line segment model image to user interface 500 (S 14 ).
  • FIG. 13 is a diagram illustrating an example of a screen display in the above case.
  • the user may change viewpoint positions and perform scaling.
  • the viewpoint of a line segment model displayed may be manipulable by the user.
  • a line segment model image may be displayed superimposed on an image.
  • the image may be a real time image that is currently being shot, or may be an image shot in the past.
  • the viewpoint of the line segment model displayed may be the same as the viewpoint of the image onto which the line segment model is superimposed.
  • FIG. 14 is a diagram illustrating an example of a screen display in the above case.
  • line segment A and line segment B are selected.
  • Information indicating the selected line segments is sent to dimension measurer 400 via controller 300 .
  • Dimension measurer 400 estimates, based on the information on the selected line segments, a plane that includes the selected line segments (S 16 ). For example, as illustrated in FIG. 15 , plane C (the floor) including line segment A and line segment B is detected. The information of the estimated plane is sent to controller 300 . Controller 300 superimposes the information of the plane onto the line segment model image, and outputs the image obtained to the user interface 500 (S 17 ).
  • FIG. 16 is a diagram illustrating an example of a screen display in the above case. In the example illustrated in FIG. 16 , line segment D and plane C are selected.
  • the plane estimation may be performed automatically by dimension measurer 400 using a shot image or, in the case where a point cloud model is generated, using the point cloud model. In this case, since a plurality of planes are estimated, the user selects, from the plurality of planes, the plane which is the dimension measurement target.
  • dimension measurer 400 measures, using the line segment model, the dimensions of the dimension measurement targets (S 19 ). Specifically, dimension measurer 400 measures the distance between the line segment and the plane that are selected using the line segment model.
  • the distance between the line segment and the plane is, for example, the minimum distance between the line segment and the plane.
  • FIG. 17 is a diagram illustrating an example of a screen display in the above case. As illustrated in FIG. 17 , information indicating the part for which the dimension is measured and the dimension (distance) are displayed.
  • the minimum distance between one point (for example, the center point) on the line segment and the plane may be calculated.
  • the minimum distances between respective points on the line segment and the plane may be calculated, and the average value or the median value of the calculated minimum distances may be calculated.
  • a plurality values out of the smallest value, the biggest value, the average value, and the median value of the minimum distances may be calculated, and the calculated values may be presented to the user.
  • the fact that the line segment and the plane are not parallel may be notified to the user.
  • a message, or the like that prompts operation by the user or indicates the details of the operation may be displayed.
  • any method such as icon displays or voice output, etc., may be used.
  • the line segment which is the dimension measurement target may be selected first, and the plane estimation (S 15 to S 17 ) performed thereafter.
  • the distance between two line segments may be measured.
  • the two line segments which are the dimension measurement targets are selected in S 18 without performing S 15 to S 17 .
  • the distance between two planes may be measured.
  • two planes are estimated by performing S 15 to S 17 twice.
  • a point and a line segment or a point and a plane may be specified by the user, and the distance between the point and the line segment or the distance between the point and the plane may be measured.
  • dimension measurement device 100 may have any one or a plurality functions from among a function of measuring the distance between a line segment and a plane, a function of measuring the distance between a line segment and a line segment, or other functions (e.g., function of measuring the distance between a plane and a plane, a point and a line segment, or a point and a plane).
  • the function to be used may be specified by the user at a predetermined timing (for example, at the start of S 11 or S 15 , etc.), and may be specified by the user at an arbitrary timing.
  • the functions may be switched automatically according to the result of the user's selection of a line segment or a plane. For example, when two line segments are selected as dimension measurement targets by the user, the distance between the two line segments may be measured, and when a line segment and a plane are selected as dimension measurement targets by the user, the distance between the line segment and the plane may be measured.
  • FIG. 18 is a flowchart of the dimension measurement process of dimension measurer 400 .
  • image obtainer 401 obtains images shot by imaging unit 200 (S 41 ).
  • preprocessor 402 performs preprocessing on the obtained images (S 42 ).
  • Preprocessing is, for example, brightness adjustment, noise removal, resolution conversion, color space conversion, lens distortion correction, projective transformation, affine transformation, edge enhancement, trimming, or a combination thereof. It should be noted that the preprocessing may be performed in line with the timing at which the dimension measurement is performed, or may be performed in advance.
  • the preprocessed images obtained through the performance of image preprocessing by preprocessor 402 may be stored in storage 304 included in controller 300 . It should be noted that the preprocessing by preprocessor 402 need not be executed. For this reason, dimension measurer 400 need not include preprocessor 402 .
  • line segment reconstructor 403 performs line segment reconstruction for calculating the three-dimensional shape of objects in a predetermined space, using images shot by imaging unit 200 (S 43 ). Specifically, for each of the images, line segment reconstructor 403 detects line segments, performs inter-image correspondence, and calculates, by geometrical calculation using the correspondence relationship, a line segment model consisting of the three-dimensional line segments of the predetermined space.
  • Plane estimator 404 estimates a three-dimensional plane using the line segment model. First, in order to estimate a plane, plane estimator 404 selects line segments included in the plane (S 44 ). For example, these line segments are selected based on a user operation. It should be noted that plane estimator 404 may perform this selection automatically. Next, plane estimator 404 estimates a plane including the selected line segments (S 45 ). It should be noted that the plane estimation process by plane estimator 404 need not be performed. For this reason, dimension measurer 400 need not include plane estimator 404 .
  • measurer 405 selects two line segments, or a line segment and a plane, which are the dimension measurement targets (S 46 ). For example, this selection is performed based on a user operation. It should be noted that measurer 405 may perform this selection automatically.
  • measurer 405 calculates the distance between the two line segments, or between the line segment and the plane, that were selected (S 47 ). Furthermore, the calculated distance is, for example, displayed on user interface 500 .
  • dimension measurement device 100 may instruct the user to perform image shooting again. For example, dimension measurement device 100 detects an edge inside the image, and determines that the edge (i.e., the target object) is cut off when the detected edge continues up to a border of the image.
  • FIG. 19 is a diagram illustrating an example of a screen display in which the entirety of a target object (i.e., a kitchen counter) does not fit within the screen display.
  • FIG. 20 is a diagram illustrating an example of a screen display on which a message is displayed in the above case. It should be noted that such a message may be displayed after shooting of a still image, or may be displayed in a state where real-time moving images are displayed on the monitor prior to shooting a still image, or may be displayed during shooting of a moving image. Furthermore, such a warning to the user may be performed by icon display or by voice.
  • dimension measurement device 100 may select, from a plurality of images, two or more images showing the entirety of the target object, and line segment reconstruction may be performed using the two or more images. In other words, dimension measurement device 100 may perform the line segment reconstruction without using an image in which the target object is cut off.
  • the dimension measurement device performs the processes illustrated in FIG. 21 .
  • the dimension measurement device generates, using a plurality of images generated by shooting a target area from a plurality of viewpoints, a line segment model which is a three-dimensional model of the target area represented by lines (S 51 ).
  • the dimension measurement device calculates a dimension of a particular part inside the target area, using the line segment model (S 52 ).
  • the dimension measurement device outputs the dimension calculated (S 53 ). For example, dimension measurement device 100 presents to the user, or outputs to another device, the calculated dimension.
  • the dimension measurement device is capable of reducing the processing amount by calculating the dimension using a line segment model. Furthermore, the selection operation by a user can be done easily compared to the case where the user is required to select two points, for example. Specifically, in order to accurately measure height L 1 of the kitchen counter illustrated in FIG. 1 by specifying two points, the user needs to specify the nearest two points of the floor and the top surface of the kitchen counter. In contrast, in the case of specifying a line and a line or a line and a plane as in the present embodiment, the shortest distance between the line and the line or the line and the plane is automatically measured, and thus the selection operation by the user is simplified.
  • the dimension measurement device further displays a user interface including the line segment model.
  • the particular part is determined based on lines which are included in the line segment model and specified by a user via the user interface. Accordingly, by having the user select lines from a line segment model, the dimension of the particular part which is based on the selected lines is measured. Accordingly, the user can easily measure the dimension of the desired part.
  • the dimension of the particular part is a distance between two lines selected by the user via the user interface. Accordingly, by having the user select two lines from the line segment model, the distance between the selected two lines is measured. Accordingly, the user can easily measure the dimension of the desired part.
  • the dimension of the particular part is the distance between a line and a plane that are selected by the user via the user interface. Accordingly, by having the user select lines from the line segment model, a dimension between the line and the plane which are based on the selected lines is measured. Accordingly, the user can easily measure the dimension of the desired part.
  • the plane selected by the user is a plane defined by two lines selected by the user via the user interface. Accordingly, the user can easily select a desired plane.
  • the dimension measurement device presents, to the user, a message prompting the user to shoot the entirety of the target object. Accordingly, the dimension measurement device is able to generate a line segment model using an image generated by shooting the entirety of the target object, and thus is able to improve the accuracy of the line segment model. Therefore, the dimension measurement device can improve dimension measurement accuracy.
  • the line is a line segment. Accordingly, the dimension measurement device is able to generate a line segment model using a line segment which has both ends, without having to use a portion that is cut off in the image, and thus is able to improve the accuracy of the line segment model.
  • the dimension measurement device includes a processor and memory, and, using the memory, the processor performs the above-described processes.
  • each of the processing units included in the dimension measurement device, and so on, according to the foregoing embodiment is implemented typically as an LSI which is an integrated circuit.
  • These processing units may be configured as individual chips or may be configured so that a part or all of the processing units are included in a single chip.
  • the method of circuit integration is not limited to LSIs, and implementation through a dedicated circuit or a general-purpose processor is also possible.
  • a field programmable gate array (FPGA) that allows for programming after the manufacture of an LSI, or a reconfigurable processor that allows for reconfiguration of the connection and the setting of circuit cells inside an LSI may be employed.
  • the respective structural components are configured using dedicated hardware
  • the respective structural components may be implemented by executing software programs suitable for the respective structural components.
  • the respective structural components may be implemented by a program executer such as a CPU or a processor reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory.
  • the present disclosure may be implemented as a dimension measurement method, and the like, performed by a dimension measurement device, and the like.
  • the separation of the function blocks in the block diagrams is merely an example, and plural function blocks may be implemented as a single function block, a single function block may be separated into plural function blocks, or part of functions of a function block may be transferred to another function block. Furthermore, the functions of function blocks having similar functions may be processed, in parallel or by time-sharing, by a single hardware or software.
  • the present disclosure is applicable to a dimension measurement device.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Geometry (AREA)
  • Software Systems (AREA)
  • Evolutionary Computation (AREA)
  • Databases & Information Systems (AREA)
  • Computing Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Artificial Intelligence (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Graphics (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Signal Processing (AREA)
  • Length Measuring Devices By Optical Means (AREA)
US17/942,548 2020-03-24 2022-09-12 Dimension measurement method and dimension measurement device Pending US20230003506A1 (en)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
JP2020052313 2020-03-24
JP2020-052313 2020-03-24
PCT/JP2021/006668 WO2021192782A1 (fr) 2020-03-24 2021-02-22 Procédé de mesure de dimension et dispositif de mesure de dimension

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2021/006668 Continuation WO2021192782A1 (fr) 2020-03-24 2021-02-22 Procédé de mesure de dimension et dispositif de mesure de dimension

Publications (1)

Publication Number Publication Date
US20230003506A1 true US20230003506A1 (en) 2023-01-05

Family

ID=77890228

Family Applications (1)

Application Number Title Priority Date Filing Date
US17/942,548 Pending US20230003506A1 (en) 2020-03-24 2022-09-12 Dimension measurement method and dimension measurement device

Country Status (5)

Country Link
US (1) US20230003506A1 (fr)
EP (1) EP4130648A4 (fr)
CN (1) CN115210529A (fr)
TW (1) TW202138757A (fr)
WO (1) WO2021192782A1 (fr)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114792343B (zh) * 2022-06-21 2022-09-30 阿里巴巴达摩院(杭州)科技有限公司 图像获取设备的标定方法、获取图像数据的方法、装置

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100166294A1 (en) * 2008-12-29 2010-07-01 Cognex Corporation System and method for three-dimensional alignment of objects using machine vision
US20130083990A1 (en) * 2011-10-04 2013-04-04 Metalforming, Inc. Using Videogrammetry to Fabricate Parts
WO2017056567A1 (fr) * 2015-10-01 2017-04-06 三菱電機株式会社 Dispositif de mesure de dimension
US20190066224A1 (en) * 2008-01-18 2019-02-28 Mitek Systems, Inc. Systems and methods for obtaining insurance offers using mobile image capture

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140375684A1 (en) * 2013-02-17 2014-12-25 Cherif Atia Algreatly Augmented Reality Technology
JP6054831B2 (ja) * 2013-09-03 2016-12-27 国立大学法人 東京大学 画像処理装置、画像処理方法及び画像処理プログラム
EP3489627B1 (fr) * 2017-11-24 2020-08-19 Leica Geosystems AG Conglomérats de modèles en 3d de taille réelle
JP7086643B2 (ja) 2018-03-02 2022-06-20 株式会社パスコ 建築構造物形状算出システムおよび建築構造物撮影装置

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190066224A1 (en) * 2008-01-18 2019-02-28 Mitek Systems, Inc. Systems and methods for obtaining insurance offers using mobile image capture
US20100166294A1 (en) * 2008-12-29 2010-07-01 Cognex Corporation System and method for three-dimensional alignment of objects using machine vision
US20130083990A1 (en) * 2011-10-04 2013-04-04 Metalforming, Inc. Using Videogrammetry to Fabricate Parts
WO2017056567A1 (fr) * 2015-10-01 2017-04-06 三菱電機株式会社 Dispositif de mesure de dimension

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
Liscio, Eugene. "Extracting accurate measurements from single surveillance camera images." Photomodeler 6 (2017): 15. (Year: 2017) *

Also Published As

Publication number Publication date
EP4130648A4 (fr) 2023-09-06
EP4130648A1 (fr) 2023-02-08
CN115210529A (zh) 2022-10-18
TW202138757A (zh) 2021-10-16
JPWO2021192782A1 (fr) 2021-09-30
WO2021192782A1 (fr) 2021-09-30

Similar Documents

Publication Publication Date Title
US20210344848A1 (en) Image processing apparatus, image processing system, image processing method, and storage medium
EP3373251A1 (fr) Colorisation d'un balayage à l'aide d'une caméra non étalonnée
US9129435B2 (en) Method for creating 3-D models by stitching multiple partial 3-D models
US8605993B2 (en) Methods and systems of merging depth data from a plurality of disparity maps
KR100953076B1 (ko) 객체 또는 배경 분리를 이용한 다시점 정합 방법 및 장치
US9747690B2 (en) Image processing device, image processing method, and program
WO2015142446A1 (fr) Éclairage à réalité augmentée avec géométrie dynamique
WO2019091117A1 (fr) Systèmes robotiques de balayage tridimensionnel (3d) et procédés de balayage
EP4023155A1 (fr) Dispositif de mesure du corps et procédé de commande associé
EP3998582A1 (fr) Procédé et dispositif de génération de modèles tridimensionnels
US20230003506A1 (en) Dimension measurement method and dimension measurement device
JP2008117305A (ja) 画像処理装置
EP4246452A1 (fr) Dispositif de densification de nuage de points tridimensionnels, procédé de densification de nuage de points tridimensionnels, et programme
US20140168375A1 (en) Image conversion device, camera, video system, image conversion method and recording medium recording a program
CN113344906B (zh) 车路协同中相机评估方法、装置、路侧设备和云控平台
JP2016099316A (ja) 地物異動判別方法及び地物異動判別装置並びに地物異動判別プログラム
EP4140552A1 (fr) Appareil de traitement d'images, procédé de traitement d'images et programme
JP6719945B2 (ja) 情報処理装置、情報処理方法、情報処理システム及びプログラム
JP6546898B2 (ja) ３次元空間特定装置、方法、及びプログラム
AU2019248023B2 (en) Computer implemented structural thermal audit systems and methods
JP2005141655A (ja) ３次元モデリング装置及び３次元モデリング方法
CN115499596B (zh) 一种处理图像的方法和装置
JP7710158B2 (ja) 寸法計測方法及び寸法計測装置
JP2009244229A (ja) 三次元画像処理方法、三次元画像処理装置および三次元画像処理プログラム
JP7251869B2 (ja) 距離計測装置、システム、方法、及びプログラム

Legal Events

Date Code Title Description
STPP Information on status: patent application and granting procedure in general

Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION

AS Assignment

Owner name: PANASONIC INTELLECTUAL PROPERTY MANAGEMENT CO., LTD., JAPAN

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:FUKUDA, MASAKI;MATSUNOBU, TORU;YOSHIKAWA, SATOSHI;AND OTHERS;SIGNING DATES FROM 20220824 TO 20220825;REEL/FRAME:062177/0747

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER

STPP Information on status: patent application and granting procedure in general

Free format text: FINAL REJECTION MAILED