Disclosure of Invention
In view of the above technical problems in the related art, the present invention provides a method and apparatus for measuring volume, which can measure the volume of an object quickly without using a measuring tool such as a tape measure.
In order to achieve the technical purpose, the technical scheme of the invention is realized as follows:
a method of measuring a volume, comprising the steps of: s1, acquiring an image of an object to be detected through an augmented reality module; s2, matching the image of the object to be detected with an image recognition library to obtain fixed point coordinates, wherein the fixed points can form a space graph; and S3, calculating the volume of the object to be measured according to the fixed point coordinates.
Preferably, the step S1 further includes the steps of: s11, generating a marking plane through the augmented reality module; and S12, placing the object to be detected on a marking plane, and shooting to obtain an image of the object to be detected.
Preferably, the image recognition library is obtained by training a classifier by collecting a spatial pattern.
Preferably, in step S2, the image of the object is transferred to the image recognition library in a byte or Bitmap manner for matching.
Preferably, the spatial pattern is a cube.
Preferably, the step S3 further includes the steps of: s31, calculating coordinates of image acquisition points through the augmented reality module; s32, mapping the fixed point of the object to be detected to the bottom surface of the object to be detected through the augmented reality module to obtain the coordinate of the mapping point; and S33, drawing a model of the object to be measured according to the mapping points, the image acquisition points and the coordinates of the fixed points, and calculating the volume of the object to be measured.
Preferably, when the object to be measured is a cube, the step S3 further includes the steps of:
s31a, calculating coordinates of image acquisition points through the augmented reality module; s32a, mapping a fixed point of an object to be detected to the bottom surface of the object to be detected through the augmented reality module to obtain a coordinate of a mapping point; s33a, selecting a pair of fixed points corresponding to each other up and down as an upper reference point and a lower reference point; calculating an included angle of a connecting line of the image acquisition point, a mapping point corresponding to the upper reference point and the lower reference point according to the coordinates of the image acquisition point and the coordinates of the lower reference point, and calculating the length of the connecting line of the upper reference point and the lower reference point according to the included angle; calculating the coordinates of the upper reference point and two fixed points adjacent to the upper reference point according to the length of the connecting line of the upper reference point and the lower reference point, and drawing a model of the object to be measured; and calculating the volume of the object to be measured according to a cubic volume formula.
Preferably, in step S2, if the image of the object fails to match the image recognition library, the image of the object is acquired again.
Preferably, after the step S1 of acquiring the image of the object, the method further includes a step of determining whether a volume is being calculated, and if so, the step of matching the image of the object with the image recognition library is skipped.
In another aspect of the present invention, there is provided an apparatus for measuring a volume, comprising: the image acquisition module is used for acquiring an image of the object to be detected through the augmented reality module; the image recognition library is used for matching the image of the object to be detected and giving fixed point coordinates of the object to be detected, and the fixed points can form a space graph; and the volume calculation module is used for calculating the volume of the object to be measured according to the fixed point coordinates.
The invention has the beneficial effects that: according to the method and the device for measuring the volume, the image of the object to be measured is obtained through the augmented reality module, the fixed-point coordinates are obtained through image recognition, and therefore the volume of the object to be measured is calculated.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments that can be derived by one of ordinary skill in the art from the embodiments given herein are intended to be within the scope of the present invention.
Fig. 1 is a flow chart of a method of measuring a volume according to the present invention.
The method for measuring the volume comprises the following steps:
s1, acquiring an image of an object to be detected through an augmented reality module;
s2, matching the image of the object to be detected with an image recognition library to obtain fixed point coordinates, wherein the fixed points can form a space graph;
and S3, calculating the volume of the object to be measured according to the fixed point coordinates.
In the present invention, the spatial pattern is a three-dimensional pattern occupying a certain space and having a volume. Many stereoscopic figures (such as curved surfaces) without volume also do not belong to spatial figures. The fixed point is a point with obvious characteristics on the image of the object to be measured, and the shape of the object to be measured can be approximately reduced according to the fixed point, so that the error is not too large when the volume of the object to be measured is calculated according to the fixed point, taking a cube as an example, 6 vertexes of the cube are usually selected as the fixed point, and the shape of the cube can be perfectly reduced according to the 6 vertexes.
Augmented Reality (AR) is a technology for calculating the position and angle of a camera image in real time and adding corresponding images, videos and 3D models, and aims to cover a virtual world on a screen in the real world and perform interaction. Virtual worlds can be associated with real worlds by augmented reality techniques.
As an example of the augmented reality module, for example, a software platform ARCore for building an augmented reality application program, which is introduced by google, can implement the following functions: 1) motion tracking: using the sensors and camera of the cell phone, the ARCore can accurately sense the position and pose of the cell phone and change the position and pose of the displayed virtual object 2) environmental understanding: a sensing plane, such as your desk, the ground, that accurately reproduces this plane in virtual space; 3) light source perception: the environment light sensor of the mobile phone is used for sensing the environment illumination condition, and correspondingly adjusting the brightness, the shadow and the material of the virtual object, so that the virtual object looks more integrated into the environment.
In the method for measuring the volume, the augmented reality module is used for shooting the object to be measured, and the augmented reality module can establish the correlation between the shot picture and the reality, so that the volume can be calculated possibly by utilizing the correlation established by the augmented reality module. When the ARCore is used as an augmented reality module, the motion tracking and environment understanding functions of the ARCore are mainly used.
Wherein, the step S1 further includes the following steps:
s11, generating a marking plane through an augmented reality module;
s12, placing the object to be detected on a marking plane, and shooting to obtain an image of the object to be detected.
Specifically, when the ARCore is used as an augmented reality module, a shooting device (such as a camera) is started, the ARCore focuses on a horizontal plane and translates, in the process, the ARCore captures feature points of a picture, the displacement of the shooting device is calculated by combining the displacement of the feature points on the picture with data captured by an IMU, three-dimensional coordinates are established, the feature points have own three-dimensional coordinates, and the feature points at the same height can be judged to be in a plane, so that a marking plane is found and generated on a screen. If the acquisition takes place without the generation of a marking plane, the arcre has not yet established the correct three-dimensional coordinates, so that a wrong correlation is established (for example, a certain level height of 1 meter, the arcre calculates 1.2 meters), which has an effect on the subsequent volume calculation step, resulting in an increased measurement error. In addition, since in actual operation, it is possible to identify a plurality of planes, such as the ground, the desktop, etc., it is necessary to determine which plane is used as the marking plane by means of focusing, and then establish a corresponding association by means of a translation step, and the ARCore generates the marking plane on the screen as the plane on which the object to be measured is placed. And after the marking plane is obtained, the object to be measured is placed on the marking plane to be shot to obtain an image of the object to be measured.
After the image of the object to be measured is acquired through the augmented reality module, the image of the object to be measured is transmitted into the image recognition library in a byte or Bitmap mode for matching so as to acquire fixed point coordinates, and the fixed point needs to be capable of forming a space graph instead of a plane graph. The space pattern formed by the fixed points is the space pattern of the object to be measured, and the volume of the object to be measured can be calculated according to the coordinates of the fixed points. By matching the image recognition library, fixed point coordinates can be directly obtained according to the image of the object to be detected, manual fixed point searching is avoided, operation can be reduced, and time is saved.
Wherein, the image recognition library is obtained by training a classifier by collecting the space pattern. By collecting relevant images, geometry, training is performed according to a general open source algorithm. Specifically, for example, training is performed by a convolutional neural network.
After the vertex coordinates are acquired, the step of calculating the volume of the object to be measured is performed, and specifically, the step S3 further includes the following steps:
s31, calculating coordinates of image acquisition points through an augmented reality module;
s32, mapping the fixed point of the object to be detected to the bottom surface of the object to be detected through the augmented reality module to obtain the coordinate of the mapping point;
and S33, drawing a model of the object to be measured according to the coordinates of the mapping points, the image acquisition points and the fixed points, and calculating the volume of the object to be measured.
The coordinates of the mapping points and the coordinates of the image acquisition points (i.e., the positions of the cameras when the images of the object are acquired) are acquired by, for example, a laser positioning method of the physical tracking of the ARCore. By performing appropriate calculation according to the coordinates, a model of the object to be measured (i.e., a spatial pattern of the object to be measured) can be drawn, and thus, the volume of the object to be measured can be calculated.
Fig. 2 is a diagram for explaining a calculation process when the object to be measured is a cube.
In particular, for example, in the logistics industry, packages are typically cubes, i.e., fixed-point composition space graphics are typically cubes. Therefore, when the image recognition library is constructed for the logistics industry, the collected space graphics can be graphics of various cubes so as to reduce workload when the image recognition library is constructed, and the algorithm of volume calculation can also be set for the cubes so as to achieve the purpose of simplifying the algorithm (otherwise, the specific algorithms of the space graphics of different shapes are different, and multiple algorithms need to be designed), after the images of the object to be detected are successfully matched, the image recognition library can give two-dimensional coordinate values of 6 ordered fixed points (such as A, B, C, D, E and F in the graph 2) of the object to be detected in a character string mode, and coordinates of every two fixed points have corresponding relations, such as A-D, B-E and C-F. Meanwhile, the six fixed points can form a space pattern. According to the two-dimensional coordinate values, a plurality of three-dimensional coordinate values required for drawing the model of the object to be measured can be calculated, and then the model of the object to be measured is drawn.
Subsequently, the flow proceeds to step S3, at which time step S3 further includes the steps of:
s31a, calculating coordinates of image acquisition points through an augmented reality module;
s32a, mapping a fixed point of an object to be detected to the bottom surface of the object to be detected through an augmented reality module to obtain a coordinate of a mapping point;
s33a, selecting a pair of fixed points corresponding to each other up and down as an upper reference point and a lower reference point, calculating an included angle of a connecting line of the image acquisition point, the mapping point corresponding to the upper reference point and the lower reference point according to the coordinates of the image acquisition point and the coordinates of the lower reference point, and calculating the length of the connecting line of the upper reference point and the lower reference point according to the included angle; calculating coordinates of an upper reference point and two fixed points adjacent to the upper reference point according to the length of a connecting line of the upper reference point and the lower reference point, and drawing a model of the object to be measured; and calculating the volume of the object to be measured according to a cubic volume formula.
Specifically, the coordinates of the acquisition point P (i.e., the position of the camera or the imaging device when the subject image is captured) can be calculated by the function provided by the ARCore. Two-dimensional coordinates of 6 fixed points can be mapped to the bottom surface of an object to be measured consisting of three points ABC through a physical tracking laser positioning mode provided by the ARCore, as shown in FIG. 2, D1, E1 and F1 are mappings on an ABC plane where a DEF three point is located. Selecting E as an upper reference point and B as a lower reference point, wherein E1 is a point E which is mapped on a plane ABC in a P point laser mode, so three points PEE1 are all on a straight line, and BE is the height (namely the length of a connecting line between the upper reference point and the lower reference point) which is required by us because the six points form a cube, so that in a plane EBE1, a cosine value of [ EE1B ] can BE obtained through the relationship between a vector and a trigonometric function:
cosangle EE1B is vector BE1 vector PE1/| vector BE1| vector PE1 |.
The corresponding [ EE1B can BE obtained according to the cosine value, and the length of BE is obtained by the following formula:
BE1*tan∠EE1B。
after the height values are obtained, the space coordinates corresponding to D, E and F can be obtained, and therefore the whole cube model is drawn. And finally, calculating the whole cube volume through a cube volume operation formula, wherein the formula is as follows:
v=|AB|*|BC|*|BE|。
finally, the calculated cube volume may be displayed on a screen.
FIG. 3 is a flow chart of one embodiment of a method of measuring volume according to the present invention.
In step S2, the matching of the image of the object with the image recognition library may fail, and in this case, the image of the object needs to be acquired again.
In addition, a certain time is often needed for calculating the volume of the object to be measured, so that after the image of the object to be measured is obtained in step S1, the method further includes a step of judging whether the volume is being calculated, if so, the step of matching the image of the object to be measured with the image recognition library is skipped, and the case of matching failure is directly handled as a case of matching failure, that is, only after one round of operation is finished, the calculation of the next volume of the object to be measured can be executed, so that the volume calculated and displayed last time is guaranteed to be the volume of the object to be measured which is shot last time, and the calculated volume and the object to be measured are prevented from corresponding to an error. The calculated volume value may be visually displayed on the photographing page together with the spatial region of the object (i.e., the outline of the object).
When the object to be measured is a cube, the overall process is as shown in fig. 3, firstly, a marker plane is generated by an augmented reality module (ARCore), and then, a camera is used for taking frames (shooting) to obtain an image of the object to be measured. And then, matching the image of the object to be measured with the image recognition library, and if the matching is unsuccessful and the vertex of the object to be measured (namely the fixed point coordinate of the object to be measured) is not recognized, performing frame taking (shooting) again. After the frame is taken (shot) through the camera, whether the volume is calculated or not needs to be judged, and if the volume is calculated, the image of the object to be detected is not matched with the image recognition library. If the matching is successful, the vertex of the object to be detected is identified, the bottom surface (such as the plane ABC in the figure 2) of the object to be detected is drawn through the 3 fixed points of the object to be detected, the vertex of the top surface (such as the point E in the figure 2) is searched to determine the height of the object to be detected (namely, the length of the connecting line of the upper reference point and the lower reference point is calculated), accordingly, a cube model of the object to be detected is constructed, the volume of the cube is calculated, and the whole process is completed.
FIG. 4 is a functional block diagram of one embodiment of a volumetric apparatus of the present invention.
The invention relates to a device for measuring volume, comprising:
the image acquisition module is used for acquiring an image of the object to be detected through the augmented reality module;
the image recognition library is used for matching images of the object to be detected and giving fixed point coordinates of the object to be detected, and the fixed points can form a space graph;
and the volume calculation module is used for calculating the volume of the object to be measured according to the fixed point coordinates.
When the device for measuring the volume is used, the image of the object to be measured under the augmented reality environment only needs to be acquired through the image acquisition module, then the image is automatically matched by the image recognition library to obtain fixed-point coordinates, and then the volume of the object to be measured is automatically calculated by the volume calculation module. The step of acquiring the image of the object to be measured, the step of matching the image recognition library, and the step of calculating the volume of the object to be measured are described in detail in the above description of the method, and are not repeated herein.
The device for measuring volume according to the present invention may be disposed on a mobile terminal, such as a mobile phone or a bargun, for example. Preferably, the image acquisition module is arranged on the mobile terminal, the image recognition library and the volume calculation module are arranged on the server so as to reduce the burden of the terminal and facilitate the update of the image recognition library, and the mobile terminal and the server are respectively provided with a signal transceiver for the communication between the mobile terminal and the server. The image acquisition module is, for example, a camera, a video camera, or other equipment capable of acquiring images, acquires an image of the object to be measured in the augmented reality environment through the image acquisition module, sends the image to the server to be matched with the image recognition library, performs volume calculation on the server, and finally sends the volume calculation result to the mobile terminal, and the mobile terminal displays the volume value on a display device (screen) of the mobile terminal.
The invention also relates to a device for measuring a volume, comprising: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores a program executable by the at least one processor, the program being executable by the at least one processor to enable the at least one processor to perform the method of measuring volume as described above or a preferred embodiment thereof. Furthermore, the apparatus for measuring a volume may further comprise a camera, e.g. a video camera, a camera, etc., for acquiring an image of the object to be measured. In addition, the apparatus for measuring volume may further include a display device, such as a display, a touch screen, etc., for displaying the information such as the image of the object, the calculated volume, etc.
The invention also relates to a computer-readable medium, in which a computer-executable program for performing the above-described method of measuring a volume or a preferred embodiment thereof is stored. The computer readable medium may be a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like. The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.