CN111833437A - A method of creating 3D documents - Google Patents

Abstract

The invention discloses a method for creating a three-dimensional document. The image parameters of at least one illustration of a practical item in the existing two-dimensional document are determined, and the 3D model of the practical item is displayed with the two-dimensional image having the determined image parameters. and, based on the comparison of the two-dimensional representation and the 3D model with the determined image parameters from the two-dimensional representation, additional information is obtained, which together with the 3D model forms a three-dimensional document. This process allows to analyze the diagrams in existing 2D documents in order to obtain additional information from them, which can then be superimposed on any view of the 3D model. A number of corresponding points are selected, and image parameters are varied until the two-dimensional representation and the 3D model are aligned, for which purpose suitable criteria for determining adequate alignment can be established.

Description

A method of creating 3D documents

technical field

The invention relates to the field of kitchenware, in particular to a method for creating a three-dimensional document.

Background technique

In most cases, documents such as manuals, operating manuals, assembly instructions, repair instructions, training documents, etc., correspond to practical items ranging from household appliances, toys, machines and machine components to highly complex technical equipment. Such documents typically contain various two-dimensional illustrations of the utility article based on which a user of the utility article will understand the functionality of the utility article or receive instructions to use the utility article. Illustrations can be simple views, isometrics, or even photographs of useful object views. Such illustrations in printed documents are necessarily two-dimensional representations of various views of the utility item. Therefore, when working with documents, users of utility articles have to transfer the 2D view to the actual 3D utility article, a very complex psychological challenge and, for many, no 3D translation capability at all.

This situation can be improved by using documentation in three-dimensional form, where utility items are represented in three dimensions, e.g. on a display unit, the view of utility items can be changed arbitrarily, and further improvements can be achieved using augmented reality, which is generally understood For a person's actual perception of reality, especially what is seen, heard or felt, in order to expand or supplement additional information. This additional information can likewise be conveyed to the person visually, acoustically or tactilely. For example, a utility item is recorded using a recording unit, such as a digital camera for a smartphone or tablet, a 3D scanner, etc., and the recorded view of the utility item is supplemented with additional information in real-time, either by superimposing it on the recorded and displayed image or by playback acoustic information. Furthermore, there are currently methods for providing documentation of utility items in three dimensions, also using augmented reality. However, creating 3D documentation of useful objects is a very complex and time-consuming task, especially when 3D animation is required, possibly also in real-time (eg in augmented reality). This requires experienced 3D designers and professional animation software products. Therefore, no three-dimensional or augmented reality documentation has been created so far.

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide a method for creating a three-dimensional document, so as to solve the problems raised in the background art. To achieve the above object, the present invention provides the following technical solutions;

A method of creating a three-dimensional document, comprising:

providing a two-dimensional document of the utility item, and obtaining image parameters of at least one two-dimensional illustration of the utility item through the two-dimensional document;

aligning the 3D model of the utility item with the image parameters of the 2D representation;

A three-dimensional documentation of the utility item is determined based on a comparison of the 3D model of the utility item with the image parameters of the two-dimensional representation.

Preferably, the method for aligning the 3D model of the utility item with the image parameters of the 2D illustration is as follows: selecting a plurality of corresponding points in the 2D illustration and the 3D model respectively, so that P1A corresponds to P1M , P2A corresponds to P2M, P3A corresponds to P3M, P4A corresponds to P4M, and the image parameters are changed until the 3D model is aligned with the 2D representation.

Preferably, the continuous components K1, K2 are determined in the two-dimensional diagram, and the eccentricity of the continuous components K1, K2 is determined, and the determined eccentricity is compared with a predetermined threshold, and the continuous component K1 is plotted in the longitudinal direction. , the two reference lines of K2, and, based on the mask of the utility item obtained from the 3D model, determine which endpoint of the two reference lines is located in the utility item.

Preferably, the search area SR for checking the annotation text using optical character recognition software is defined around the other end point of one of the two reference lines.

Preferably, the end points of the reference line located within the utility item are associated with a portion of the utility item.

Preferably, the two-dimensional illustration includes a photograph or a CAD drawing of the practical item, and the higher the resolution of the two-dimensional illustration, the better.

Preferably, the two-dimensional document includes one or more of an operation manual, assembly instructions, repair instructions or training documents.

Compared with the prior art, the beneficial effects of the present invention are: the present invention provides a simple method for providing a document of a utility item in a three-dimensional manner, which can create a three-dimensional document of a utility item in a fast, simple and usable manner, and It is less time-consuming and can be converted into 3D animation if necessary, without the need for experienced 3D designers and professional animation software products, so it is very suitable for home use.

Description of drawings

Fig. 1-Fig. 2 is the process of determining the image parameter of two-dimensional illustration in two-dimensional document provided by the present invention;

Figures 3-5 illustrate the process of determining annotated reference lines with illustrations in a two-dimensional document provided by the present invention.

Detailed ways

The technical aspects of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

Please refer to FIG. 1-FIG. 5, an embodiment provided by the present invention:

A method of creating a three-dimensional document, comprising:

S1: Provide a two-dimensional document of a practical item, and obtain image parameters of at least one two-dimensional illustration of the practical item through the two-dimensional document;

S2: Align the 3D model of the utility item with the image parameters of the two-dimensional illustration;

S3: Based on the comparison of the 3D model of the utility item with the image parameters of the two-dimensional illustration, a three-dimensional document of the utility item is determined.

Specifically, the two-dimensional documentation may exist in the form of existing printed documents with two-dimensional images, usually in the form of manuals, operation manuals, assembly instructions, repair instructions, training documents, and the like. However, in an embodiment of the present invention, it is also possible to first recreate the two-dimensional document used to create the three-dimensional document. For example, utility items can be photographed in various views and configurations, and photographs can be used as two-dimensional illustrations prior to using the method according to the invention, both methods being understood as two-dimensional documents within the meaning of the present invention. The present invention is based on creating a three-dimensional document of a utility item 1 in an automated manner from an existing two-dimensional document of the utility item 1 in printed form (or digitized). Then, three-dimensional documents can also be used, for example, for augmented reality applications, for web-based training, or for animated documents. For this purpose, at least one existing two-dimensional representation A of the utility item 1 in the two-dimensional document 2 is analyzed and information for the three-dimensional document is obtained therefrom. For this purpose, the 2D representation A is naturally present in the 2D document 2 in digital form, e.g. the 2D representation A in the 2D document 2 is scanned in two dimensions with sufficient resolution, or the 2D document 2 Does it already exist in digital format, it makes sense to choose a resolution that corresponds to the level of detail in the figure.

A prerequisite of the present invention is the existence of a digital 3D model M of the utility item 1 , which may be, for example, in the form of a 3D CAD drawing. Since two-dimensional documents 2 are typically created by manufacturers of utility objects 1, and development and design of utility objects 1 based on or using 3D drawings are now commonplace, such 3D CAD drawings will be available in most cases. The advantage of a 3DCAD drawing is that all parts are contained and identifiable. Alternatively, the utility item 1 can be 3D scanned. Likewise, individual parts of the utility item 1 can be individually scanned in three dimensions and then combined into a 3D model of the utility item 1 . 3D scanners and related software are available for this purpose, allowing such 3D scans to be performed. An example mentioned here is Kinect.RTM. Sensor from Microsoft.RTM. Combined with the open source software package KinFu.

In the first step of the method of the invention, specific image parameters for creating a two-dimensional representation A of a view of the utility item 1 in the printed document 2 must be determined, whether or not the two-dimensional representation A is a photograph or drawing. The two basic image parameters are the viewing position in space relative to the utility item 1, and the focal length at which the utility item 1 is viewed. Using the example of a photograph, it is clear that when the viewing position of the camera relative to the utility item 1 changes, or when the camera settings (especially the focal length) change, the image changes.

In one possible embodiment of the method, the user marks in image A and in image A a plurality, preferably four or more corresponding points P1A-P1M, P2A-P2M, P3A-P3M, P4A- P4M. 3D model M, as shown in FIG. 3 . Corresponding points P1A-P1M, P2A-P2M, P3A-P3M, P4A-P4M are the same points of utility item 1 in image A and 3D model M. Points P1A, P2A, P3A, P4A are represented in image A of utility item 1. This involves marking at specific points P1A, P2A, P3A, P4A of utility item 1 in image A, and equivalent points P1M, P2M, P3M, P4M in 3D model M, as shown in Figure 2, by points P3A, P3M The double arrows in between indicate. The 3D model M can be aligned with the 2D representation A by corresponding points P1A-P1M, P2A-P2M, P3A-P3M, P4A-P4M in such a way that the corresponding points P1A-P1M, P2A-P2M, P3A-P3M, P4A -P4M stacks when overlapping each other. Next, estimate the focal length as an image parameter if not known, which is often the case. By using the two-dimensional representation A, corresponding to the points P1A-P1M, P2A-P2M, P3A-P3M, P4A-P4M and the estimated focal length, the observation position BP has produced a two-dimensional representation A of the view of the utility item, which can be digitally Available in image processing, well-known algorithms, such as the known POSIT algorithm, determine the algorithm shown in FIG. 1 . By superimposing the views of the two-dimensional representation A and the 3D model M with the determined viewing position BP1 , eg by displaying them simultaneously on the screen, the results can be verified as shown in FIG. 4 . If the deviation is too large (left side of Figure 2), the focus can be changed and/or other or additional corresponding points can be selected. This can be repeated iteratively until a sufficiently accurate alignment of the two-dimensional representation A with the view of the 3D model M results in a determined viewing position BPn (right side of Figure 2). The viewing position BP can be easily determined by the user, whereby the user himself/herself decides when sufficient alignment has been achieved. Likewise, the viewing position BP can be determined using standard methods in digital image processing, such as methods for point recognition and methods for determining image alignment, or in an automatic manner.

After aligning the two-dimensional representation A in the printed document 2 and the 3D model M as described, the two-dimensional representation A can be analyzed with respect to representations of the type described above, the basic procedure being, based on the A comparison of the views of the 3D model with the determined image parameters, additional information is obtained from the two-dimensional representation, which can then be superimposed on the views of the two-dimensional image. The 3D model of the practical article, in the form of a view of the 3D model. For this purpose, the additional information is preferably associated with the various parts of the utility article 1 so that the additional information can always be displayed correctly in a particular view.

Based on the 3D model M in the view from the determined viewing position BP, a two-dimensional mask S (right side of Fig. 3) is created which contains all the pixels of the determined view of the 3D model M. Using the mask S, it is now possible to determine all the pixels in the digitized two-dimensional representation A that are outside the view of the utility item 1 (which are aligned with the view of the 3D model M). By using digital image processing methods, the continuous components in the digitized two-dimensional representation A are now searched. One possible algorithm for this purpose is the Maximum Stable Extremal Region (MSER) algorithm, which represents an efficient method of segmenting an image into continuous components. Therefore, each continuous component K determined in this way includes a digitized two-dimensional representation. A number of pixels in A (indicated by dots in Figure 4). Of all the determined continuous components K, those pixels of the reference line ₃₁ can be represented, for which purpose the eccentricity of the adjoining part K can be determined. Therefore, the eccentricity is determined as the ratio of the maximum longitudinal extension to the maximum transverse extension (perpendicular to the longitudinal extension) of the continuous component K. For reference lines 3 ₁ , 3 ₂ , it can be assumed that the eccentricity normalized to 1 must be approximately 1. Here an appropriate threshold can be defined for any continuous component K1.

Next, the endpoints E11, E12 of the reference lines 3 ₁ , 3 ₂ will be determined. For this purpose, any desired method can be used to draw the best possible straight line in the direction of longitudinal extension into the pixels adjoining the components K1, K2. For example, an ellipse having a major axis H in the longitudinal extension direction can be fitted around the pixels of the adjoining parts K1, K2, as shown in FIG. 2 . 4 is the component K1. Then, the vertices of the ellipse on the main axis H are interpreted as the endpoints E11, E12 of the sought reference line ₃₁ . Based on the mask S, it is now possible to easily determine which endpoints E11 , E12 are located in the inner area and in the area outside the utility item 1 .

For the end point E12 within the utility item 1, it is also possible to determine, based on the 3D model M, which part of the utility item 1 determines the reference line ₃₁ point. The association can also be stored in a dedicated part memory, which contains all individually identifiable parts of the utility item 1 . The end point E12 determined inside the utility item ₁ , as the anchor point AP1 of the reference line 31 (FIG. 5), the association for the specified part of the three-dimensional document of the utility item 1 is stored together in the parts memory. However, the anchor point AP1 can also be moved to the corresponding body center point of the associated part T, which may be advantageous for the subsequent three-dimensional representation of the utility item 1 .

The search area SR (now can build search text or numbers around the end point E11 located outside the utility item 1, for example by conventional optical character recognition (OCR) software. Alternatively, the annotation text can also be an annotation text determined in this way as well Stored in the reference line ₃₁ , eg again in the component memory.

If the view of the utility item 1 is now changed in the augmented reality application, eg by changing the camera position used to record the utility item 1 , the determined reference line 3 ₁ and the superimposition of the associated annotations can follow the text accordingly via the established anchor point AP1 . The anchor point AP1 remains on the identified part of the utility item 1 and the other endpoint E11 of the reference line 3 ₁ with the annotation text can be located in the augmented reality representation at any given location, preferably outside the utility item 1, As shown in Figure 1. Of course, the reference line 31 with the annotated text can be represented in any given three-dimensional view of the utility item ₁ .

Although the present invention has been described in detail with reference to the foregoing embodiments, for those skilled in the art, it is still possible to modify the technical solutions described in the foregoing embodiments, or to perform equivalent replacements for some of the technical features. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims (7)

1. a method for creating a three-dimensional document, characterized in that: comprising: providing a two-dimensional document of the utility item, and obtaining image parameters of at least one two-dimensional illustration of the utility item through the two-dimensional document; aligning the 3D model of the utility item with the image parameters of the 2D representation; A three-dimensional documentation of the utility item is determined based on a comparison of the 3D model of the utility item with the image parameters of the two-dimensional representation. 2. A method for creating a three-dimensional document according to claim 1, wherein the method for aligning the 3D model of the utility item with the image parameters of the two-dimensional illustration is: in the two-dimensional image Select multiple corresponding points in the 3D model, make P1A correspond to P1M, P2A correspond to P2M, P3A correspond to P3M, and P4A correspond to P4M, and change the image parameters until the 3D model is aligned with the 2D icon . 3. A method for creating a three-dimensional document according to claim 2, wherein the continuous components K1, K2 are determined in the two-dimensional representation, and the eccentricity of the continuous components K1, K2 is determined, and the The determined eccentricity is compared with a predetermined threshold, two reference lines are drawn longitudinally to the continuous components K1, K2, and, based on the mask of the utility item obtained from the 3D model, which of the two reference lines is determined Endpoints are in utility items. 4. A method of creating a three-dimensional document according to claim 3, wherein the search area SR for checking the annotation text using optical character recognition software is defined around the other end point of one of the two reference lines. 5. A method of creating a three-dimensional document according to claim 4, wherein an end point of the reference line within the utility item is associated with a portion of the utility item. 6 . The method for creating a three-dimensional document according to claim 5 , wherein the two-dimensional illustration includes a photo or a CAD drawing of the utility item, and the higher the resolution of the two-dimensional illustration, the better. 7 . . 7. The method for creating a three-dimensional document according to claim 6, wherein the two-dimensional document comprises one or more of an operation manual, an assembly instruction, a repair instruction or a training document.

Images