CN111091611A - An augmented reality system and method for workshop digital twin - Google Patents
An augmented reality system and method for workshop digital twin Download PDFInfo
- Publication number
- CN111091611A CN111091611A CN201911352218.2A CN201911352218A CN111091611A CN 111091611 A CN111091611 A CN 111091611A CN 201911352218 A CN201911352218 A CN 201911352218A CN 111091611 A CN111091611 A CN 111091611A
- Authority
- CN
- China
- Prior art keywords
- workshop
- equipment
- information
- current frame
- frame image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000003190 augmentative effect Effects 0.000 title claims abstract description 32
- 238000000034 method Methods 0.000 title claims abstract description 28
- 238000002372 labelling Methods 0.000 claims abstract description 65
- 238000003384 imaging method Methods 0.000 claims description 63
- 230000036544 posture Effects 0.000 claims description 36
- 239000003086 colorant Substances 0.000 claims description 16
- 238000006243 chemical reaction Methods 0.000 claims description 14
- 238000001514 detection method Methods 0.000 claims description 12
- 238000013507 mapping Methods 0.000 claims description 9
- 239000003795 chemical substances by application Substances 0.000 claims description 6
- 230000015572 biosynthetic process Effects 0.000 description 19
- 238000003786 synthesis reaction Methods 0.000 description 19
- 238000004891 communication Methods 0.000 description 2
- 238000004088 simulation Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000003754 machining Methods 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 229940098006 multigen Drugs 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 230000002194 synthesizing effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T15/00—3D [Three Dimensional] image rendering
- G06T15/04—Texture mapping
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/006—Mixed reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/20—Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Graphics (AREA)
- Computer Hardware Design (AREA)
- General Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Multimedia (AREA)
- Architecture (AREA)
- Processing Or Creating Images (AREA)
Abstract
本发明涉及一种面向车间数字孪生的增强现实系统及方法，包括物理车间、车间数字孪生系统，摄像头组、图像采集模块、车间三维模型标注模块、设备识别模块、数据查询及AR注册模块和AR显示模块，通过摄像头组采集车间视频图像，通过设备识别确定视频图像中包含的设备，并利用增强现实技术将数字孪生信息动态叠加在视频图像上设备图像区域或附近，从而实现车间设备信息的AR显示。本发明不需要建立物理车间系统的复杂三维模型，使用视频图像代替车间三维模型显示，画面显示更流畅；能够设置各设备信息的是否显示、显示界面类型、界面参数等，显示方式更灵活，操作更简便；能够判断鼠标所指设备标识，并显示该设备的相关信息。
The invention relates to a workshop digital twin-oriented augmented reality system and method, including a physical workshop, a workshop digital twin system, a camera group, an image acquisition module, a workshop three-dimensional model labeling module, a device identification module, a data query and AR registration module, and an AR registration module. The display module collects video images of the workshop through the camera group, determines the devices contained in the video images through device identification, and uses augmented reality technology to dynamically superimpose the digital twin information on or near the device image area on the video image, so as to realize AR of workshop equipment information show. The present invention does not need to establish a complex three-dimensional model of the physical workshop system, and uses video images to replace the three-dimensional model display of the workshop, and the screen display is smoother; whether the information of each equipment is displayed, the display interface type, interface parameters, etc. can be set, the display mode is more flexible, and the operation is more flexible. It is more convenient; it can judge the identification of the device pointed by the mouse, and display the relevant information of the device.
Description
Technical Field
The invention relates to a workshop digital twin oriented augmented reality system and a method, and belongs to the field of intelligent manufacturing.
Background
The current intelligent factory and intelligent workshop digital twin models need to establish a complex three-dimensional model of a workshop, then digital twin information generated by a workshop digital twin system is displayed on the three-dimensional model corresponding to workshop equipment, the whole modeling process works greatly, and once a workshop structure changes, the three-dimensional model needs to be changed, so that the later maintenance workload is large, and the calculation resources occupied when the complex model is rendered are large.
Disclosure of Invention
In order to solve the technical problem, the invention provides a workshop digital twin-oriented augmented reality system, which is short in development period and flexible in display effect, and display equipment information is superimposed on a workshop video image without establishing a complex three-dimensional model of a physical workshop system.
The technical scheme of the invention is as follows:
the augmented reality system for the workshop digital twin comprises a physical workshop and a workshop digital twin system, wherein the workshop digital twin system outputs digital twin information, and the augmented reality system further comprises the following modules: a camera group: the method comprises the steps of fixing in a physical workshop, and collecting a video image of the current state of the physical workshop; an image acquisition module: acquiring a workshop video image shot by a camera selected by a user; a workshop three-dimensional model labeling module: constructing a virtual three-dimensional model of a physical workshop, wherein the virtual three-dimensional model of equipment in the physical workshop is constructed by using proxy shape bodies, each proxy shape body represents the space shape and position of the equipment in the physical workshop, then labeling each proxy shape body, establishing a one-to-one correspondence relationship between labels and equipment identifiers, and generating a three-dimensional labeling model of the workshop; an equipment identification module: according to user setting, identifying equipment needing to display equipment information, specifically, acquiring a current frame image from the workshop video image, and identifying equipment identification corresponding to the equipment needing to be identified in the current frame image and imaging areas of the equipment on the current frame image according to the position corresponding relation between each proxy-shaped body in the three-dimensional labeling model and the equipment in the current frame image and the labels of the proxy-shaped bodies; the data query and AR registration module: inquiring a workshop digital twin system according to the equipment identification to obtain corresponding digital twin information, and meanwhile, determining an information display area of each equipment according to an imaging area of each equipment on a current frame image; the AR display module: and superposing the digital twin information of the equipment on the information display area of the equipment on the current frame image, thereby realizing AR display of the workshop equipment information.
Preferably, the camera group comprises a plurality of cameras installed in different areas, each camera comprises a lens, an image sensor, a cradle head and an image sensor posture detection module, the cradle head is used for controlling the orientation of the image sensor, and the image sensor posture detection module is used for detecting the direction of the image sensor; the image acquisition module is also used for acquiring the position of the image sensor of the currently selected camera and the posture of the image sensor; the equipment identification module acquires the position and the posture of the image sensor and sends the information to the workshop three-dimensional model marking module; the workshop three-dimensional model labeling module sets the position and the posture of a currently selected virtual image sensor of the virtual camera according to the conversion relation between the physical workshop coordinate system and the workshop three-dimensional labeling model coordinate system, so that a virtual imaging model in the workshop three-dimensional labeling model is consistent with an imaging model of the physical workshop, and a virtual synthetic image of the workshop three-dimensional labeling model is synthesized according to the camera imaging model; the identification module determines the corresponding relation of the position of the proxy shape body in the virtual composite image and the position of the equipment in the current frame image according to the consistency of the imaging model, the imaging position and the imaging posture of the virtual composite image and the current frame image, reads the mark of the equipment which needs to be identified in the virtual composite image according to the equipment which needs to be identified and is set by a user, and determines the equipment identification and the imaging area of the equipment in the current frame image according to the mark.
Preferably, the conversion relationship between the physical workshop coordinate system and the workshop three-dimensional labeling model coordinate system is an equivalent relationship, specifically: the workshop three-dimensional model labeling module further executes: unifying the coordinate system of the three-dimensional workshop marking model and the coordinate system of the physical workshop to ensure that the equipment coordinate of the physical workshop is consistent with the coordinate of the corresponding proxy shape body in the three-dimensional workshop marking model and the coordinate of each camera in the physical workshop is consistent with the coordinate of the corresponding virtual camera in the three-dimensional workshop marking model.
Preferably, the label is a color label, the proxy shapes corresponding to different devices render different colors, and a one-to-one mapping relationship between the colors and the device identifiers is established; and reading the color value of the pixel according to the corresponding relation between the proxy-shaped body in the virtual synthesized image and the position of the equipment in the current frame image, determining the equipment identification of the equipment corresponding to the color value in the current frame image, and determining the imaging area of the equipment in the current frame image according to the area range of the color value pixel.
Preferably, the system further comprises a workshop information display setting module and an AR information display interface library, wherein the AR display interface library defines various types of display interfaces, and the workshop information display setting module is used for setting the display interface type of each equipment parameter and/or digital twin information and the information or parameters displayed by the interface; after the data query and AR registration module acquires digital twin information of the equipment and determines an information display area of the equipment, the data query and AR registration module also sends a display interface type, the information display area and information or parameters to be displayed to the AR display module according to the setting of the equipment in the workshop information display setting module; the AR display module acquires a display interface of a corresponding type from the AR information display interface library, superposes the display interface on the information display area of the equipment in the current frame image, and displays the information or parameters of the equipment through the display interface.
Preferably, the device that needs to display device information is identified according to user settings, specifically: setting to display all equipment information, and identifying all equipment in the current frame image; or, the object recognition module reads the position of the mouse on the current frame image, determines the position of the mouse in the three-dimensional labeling model according to the conversion relation between the physical workshop coordinate system and the workshop three-dimensional labeling model coordinate system, thereby determining the label corresponding to the proxy shape body at the position, and recognizes the device identification corresponding to the device pointed by the mouse in the current frame image and the imaging area of the device on the current frame image according to the label.
The invention also provides a workshop digital twin oriented augmented reality method.
A workshop digital twin oriented augmented reality method comprises the following steps: step 1, constructing a virtual three-dimensional model of a physical workshop, wherein the virtual three-dimensional model of equipment in the physical workshop is constructed by using proxy shape bodies, each proxy shape body represents the space shape and the position of the equipment in the physical workshop, then labeling each proxy shape body, establishing a one-to-one correspondence relationship between labels and equipment identifiers, and generating a three-dimensional labeling model of the workshop; step 2, fixing a plurality of cameras in the physical workshop, and acquiring a video image of the current state of the physical workshop through the cameras; step 3, acquiring a workshop video image shot by a camera selected by a user; and 4, identifying the equipment needing to display equipment information according to user setting: acquiring a current frame image from the workshop video image, and identifying equipment identifications corresponding to equipment to be identified in the current frame image and imaging areas of the equipment on the current frame image according to the position corresponding relation between each agent shape body in the three-dimensional labeling model and the equipment in the current frame image and the labels of the agent shape bodies; step 5, inquiring a workshop digital twin system according to the equipment identification to obtain corresponding digital twin information, and meanwhile, determining an information display area of the equipment according to an imaging area of the equipment on the current frame image; and 6, superposing the digital twin information of the equipment on the information display area of the equipment on the current frame image, thereby realizing AR display of the workshop equipment information.
Preferably, in step 2, the camera group includes a plurality of cameras installed in different areas, each camera includes a lens, an image sensor, a pan-tilt and an image sensor posture detection module, the pan-tilt is used to control the orientation of the image sensor, and the image sensor posture detection module is used to detect the direction of the image sensor; in the step 3, the position of the image sensor of the currently selected camera and the posture of the image sensor are also acquired; in the step 4, firstly, the position and the posture of the virtual image sensor of the virtual camera corresponding to the currently selected camera are set according to the position and the posture of the image sensor and the conversion relation between the physical workshop coordinate system and the workshop three-dimensional labeling model coordinate system, so that a virtual imaging model in the workshop three-dimensional labeling model is consistent with an imaging model of the physical workshop, and a virtual synthetic image of the workshop three-dimensional labeling model is synthesized according to the camera imaging model; then, according to the consistency of the imaging models, the imaging positions and the imaging postures of the virtual synthetic image and the current frame image, the corresponding relation of the positions of the proxy-shaped body in the virtual synthetic image and the equipment in the current frame image is determined, the mark of the equipment which needs to be identified in the virtual synthetic image is read according to the equipment which needs to be identified and is set by a user, and the equipment identification and the imaging area of the equipment in the current frame image are determined according to the mark.
Preferably, the label is a color label, the proxy shapes corresponding to different devices render different colors, and a one-to-one mapping relationship between the colors and the device identifiers is established; and reading the color value of the pixel according to the corresponding relation between the proxy-shaped body in the virtual synthesized image and the position of the equipment in the current frame image, determining the equipment identification of the equipment corresponding to the color value in the current frame image, and determining the imaging area of the equipment in the current frame image according to the area range of the color value pixel.
Preferably, the method further comprises a workshop information display setting module and an AR information display interface library, wherein the AR display interface library defines various types of display interfaces, and the workshop information display setting module is used for setting the display interface type of each equipment parameter and/or digital twin information and the information or parameters displayed by the interface; in the step 5, after the digital twin information of the equipment and the information display area of the equipment are obtained, the setting of the equipment in the workshop information display setting module is read, and the type of the display interface and the information or parameters to be displayed are obtained; in the step 6, a display interface of a corresponding type is acquired from the AR information display interface library according to the acquired display interface type, the display interface is superimposed on the information display area of the device in the current frame image, and the device information or parameters to be displayed are displayed through the display interface.
Preferably, the device that needs to display device information is identified according to user settings, specifically: setting to display all the device information, and in the step 4, identifying all the devices in the current frame image; or, setting to display the information of the device pointed by the mouse, in the step 4, reading the position of the mouse on the current frame image, determining the position of the mouse in the three-dimensional labeling model according to the conversion relation between the physical workshop coordinate system and the workshop three-dimensional labeling model coordinate system, thereby determining the label corresponding to the proxy shape body at the position, and identifying the device identifier corresponding to the device pointed by the mouse in the current frame image and the imaging area of the device on the current frame image according to the label.
The invention has the following beneficial effects:
1. the invention relates to a workshop digital twin-oriented augmented reality system and a workshop digital twin-oriented augmented reality method.
2. According to the augmented reality system and method for the digital twin workshop, a complex three-dimensional model of a physical workshop system does not need to be established, the video image is used for replacing the workshop three-dimensional model for displaying, and the picture display is smoother.
3. The augmented reality system and method for the workshop digital twin can set whether information of each device is displayed or not, display interface types, interface parameters and the like, and are more flexible in display mode and simpler and more convenient to operate.
4. The invention relates to a workshop digital twin oriented augmented reality system and a method, which can judge the device identification pointed by a mouse and display the related information of the device.
Drawings
FIG. 1 is a system block diagram of a workshop digital twin oriented augmented reality system of the present invention;
FIG. 2 is a flow chart of the augmented reality method of the present invention showing information of all devices;
fig. 3 is a flowchart of displaying device information of a device pointed by a mouse according to the augmented reality method of the present invention.
Detailed Description
The invention is described in detail below with reference to the figures and the specific embodiments.
Referring to fig. 1 and 2, an augmented reality system for a workshop digital twin includes a physical workshop, a workshop digital twin system, a camera group, an image acquisition module, a workshop three-dimensional model labeling module, an equipment identification module, a data query and AR registration module, and an AR display module. The workshop digital twin system outputs digital twin information, and the camera group generally consists of at least two cameras, is fixed in a physical workshop and collects video images of the current state of the physical workshop; the image acquisition module acquires a workshop video image shot by a camera selected by a user; the method comprises the following steps that a workshop three-dimensional model marking module builds a virtual three-dimensional model of a physical workshop through three-dimensional modeling software (such as a Multigen Creator), wherein a virtual three-dimensional model of equipment in the physical workshop is built by using proxy shape bodies (such as basic shapes of cuboids, spheres, ellipsoids and the like), each proxy shape body represents the space shape and position of the equipment in the physical workshop, then each proxy shape body is marked, the one-to-one correspondence relationship between marks and equipment marks is built, and a workshop three-dimensional marking model is generated; the equipment identification module identifies equipment needing to display equipment information according to user setting, specifically, obtains a current frame image from the workshop video image, and identifies equipment identification corresponding to the equipment needing to be identified in the current frame image and imaging areas of the equipment on the current frame image according to the position corresponding relation between each proxy-shaped body in the three-dimensional labeling model and the equipment in the current frame image and the labels of the proxy-shaped bodies; the data query and AR registration module: inquiring a workshop digital twin system according to the equipment identification to obtain corresponding digital twin information, and meanwhile, determining an information display area of each equipment according to an imaging area of each equipment on a current frame image; and the AR display module superposes the digital twin information of the equipment on the information display area of the equipment on the current frame image, so that AR display of the workshop equipment information is realized. Since the current frame image is extracted from the video image, the current frame image is dynamically transformed, and device identification, data query and AR display are a dynamic loop process.
The information display area may be within the imaging area or in the vicinity of the imaging area.
The workshop digital twin system comprises a sensor, edge computing equipment, a bus communication module, a digital twin model system and digital twin information. The sensor and the edge computing equipment are used for detecting state information of a workshop, such as machining information, machine task information, logistics information, robot body information and the like of a machine tool body, the state information is transmitted to the digital twin model system through the bus communication system, the workshop digital twin model system is a mapping of a physical workshop in a computer and comprises a simulation and prediction model of the physical workshop, and the workshop digital twin model system takes the workshop state information as input and generates digital twin information comprising the digital twin simulation prediction information and the state information.
The camera group comprises a plurality of cameras installed in different areas, each camera comprises a lens, an image sensor, a holder and an image sensor posture detection module, the holder is used for controlling the position of the image sensor, and the image sensor posture detection module is used for detecting the direction of the image sensor.
Now, taking the conversion relationship between the physical workshop coordinate system and the workshop three-dimensional labeling model coordinate system as an equivalent relationship, and taking the color label as an example, the equipment identification process is specifically explained.
The workshop three-dimensional model labeling module further executes the steps of rendering different colors for agent-shaped bodies corresponding to different devices, establishing a one-to-one mapping relation between the colors and device identifiers, unifying a workshop three-dimensional labeling model coordinate system and a physical workshop coordinate system, enabling device coordinates of a physical workshop to be consistent with corresponding proxy shape body coordinates in the workshop three-dimensional labeling model, enabling coordinates of each camera of the physical workshop to be consistent with corresponding virtual camera coordinates in the workshop three-dimensional labeling model, acquiring a workshop video image shot by a camera selected by a user, a position P (x, y, z) of an image sensor of the currently selected camera and a posture Q (α, theta) of the image sensor, acquiring the position P (x, y, z) and the posture Q (α, theta) of the image sensor by the device identification module, sending the information to the workshop three-dimensional model labeling module, reading a current frame image of the workshop video image by the device identification module, setting the position P (x, y) of the virtual image sensor of the virtual camera to be consistent with the position P (x, y, z) of the image sensor, setting the information to the workshop three-dimensional model labeling module, reading a current frame image synthesis device image synthesis model, and determining an imaging color value of the corresponding to the current frame of the workshop three-dimensional synthesis device synthesis model according to the corresponding virtual image synthesis device synthesis model, and the image synthesis device synthesis color synthesis model identification of the image synthesis device identification, and the imaging device synthesis model, and determining the imaging device synthesis area of the current frame of the workshop three-dimensional synthesis device synthesis model synthesis area, and determining the image synthesis area of the workshop three-dimensional synthesis device synthesis model synthesis area according to the corresponding.
In this embodiment, the augmented reality system for a workshop digital twin further comprises a workshop information display setting module and an AR information display interface library, wherein the AR display interface library defines various display interfaces, such as a dashboard display interface control, a nixie tube display interface control and a virtual oscilloscope display interface control; the workshop information display setting module is used for setting the display interface type of each equipment parameter and/or digital twin information and the information or parameters displayed by the interface; after the data query and AR registration module acquires digital twin information of the equipment and determines an information display area of the equipment, the data query and AR registration module also sends a display interface type, the information display area and information or parameters to be displayed to the AR display module according to the setting of the equipment in the workshop information display setting module; the AR display module acquires a display interface of a corresponding type from the AR information display interface library, superposes the display interface on the information display area of the equipment in the current frame image, and displays the information or parameters of the equipment through the display interface.
In this embodiment, the identifying the device that needs to display the device information according to the user setting includes that the user setting is to display all device information in the current frame image, then all devices in the current frame image are identified, and then the information of all identified devices is obtained and displayed by being superimposed on the current frame image; or, the method is configured to display device information of a device pointed by a mouse of a user, please refer to fig. 3, where the object identification module reads a position of the mouse on a current frame image when identifying the device, then determines a position of the mouse in the three-dimensional annotation model according to a conversion relationship between a physical workshop coordinate system and a workshop three-dimensional annotation model coordinate system, thereby determining an annotation corresponding to a proxy shape body of the position, identifies a device identifier corresponding to the device pointed by the mouse in the current frame image and an imaging area of the device on the current frame image according to the annotation, and then superimposes digital twin information of the device on an information display area of the device on the current frame image.
Example two
Referring to fig. 1 and 2, an augmented reality method facing a workshop digital twin includes the following steps: step 1, constructing a virtual three-dimensional model of a physical workshop, wherein the virtual three-dimensional model of equipment in the physical workshop is constructed by using proxy shape bodies, each proxy shape body represents the space shape and the position of the equipment in the physical workshop, then labeling each proxy shape body, establishing a one-to-one correspondence relationship between labels and equipment identifiers, and generating a three-dimensional labeling model of the workshop; step 2, fixing a plurality of cameras in the physical workshop, and acquiring a video image of the current state of the physical workshop through the cameras; step 3, acquiring a workshop video image shot by a camera selected by a user; and 4, identifying the equipment needing to display equipment information according to user setting: acquiring a current frame image from the workshop video image, and identifying equipment identifications corresponding to equipment to be identified in the current frame image and imaging areas of the equipment on the current frame image according to the position corresponding relation between each agent shape body in the three-dimensional labeling model and the equipment in the current frame image and the labels of the agent shape bodies; step 5, inquiring a workshop digital twin system according to the equipment identification to obtain corresponding digital twin information, and meanwhile, determining an information display area of the equipment according to an imaging area of the equipment on the current frame image; and 6, superposing the digital twin information of the equipment on the information display area of the equipment on the current frame image, thereby realizing AR display of the workshop equipment information.
In step 2, the camera group includes a plurality of cameras installed in different areas, each camera includes a lens, an image sensor, a cradle head and an image sensor posture detection module, the cradle head is used for controlling the orientation of the image sensor, and the image sensor posture detection module is used for detecting the direction of the image sensor. And in the step 3, the position of the image sensor of the currently selected camera and the posture of the image sensor are also acquired. In the step 4, firstly, the position and the posture of the virtual image sensor of the virtual camera corresponding to the currently selected camera are set according to the position and the posture of the image sensor and the conversion relation between the physical workshop coordinate system and the workshop three-dimensional labeling model coordinate system, so that a virtual imaging model in the workshop three-dimensional labeling model is consistent with an imaging model of the physical workshop, and a virtual synthetic image of the workshop three-dimensional labeling model is synthesized according to the camera imaging model; then, according to the consistency of the imaging models, the imaging positions and the imaging postures of the virtual synthetic image and the current frame image, the corresponding relation of the positions of the proxy-shaped body in the virtual synthetic image and the equipment in the current frame image is determined, the mark of the equipment which needs to be identified in the virtual synthetic image is read according to the equipment which needs to be identified and is set by a user, and the equipment identification and the imaging area of the equipment in the current frame image are determined according to the mark.
In this embodiment, the label is a color label, the proxy shapes corresponding to different devices render different colors, and a one-to-one mapping relationship between the colors and the device identifiers is established; and reading the color value of the pixel according to the corresponding relation between the proxy-shaped body in the virtual synthesized image and the position of the equipment in the current frame image, determining the equipment identification of the equipment corresponding to the color value in the current frame image, and determining the imaging area of the equipment in the current frame image according to the area range of the color value pixel. For example, the statistical pixel color value is the maximum coordinate value and the minimum coordinate value of the abscissa and the ordinate of all the pixels of the color of the device a, and a rectangular region can be defined, and the region can be estimated as the imaging region of the device a.
In this embodiment, a workshop information display setting module and an AR information display interface library are further employed, the AR display interface library defines multiple types of display interfaces, and the workshop information display setting module is configured to set a display interface type of each device parameter and/or digital twin information and information or parameters displayed by the interface; in the step 5, after the digital twin information of the equipment and the information display area of the equipment are obtained, the setting of the equipment in the workshop information display setting module is read, and the type of the display interface and the information or parameters to be displayed are obtained; in the step 6, a display interface of a corresponding type is acquired from the AR information display interface library according to the acquired display interface type, the display interface is superimposed on the information display area of the device in the current frame image, and the device information or parameters to be displayed are displayed through the display interface.
The device that needs to display device information is identified according to user settings, specifically: setting to display all the device information, and in the step 4, identifying all the devices in the current frame image;
or, setting to display the information of the device pointed by the mouse, in the step 4, reading the position of the mouse on the current frame image, determining the position of the mouse in the three-dimensional labeling model according to the conversion relation between the physical workshop coordinate system and the workshop three-dimensional labeling model coordinate system, thereby determining the label corresponding to the proxy shape body at the position, and identifying the device identifier corresponding to the device pointed by the mouse in the current frame image and the imaging area of the device on the current frame image according to the label.
Referring to fig. 3, a method for determining a currently selected device and outputting digital twin information of the device by moving a mouse is provided.
A preparation stage:
step 10, constructing a virtual three-dimensional model of a physical workshop, wherein the virtual three-dimensional model of equipment in the physical workshop is constructed by using proxy shape bodies, each proxy shape body represents the space shape and the position of the equipment in the physical workshop, then color labeling is carried out on each proxy shape body, the proxy shape bodies corresponding to different equipment render different colors, a one-to-one mapping relation between the colors and equipment identifications is established, and the workshop three-dimensional labeling model is generated; the virtual three-dimensional model of the workshop also comprises virtual cameras corresponding to the spatial positions of the cameras in the physical workshop; unifying the coordinate system of the workshop three-dimensional labeling model and the coordinate system of the physical workshop to ensure that the equipment coordinate of the physical workshop is consistent with the coordinate of the corresponding proxy shape body in the workshop three-dimensional labeling model and the coordinate of each camera of the physical workshop is consistent with the coordinate of the corresponding virtual camera in the workshop three-dimensional labeling model;
step 20, defining an AR information display interface library, which comprises an instrument panel display interface control, a nixie tube display interface control and a virtual oscilloscope display interface control;
step 30, defining a workshop information display setting module, setting the display interface type of each equipment parameter and/or digital twin information and the information or parameters displayed by the interface, for example, defining parameters of an interface control, such as a display range, a resolution and the like;
operating a cycle stage:
step 40, collecting a current frame image of a workshop video image shot by a camera selected by a user, and collecting the position P (x, y, z) and the attitude Q (α, theta) of an image sensor of the currently selected camera;
step 50, reading the position P (x, y, z) and the posture Q (α, theta) of the image sensor of the currently selected camera, setting the position of the virtual image sensor of the virtual camera to be P (x, y, z) and the posture to be Q (α, theta), enabling a virtual imaging model in a workshop three-dimensional labeling model to be consistent with an imaging model of a physical workshop, synthesizing a virtual synthetic image of the workshop three-dimensional labeling model according to the camera imaging model, wherein different colors of the virtual synthetic image correspond to different devices, reading the coordinate (m, n) of a mouse on a current frame image, reading the color value of a pixel with the pixel coordinate (m, n) from the virtual synthetic image, identifying a device identification corresponding to the pixel on the virtual synthetic image according to the one-to-one mapping relation between the colors and the physical devices, wherein the device identification is the device pointed by the mouse on the current frame image, and simultaneously counting the area of the pixel with the color value to obtain the imaging area of the device;
step 60, inquiring a workshop digital twin system according to the equipment identification to obtain corresponding digital twin information; reading the setting of the equipment in the workshop information display setting module, and acquiring the type of a display interface and information or parameters to be displayed;
step 70, acquiring a display interface of a corresponding type from an AR information display interface library according to the acquired display interface type, determining an information display area of the equipment according to an imaging area of the equipment, and superposing digital twin information of the equipment on the information display area of the equipment on a current frame image so as to realize AR display of workshop equipment information;
and 80, judging whether the program is finished or not, if not, returning to the step 40, and if so, finishing the exit program.
The above description is only an embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes performed by the present specification and drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (11)
1. The augmented reality system for the workshop digital twin comprises a physical workshop and a workshop digital twin system, wherein the workshop digital twin system outputs digital twin information, and is characterized by further comprising the following modules:
a camera group: the method comprises the steps of fixing in a physical workshop, and collecting a video image of the current state of the physical workshop;
an image acquisition module: acquiring a workshop video image shot by a camera selected by a user;
a workshop three-dimensional model labeling module: constructing a virtual three-dimensional model of a physical workshop, wherein the virtual three-dimensional model of equipment in the physical workshop is constructed by using proxy shape bodies, each proxy shape body represents the space shape and position of the equipment in the physical workshop, then labeling each proxy shape body, establishing a one-to-one correspondence relationship between labels and equipment identifiers, and generating a three-dimensional labeling model of the workshop;
an equipment identification module: according to user setting, identifying equipment needing to display equipment information, specifically, acquiring a current frame image from the workshop video image, and identifying equipment identification corresponding to the equipment needing to be identified in the current frame image and imaging areas of the equipment on the current frame image according to the position corresponding relation between each proxy-shaped body in the three-dimensional labeling model and the equipment in the current frame image and the labels of the proxy-shaped bodies;
the data query and AR registration module: inquiring a workshop digital twin system according to the equipment identification to obtain corresponding digital twin information, and meanwhile, determining an information display area of each equipment according to an imaging area of each equipment on a current frame image;
the AR display module: and superposing the digital twin information of the equipment on the information display area of the equipment on the current frame image, thereby realizing AR display of the workshop equipment information.
2. The workshop digital twin-oriented augmented reality system according to claim 1, wherein: the camera group comprises a plurality of cameras arranged in different areas, each camera comprises a lens, an image sensor, a holder and an image sensor posture detection module, the holder is used for controlling the direction of the image sensor, and the image sensor posture detection module is used for detecting the direction of the image sensor;
the image acquisition module is also used for acquiring the position of the image sensor of the currently selected camera and the posture of the image sensor;
the equipment identification module acquires the position and the posture of the image sensor and sends the information to the workshop three-dimensional model marking module;
the workshop three-dimensional model labeling module sets the position and the posture of a currently selected virtual image sensor of the virtual camera according to the conversion relation between the physical workshop coordinate system and the workshop three-dimensional labeling model coordinate system, so that a virtual imaging model in the workshop three-dimensional labeling model is consistent with an imaging model of the physical workshop, and a virtual synthetic image of the workshop three-dimensional labeling model is synthesized according to the camera imaging model;
the identification module determines the corresponding relation of the position of the proxy shape body in the virtual composite image and the position of the equipment in the current frame image according to the consistency of the imaging model, the imaging position and the imaging posture of the virtual composite image and the current frame image, reads the mark of the equipment which needs to be identified in the virtual composite image according to the equipment which needs to be identified and is set by a user, and determines the equipment identification and the imaging area of the equipment in the current frame image according to the mark.
3. The workshop digital twin-oriented augmented reality system according to claim 2, wherein: the conversion relation between the physical workshop coordinate system and the workshop three-dimensional labeling model coordinate system is an equivalent relation, and specifically: the workshop three-dimensional model labeling module further executes: unifying the coordinate system of the three-dimensional workshop marking model and the coordinate system of the physical workshop to ensure that the equipment coordinate of the physical workshop is consistent with the coordinate of the corresponding proxy shape body in the three-dimensional workshop marking model and the coordinate of each camera in the physical workshop is consistent with the coordinate of the corresponding virtual camera in the three-dimensional workshop marking model.
4. The shop digital twin oriented augmented reality system according to any one of claims 1 to 3, wherein: the labels are color labels, the proxy shapes corresponding to different devices render different colors, and a one-to-one mapping relation between the colors and the device identifiers is established; and reading the color value of the pixel according to the corresponding relation between the proxy-shaped body in the virtual synthesized image and the position of the equipment in the current frame image, determining the equipment identification of the equipment corresponding to the color value in the current frame image, and determining the imaging area of the equipment in the current frame image according to the area range of the color value pixel.
5. The workshop digital twin-oriented augmented reality system according to claim 1, wherein: the workshop information display setting module is used for setting the display interface type of each equipment parameter and/or digital twin information and the information or the parameters displayed by the interface; after the data query and AR registration module acquires digital twin information of the equipment and determines an information display area of the equipment, the data query and AR registration module also sends a display interface type, the information display area and information or parameters to be displayed to the AR display module according to the setting of the equipment in the workshop information display setting module; the AR display module acquires a display interface of a corresponding type from the AR information display interface library, superposes the display interface on the information display area of the equipment in the current frame image, and displays the information or parameters of the equipment through the display interface.
6. The workshop digital twin-oriented augmented reality system according to claim 1, wherein: the device that needs to display device information is identified according to user settings, specifically: setting to display all equipment information, and identifying all equipment in the current frame image;
or, the object recognition module reads the position of the mouse on the current frame image, determines the position of the mouse in the three-dimensional labeling model according to the conversion relation between the physical workshop coordinate system and the workshop three-dimensional labeling model coordinate system, thereby determining the label corresponding to the proxy shape body at the position, and recognizes the device identification corresponding to the device pointed by the mouse in the current frame image and the imaging area of the device on the current frame image according to the label.
7. A workshop digital twin oriented augmented reality method is characterized by comprising the following steps:
step 1, constructing a virtual three-dimensional model of a physical workshop, wherein the virtual three-dimensional model of equipment in the physical workshop is constructed by using proxy shape bodies, each proxy shape body represents the space shape and the position of the equipment in the physical workshop, then labeling each proxy shape body, establishing a one-to-one correspondence relationship between labels and equipment identifiers, and generating a three-dimensional labeling model of the workshop;
step 2, fixing a plurality of cameras in the physical workshop, and acquiring a video image of the current state of the physical workshop through the cameras;
step 3, acquiring a workshop video image shot by a camera selected by a user;
and 4, identifying the equipment needing to display equipment information according to user setting: acquiring a current frame image from the workshop video image, and identifying equipment identifications corresponding to equipment to be identified in the current frame image and imaging areas of the equipment on the current frame image according to the position corresponding relation between each agent shape body in the three-dimensional labeling model and the equipment in the current frame image and the labels of the agent shape bodies;
step 5, inquiring a workshop digital twin system according to the equipment identification to obtain corresponding digital twin information, and meanwhile, determining an information display area of the equipment according to an imaging area of the equipment on the current frame image;
and 6, superposing the digital twin information of the equipment on the information display area of the equipment on the current frame image, thereby realizing AR display of the workshop equipment information.
8. The workshop digital twin oriented augmented reality method of claim 7, wherein: in the step 2, the camera group comprises a plurality of cameras installed in different areas, each camera comprises a lens, an image sensor, a cradle head and an image sensor posture detection module, the cradle head is used for controlling the orientation of the image sensor, and the image sensor posture detection module is used for detecting the direction of the image sensor;
in the step 3, the position of the image sensor of the currently selected camera and the posture of the image sensor are also acquired;
in the step 4, firstly, the position and the posture of the virtual image sensor of the virtual camera corresponding to the currently selected camera are set according to the position and the posture of the image sensor and the conversion relation between the physical workshop coordinate system and the workshop three-dimensional labeling model coordinate system, so that a virtual imaging model in the workshop three-dimensional labeling model is consistent with an imaging model of the physical workshop, and a virtual synthetic image of the workshop three-dimensional labeling model is synthesized according to the camera imaging model; then, according to the consistency of the imaging models, the imaging positions and the imaging postures of the virtual synthetic image and the current frame image, the corresponding relation of the positions of the proxy-shaped body in the virtual synthetic image and the equipment in the current frame image is determined, the mark of the equipment which needs to be identified in the virtual synthetic image is read according to the equipment which needs to be identified and is set by a user, and the equipment identification and the imaging area of the equipment in the current frame image are determined according to the mark.
9. The workshop digital twin oriented augmented reality method of claim 7 or 8, wherein: the labels are color labels, the proxy shapes corresponding to different devices render different colors, and a one-to-one mapping relation between the colors and the device identifiers is established; and reading the color value of the pixel according to the corresponding relation between the proxy-shaped body in the virtual synthesized image and the position of the equipment in the current frame image, determining the equipment identification of the equipment corresponding to the color value in the current frame image, and determining the imaging area of the equipment in the current frame image according to the area range of the color value pixel.
10. The workshop digital twin oriented augmented reality method of claim 7, wherein: the workshop information display setting module is used for setting the display interface type of each equipment parameter and/or digital twin information and the information or the parameters displayed by the interface; in the step 5, after the digital twin information of the equipment and the information display area of the equipment are obtained, the setting of the equipment in the workshop information display setting module is read, and the type of the display interface and the information or parameters to be displayed are obtained; in the step 6, a display interface of a corresponding type is acquired from the AR information display interface library according to the acquired display interface type, the display interface is superimposed on the information display area of the device in the current frame image, and the device information or parameters to be displayed are displayed through the display interface.
11. The workshop digital twin oriented augmented reality method of claim 7, wherein: the device that needs to display device information is identified according to user settings, specifically: setting to display all the device information, and in the step 4, identifying all the devices in the current frame image; or, setting to display the information of the device pointed by the mouse, in the step 4, reading the position of the mouse on the current frame image, determining the position of the mouse in the three-dimensional labeling model according to the conversion relation between the physical workshop coordinate system and the workshop three-dimensional labeling model coordinate system, thereby determining the label corresponding to the proxy shape body at the position, and identifying the device identifier corresponding to the device pointed by the mouse in the current frame image and the imaging area of the device on the current frame image according to the label.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911352218.2A CN111091611B (en) | 2019-12-25 | 2019-12-25 | An augmented reality system and method for workshop digital twin |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911352218.2A CN111091611B (en) | 2019-12-25 | 2019-12-25 | An augmented reality system and method for workshop digital twin |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111091611A true CN111091611A (en) | 2020-05-01 |
| CN111091611B CN111091611B (en) | 2023-05-26 |
Family
ID=70397137
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911352218.2A Active CN111091611B (en) | 2019-12-25 | 2019-12-25 | An augmented reality system and method for workshop digital twin |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111091611B (en) |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111596604A (en) * | 2020-06-12 | 2020-08-28 | 中国科学院重庆绿色智能技术研究院 | An intelligent fault diagnosis and self-healing control system and method for engineering equipment based on digital twin |
| CN111627262A (en) * | 2020-06-12 | 2020-09-04 | 上海商汤智能科技有限公司 | Sand table display system, method, computer equipment and storage medium |
| CN111833426A (en) * | 2020-07-23 | 2020-10-27 | 四川长虹电器股份有限公司 | Three-dimensional visualization method based on digital twinning |
| CN111857520A (en) * | 2020-06-16 | 2020-10-30 | 广东希睿数字科技有限公司 | 3D visual interactive display method and system based on digital twins |
| CN111966068A (en) * | 2020-08-27 | 2020-11-20 | 上海电机系统节能工程技术研究中心有限公司 | Augmented reality monitoring method and device for motor production line, electronic equipment and storage medium |
| CN112150507A (en) * | 2020-09-29 | 2020-12-29 | 厦门汇利伟业科技有限公司 | Method and system for synchronously reproducing 3D model of object posture and displacement |
| WO2022040920A1 (en) * | 2020-08-25 | 2022-03-03 | 南京翱翔智能制造科技有限公司 | Digital-twin-based ar interactive system and method |
| CN114157826A (en) * | 2022-02-07 | 2022-03-08 | 西安塔力科技有限公司 | Cooperative operation method based on digital twin body |
| CN114299390A (en) * | 2021-12-27 | 2022-04-08 | 烟台杰瑞石油服务集团股份有限公司 | Method and device for determining maintenance component demonstration video and safety helmet |
| CN115077488A (en) * | 2022-05-26 | 2022-09-20 | 燕山大学 | Factory personnel real-time positioning monitoring system and method based on digital twin |
| CN116543134A (en) * | 2023-07-06 | 2023-08-04 | 金锐同创(北京)科技股份有限公司 | Method, device, computer equipment and medium for constructing digital twin model |
| CN117156108A (en) * | 2023-10-31 | 2023-12-01 | 中海物业管理有限公司 | Enhanced display system and method for machine room equipment monitoring picture |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160328883A1 (en) * | 2015-05-05 | 2016-11-10 | PTC, Inc. | Augmented reality system |
| CN109359507A (en) * | 2018-08-24 | 2019-02-19 | 南京理工大学 | A rapid construction method of workshop personnel digital twin model |
| CN109819233A (en) * | 2019-01-21 | 2019-05-28 | 哈工大机器人（合肥）国际创新研究院 | A kind of digital twinned system based on virtual image technology |
-
2019
- 2019-12-25 CN CN201911352218.2A patent/CN111091611B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160328883A1 (en) * | 2015-05-05 | 2016-11-10 | PTC, Inc. | Augmented reality system |
| CN109359507A (en) * | 2018-08-24 | 2019-02-19 | 南京理工大学 | A rapid construction method of workshop personnel digital twin model |
| CN109819233A (en) * | 2019-01-21 | 2019-05-28 | 哈工大机器人（合肥）国际创新研究院 | A kind of digital twinned system based on virtual image technology |
Non-Patent Citations (1)
| Title |
|---|
| 陶飞;程颖;程江峰;张萌;徐文君;戚庆林: "数字孪生车间信息物理融合理论与技术", 计算机集成制造系统 * |
Cited By (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111627262A (en) * | 2020-06-12 | 2020-09-04 | 上海商汤智能科技有限公司 | Sand table display system, method, computer equipment and storage medium |
| CN111596604A (en) * | 2020-06-12 | 2020-08-28 | 中国科学院重庆绿色智能技术研究院 | An intelligent fault diagnosis and self-healing control system and method for engineering equipment based on digital twin |
| CN111857520A (en) * | 2020-06-16 | 2020-10-30 | 广东希睿数字科技有限公司 | 3D visual interactive display method and system based on digital twins |
| CN111833426B (en) * | 2020-07-23 | 2022-09-02 | 四川长虹电器股份有限公司 | Three-dimensional visualization method based on digital twinning |
| CN111833426A (en) * | 2020-07-23 | 2020-10-27 | 四川长虹电器股份有限公司 | Three-dimensional visualization method based on digital twinning |
| WO2022040920A1 (en) * | 2020-08-25 | 2022-03-03 | 南京翱翔智能制造科技有限公司 | Digital-twin-based ar interactive system and method |
| CN111966068A (en) * | 2020-08-27 | 2020-11-20 | 上海电机系统节能工程技术研究中心有限公司 | Augmented reality monitoring method and device for motor production line, electronic equipment and storage medium |
| CN112150507A (en) * | 2020-09-29 | 2020-12-29 | 厦门汇利伟业科技有限公司 | Method and system for synchronously reproducing 3D model of object posture and displacement |
| CN112150507B (en) * | 2020-09-29 | 2024-02-02 | 厦门汇利伟业科技有限公司 | 3D model synchronous reproduction method and system for object posture and displacement |
| CN114299390A (en) * | 2021-12-27 | 2022-04-08 | 烟台杰瑞石油服务集团股份有限公司 | Method and device for determining maintenance component demonstration video and safety helmet |
| CN114157826B (en) * | 2022-02-07 | 2022-08-02 | 西安塔力科技有限公司 | Cooperative operation method based on digital twin body |
| CN114157826A (en) * | 2022-02-07 | 2022-03-08 | 西安塔力科技有限公司 | Cooperative operation method based on digital twin body |
| CN115077488A (en) * | 2022-05-26 | 2022-09-20 | 燕山大学 | Factory personnel real-time positioning monitoring system and method based on digital twin |
| CN116543134A (en) * | 2023-07-06 | 2023-08-04 | 金锐同创(北京)科技股份有限公司 | Method, device, computer equipment and medium for constructing digital twin model |
| CN116543134B (en) * | 2023-07-06 | 2023-09-15 | 金锐同创(北京)科技股份有限公司 | Method, device, computer equipment and medium for constructing digital twin model |
| CN117156108A (en) * | 2023-10-31 | 2023-12-01 | 中海物业管理有限公司 | Enhanced display system and method for machine room equipment monitoring picture |
| CN117156108B (en) * | 2023-10-31 | 2024-03-15 | 中海物业管理有限公司 | Enhanced display system and method for machine room equipment monitoring picture |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111091611B (en) | 2023-05-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111091611A (en) | An augmented reality system and method for workshop digital twin | |
| CN111062873B (en) | A Parallax Image Mosaic and Visualization Method Based on Multiple Pairs of Binocular Cameras | |
| EP3067861B1 (en) | Determination of a coordinate conversion parameter | |
| EP3690815B1 (en) | Method, medium and apparatus for automatically labeling target object within image | |
| JP6223122B2 (en) | Automatic reference frame calibration for augmented reality | |
| TWI419081B (en) | Method and system for providing augmented reality based on marker tracing, and computer program product thereof | |
| EP3678101A2 (en) | Ar-enabled labeling using aligned cad models | |
| US8751969B2 (en) | Information processor, processing method and program for displaying a virtual image | |
| US10699165B2 (en) | System and method using augmented reality for efficient collection of training data for machine learning | |
| JP4137078B2 (en) | Mixed reality information generating apparatus and method | |
| US7782320B2 (en) | Information processing method and information processing apparatus | |
| US20100134601A1 (en) | Method and device for determining the pose of video capture means in the digitization frame of reference of at least one three-dimensional virtual object modelling at least one real object | |
| CN112912936B (en) | Mixed reality system, program, mobile terminal device and method | |
| CN108430032B (en) | Method and equipment for realizing position sharing of VR/AR equipment | |
| JP5387668B2 (en) | Information processing terminal, information processing method, and program | |
| CN116433848B (en) | Screen model generation method, device, electronic equipment and storage medium | |
| Sun et al. | Towards industrial iot-ar systems using deep learning-based object pose estimation | |
| US20180224825A1 (en) | Image processing system, image processing device, method of reconfiguring circuit in fpga, and program for reconfiguring circuit in fpga | |
| CN116205980A (en) | Method and device for positioning and tracking virtual reality in mobile space | |
| CN209746614U (en) | Simulation interaction visualization system of virtual robot workstation | |
| CN106445208B (en) | A method of the mark figure based on serial ports follows display | |
| CN112150507B (en) | 3D model synchronous reproduction method and system for object posture and displacement | |
| EP3682196A1 (en) | Systems and methods for calibrating imaging and spatial orientation sensors | |
| CN111429549B (en) | Route image generation method, device and storage medium | |
| CN112995641A (en) | 3D module imaging device and method and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |