CN110824722B - Structured light projection module assembly device and projection module assembly and detection method - Google Patents
Structured light projection module assembly device and projection module assembly and detection method Download PDFInfo
- Publication number
- CN110824722B CN110824722B CN201810890954.2A CN201810890954A CN110824722B CN 110824722 B CN110824722 B CN 110824722B CN 201810890954 A CN201810890954 A CN 201810890954A CN 110824722 B CN110824722 B CN 110824722B
- Authority
- CN
- China
- Prior art keywords
- projection module
- center
- light source
- degree
- assembly
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/62—Optical apparatus specially adapted for adjusting optical elements during the assembly of optical systems
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B7/00—Mountings, adjusting means, or light-tight connections, for optical elements
- G02B7/003—Alignment of optical elements
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Optics & Photonics (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
The invention provides a structured light projection module assembly device, which comprises: a camera having a lens center; a target having a target center; a first multi-degree-of-freedom platform; and a second multiple degree of freedom platform; the camera is arranged at the top of the structured light projection module assembly device, the target is arranged below the camera, and a connecting line between the center of the lens of the camera and the center of the target passes through the center of the target and is perpendicular to the target; the first multi-degree-of-freedom platform is suitable for fixing the projection module light source assembly, and the second multi-degree-of-freedom platform is suitable for fixing the lens assembly; the first multi-degree-of-freedom platform can adjust the translation and the inclination of the projection module light source assembly, and the second multi-degree-of-freedom platform can adjust the translation and the inclination of the lens assembly. The invention also provides a corresponding method for assembling and detecting the structured light projection module. The invention can rapidly and effectively assemble and detect the structured light projection module.
Description
Technical Field
The invention relates to the technical field of optics, in particular to a structured light projection module assembling device and a projection module assembling and detecting method.
Background
The 3D vision technology obtains the depth information of the object, can be applied to face recognition and the like, and is applied to mobile terminals such as mobile phones and the like. The structured light depth scheme is a widely applied scheme at present, and mainly comprises an infrared projector, an infrared receiver, an RGB visible light image sensor and a processing chip.
The infrared projector is used for projecting infrared light of a specific pattern, and mainly comprises a projection module light source assembly (VCSEL) and a lens assembly, wherein the lens assembly specifically comprises a collimating element and an optical diffraction element. When the infrared projector works, the laser projector emits near infrared light, the near infrared light is finished by the collimating element to form uniform and parallel light beams, and the uniform and parallel light beams are modulated and copied by the DOE to form a specific optical pattern and are projected in a projection field. The relative offset and the inclination of each part mounting position can influence the projected pattern, therefore, in the assembling process of each part of the infrared projector, the infrared light center projected by the projection module light source assembly is required to be ensured to be perpendicular to the collimating element and the optical diffraction element so as to ensure that the projected specific pattern is clear and can be received and acquired by the infrared receiver.
Since the assembly accuracy of the structured light projection module is extremely high, it is necessary to use a precise apparatus and an efficient method for assembly and inspection.
Disclosure of Invention
The present invention aims to provide a solution that overcomes at least one of the drawbacks of the prior art.
According to an aspect of the present invention, there is provided a structured light projection module assembly apparatus, comprising:
a camera having a lens center;
a target having a target center;
a first multi-degree-of-freedom platform; and
a second multiple degree of freedom platform;
the camera is mounted at the top of the structured light projection module assembly device, the target is mounted below the camera, and a connecting line between the center of the lens of the camera and the center of the target passes through the center of the target and is perpendicular to the target; the first multi-degree-of-freedom platform is suitable for fixing the projection module light source assembly, and the second multi-degree-of-freedom platform is suitable for fixing the lens assembly; the first multi-degree-of-freedom platform can adjust the translation and the inclination of the projection module light source assembly, and the second multi-degree-of-freedom platform can adjust the translation and the inclination of the lens assembly.
The multi-degree-of-freedom platform comprises a plurality of degrees of freedom including front and back translation, left and right translation, front and back inclination and left and right inclination, wherein the multi-degree-of-freedom platform comprises the first multi-degree-of-freedom platform or the second multi-degree-of-freedom platform.
The multi-degree-of-freedom platform further comprises freedom degrees of up-down movement and rotation.
The first multi-degree-of-freedom platform is provided with a fixing clamp, and the projection module light source assembly is installed on the first multi-degree-of-freedom platform through the fixing clamp.
The second multi-degree-of-freedom platform is provided with a mechanical arm, and the lens assembly is arranged on the second multi-degree-of-freedom platform through the mechanical arm.
The multi-degree-of-freedom platform comprises a motion platform, an actuator and a base;
the motion table is disposed at the top, the actuator is disposed below the motion table, the base is disposed below the actuator, and the actuator can drive a load disposed on the motion table to perform a multi-degree-of-freedom motion and fix the motion table and the load on the motion table when reaching a target position.
According to another aspect of the present invention, there is provided a method for assembling a structured light projection module of the structured light projection module assembling apparatus, comprising:
1) fixing the projection module light source assembly on the first multi-freedom-degree platform on the projection module assembling device through the fixing clamp;
2) projecting patterns to the target plate through the projection module light source assembly, shooting the patterns projected on the target plate by the projection module light source assembly through the camera, and transmitting the images to the computer, wherein the computer calculates the translation amount and the inclination amount of the projection center of the projection module light source assembly relative to the center of the target plate by analyzing the images shot by the camera;
3) according to the translation amount and the inclination amount of the projection center of the projection module light source assembly relative to the center of the target calculated by the computer, the first multi-freedom-degree platform adjusts the projection module light source assembly positioned on the fixed clamp, so that the projection center of the projection module light source assembly and the center of the target are positioned on the same straight line, and the straight line is perpendicular to the target; and
4) and fixing the projection module light source assembly and the lens assembly so that the projection module light source assembly and the lens assembly are kept at the relative position determined by active calibration.
In the step 3), the translation amount and the inclination amount of the projection center of the projection module light source assembly relative to the reticle center are obtained by searching the relationship between the translation amount and the inclination amount of the pre-stored projection center of the standard projection module light source assembly relative to the reticle center.
The relationship between the translation amount and the inclination amount of the projection center of the pre-stored standard projection module light source assembly relative to the center of the target is obtained through the following steps:
10) fixing the standard projection module light source assembly to the first multi-degree-of-freedom platform on the projection module assembling device;
20) the standard projection module light source assembly is driven to move through the first multi-degree-of-freedom platform, the camera continuously shoots images projected on the target plate and transmits the images to the computer, and the computer establishes a corresponding relation between the translation amount of the standard projection module light source assembly and image change; and
30) the standard projection module light source assembly is driven to incline by the first multi-degree-of-freedom platform, the camera continuously shoots images projected on the target plate and transmits the images to the computer, and the computer establishes a corresponding relation between the inclination and the image change.
Wherein, between step 3) and step 4), further comprising:
31) the second multi-degree-of-freedom platform arranges the lens assembly above the projection module light source assembly fixed on the first multi-degree-of-freedom platform through the mechanical arm;
32) the projection module light source assembly projects patterns to the target plate through the lens assembly, the camera shoots the patterns projected on the target plate by the projection module light source assembly through the lens assembly and transmits images shot by the camera to the computer, and the computer calculates the translation amount and the inclination amount of the optical axis center of the lens assembly relative to the center of the target plate by analyzing the images; and
33) and according to the translation amount and the inclination amount of the optical axis center of the lens assembly relative to the target center calculated by the computer, the second multi-degree-of-freedom platform adjusts the lens assembly on the mechanical arm to enable the optical axis center of the lens assembly and the target center of the target to be in the same straight line, and the straight line is perpendicular to the target.
In the step 32), the translation amount and the inclination amount of the optical axis center of the lens assembly and the reticle center are obtained by searching the relationship between the translation amount and the inclination amount of the optical axis center of the standard lens assembly relative to the reticle center, which is prestored.
The relationship between the translation amount and the inclination amount of the optical axis center of the pre-stored standard lens assembly relative to the center of the target plate is obtained through the following steps:
100) fixing the standard lens assembly to the second multi-degree-of-freedom platform on the projection module assembly device;
200) the standard lens component is driven to move through the second multi-degree-of-freedom platform, the camera continuously shoots images projected on the target plate and transmits the images to the computer, and the computer establishes a corresponding relation between the translation amount of the standard lens component and the image change; and
300) the standard lens assembly is driven to incline by the second multi-degree-of-freedom platform, the camera continuously shoots images projected on the target plate and transmits the images to the computer, and the computer establishes a corresponding relation between the inclination amount and the image change.
In the step 4), the projection module light source assembly and the lens assembly are connected through a bonding or welding process.
According to another aspect of the present invention, there is provided a method for inspecting a structured light projection module using the apparatus for assembling a structured light projection module, comprising:
1000) fixing the structured light projection module to the first multi-degree-of-freedom platform;
2000) starting a light source component of the projection module, and shooting a structured light projection pattern received by the target by using the camera; and
3000) and judging whether the pattern projected by the structured light projection module meets the standard or not according to the quality of the structured light projection pattern shot by the camera.
Compared with the prior art, the invention has at least one of the following technical effects:
1. the structured light projection module assembling device can adjust the projection module light source assembly and the lens assembly respectively, and can solve the problem that the adjustment of the translation or the inclination of a component exceeds the adjustment range when only one component is adjusted.
2. The target of the structured light projection module assembling device is provided with a target center, the projection module light source assembly and the lens assembly are adjusted by the target center, a uniform reference point is provided, and the adjustment precision is high.
3. The first six-degree-of-freedom platform drives the standard projection module light source assembly to incline, the camera continuously shoots images projected on the target plate, and the corresponding relation between the inclination of the projection module light source assembly and the change of the images is established according to the shot images.
4. The second six-degree-of-freedom platform drives the standard lens component to incline, the camera continuously shoots images projected on the target plate, and a corresponding relation between the inclination amount of the lens component and the change of the images is established according to the shot images.
5. The first six-degree-of-freedom platform drives the standard projection module light source assembly to translate, the camera continuously shoots images projected on the target plate, and the corresponding relation between the translation amount of the projection module light source assembly and the image change is established according to the shot images.
6. The second six-degree-of-freedom platform drives the standard lens assembly to translate, the camera continuously shoots images projected on the target plate, and a corresponding relation between the translation amount of the lens assembly and image change is established according to the shot images.
7. And adjusting the relative positions of the projection module light source assembly and the lens assembly according to the corresponding relation between the image change and the inclination amount and the corresponding relation between the image change and the position translation, thereby realizing the quick assembly of the structured light projection module.
Drawings
Exemplary embodiments are illustrated in referenced figures of the drawings. The embodiments and figures disclosed herein are to be regarded as illustrative rather than restrictive.
FIG. 1 is a schematic view of an assembly apparatus for a structured light projection module according to one embodiment of the present invention;
FIG. 2 illustrates a schematic structural diagram of a six degree-of-freedom platform;
FIG. 3 is a schematic view of an assembly apparatus of a structured light projection module with a fixed projection module light source assembly according to one embodiment of the present invention;
FIG. 4 is a schematic view of the apparatus for assembling a structured light projection module with the light source module fixed thereon adjusted in translation and tilt according to one embodiment of the present invention;
FIG. 5 illustrates a schematic view of an assembly apparatus of a structured light projection module with a lens assembly 60 secured thereto, in accordance with one embodiment of the present invention;
FIG. 6 is a schematic diagram of an assembly apparatus for a structured light projection module with the lens assembly 60 mounted thereon and adjusted in translation and tilt according to one embodiment of the present invention;
FIG. 7 is a flow chart illustrating a method for assembling a structured light projection module, in accordance with one embodiment of the present invention.
Detailed Description
For a better understanding of the present application, various aspects of the present application will be described in more detail with reference to the accompanying drawings. It should be understood that the detailed description is merely illustrative of exemplary embodiments of the present application and does not limit the scope of the present application in any way. Like reference numerals refer to like elements throughout the specification. The expression "and/or" includes any and all combinations of one or more of the associated listed items.
It should be noted that the expressions first, second, etc. in this specification are used only to distinguish one feature from another feature, and do not indicate any limitation on the features. Thus, a first body discussed below may also be referred to as a second body without departing from the teachings of the present application.
In the drawings, the thickness, size, and shape of an object have been slightly exaggerated for convenience of explanation. The figures are purely diagrammatic and not drawn to scale.
It will be further understood that the terms "comprises," "comprising," "includes," "including," "has," "including," and/or "including," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. Moreover, when a statement such as "at least one of" appears after a list of listed features, the entirety of the listed features is modified rather than modifying individual elements in the list. Furthermore, when describing embodiments of the present application, the use of "may" mean "one or more embodiments of the present application. Also, the term "exemplary" is intended to refer to an example or illustration.
As used herein, the terms "substantially," "about," and the like are used as terms of table approximation and not as terms of table degree, and are intended to account for inherent deviations in measured or calculated values that will be recognized by those of ordinary skill in the art.
Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.
It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present application will be described in detail below with reference to the embodiments with reference to the attached drawings.
Fig. 1 shows a schematic view of an assembling apparatus of a structured light projection module according to an embodiment of the present invention. As shown in fig. 1, an assembling apparatus 100 of a structured light projection module includes a camera 10, a target 20, a first six-degree-of-freedom platform 30, a second six-degree-of-freedom platform 40, and a hollow rotary platform 70, wherein a fixing jig 90 is disposed on the first six-degree-of-freedom platform 30, and a robot arm 80 is disposed on the second six-degree-of-freedom platform 40. In fig. 1, the projection module light source assembly 50 to be assembled is fixed on the first six-degree-of-freedom platform 30 by a fixing jig 90, and the lens assembly 60 to be assembled is disposed at one side of the second six-degree-of-freedom platform 40 by a robot arm 80.
In fig. 1, the target 20 may be a tempered glass or a steel mesh, and a projection receiving curtain for receiving the structured light pattern projected by the projection module may be optionally disposed on the target 20. The camera 10 is connected to a computer and can capture the pattern of the structured light projected by the projection module light source assembly 50 onto the projection receiving curtain on the target 20, and can transmit the captured image to the computer.
Fig. 2 shows a schematic structural diagram of a six-degree-of-freedom platform, which includes a motion stage 301, actuators 302 and a base 303, as shown in fig. 2, wherein 6 actuators 302 can drive a load disposed on the motion stage 301 to perform a motion with 6 degrees of freedom, and fix the motion stage 301 and the load thereon when a target position is reached. Therefore, the first six-degree-of-freedom platform 30 can perform a 6-degree-of-freedom motion on the projection module light source assembly 50 fixed on the first six-degree-of-freedom platform 30 by the fixing clamp 90, so as to adjust the position of the projection module light source assembly 50 from 6 different directions. Likewise, the second six-dof platform 40 may perform a movement of 6 degrees of freedom with the lens assembly 60 disposed on one side of the second six-dof platform 40 (above the first six-dof platform 30) by the robot arm 80, thereby adjusting the position of the lens assembly 60 from 6 different directions. The 6 degrees of freedom of the first six-degree-of-freedom platform 30 or the second six-degree-of-freedom platform 40 are respectively front-back translation, left-right translation, up-down movement, front-back inclination, left-right inclination, and rotation.
Referring also to fig. 1, the camera 10 has a lens center 101 and the target 20 has a target center 201, wherein a line connecting the lens center 101 and the target center 201 passes through the target center 201 and is perpendicular to the target 20. The first six-dof platform 30 is located below the camera 10, the second six-dof platform 40 is located at one side of the first six-dof platform 30, and the second six-dof platform 40 arranges the lens assembly 60 above the projection module light source assembly 50 fixed on the first six-dof platform 30 through the mechanical arm 80.
In fig. 1, the projection module light source assembly 50 has a projection center 501, the computer calculates the translation and tilt of the projection center 501 of the projection module light source assembly 50 with respect to the reticle center 201, and the first six-degree-of-freedom stage 30 adjusts the projection module light source assembly 50 on the fixing clamp 90 so that the projection center 501 of the projection module light source assembly 50 and the reticle center 201 of the reticle 20 are in the same straight line, and the straight line is perpendicular to the reticle 20.
Also referring to fig. 1, the lens assembly 60 has an optical axis center 601, after the first six-degree-of-freedom platform 30 adjusts the projection module light source assembly 50 on the fixing fixture 90, the computer calculates the translation and tilt of the optical axis center 601 of the lens assembly 60 relative to the target center 201, and the second six-degree-of-freedom platform 40 adjusts the lens assembly 60 on the robot arm 80 such that the optical axis center 601 of the lens assembly 60 and the target center 201 of the target 20 are in the same straight line, and the straight line is perpendicular to the target 20.
The distance between the camera and the target is calculated, so that the camera can shoot all the structured light patterns projected onto the target by the projection module light source assembly 50, and the shot images are required to be clear and distinguishable.
The above-described structured light projection module assembly apparatus can adjust the projection module light source assembly 50 and the lens assembly 60, respectively, and can solve the problem that the adjustment of the translation or the inclination of the component exceeds the adjustment range when only one component is adjusted. In addition, the target of the structured light projection module assembling device has a target center, so that the projection module light source assembly 50 and the lens assembly 60 can be adjusted by the target center, and the structured light projection module assembling device has a uniform reference point and high adjustment precision.
FIG. 7 is a flow chart illustrating a method for assembling a structured light projection module, in accordance with one embodiment of the present invention.
Referring to fig. 7, the assembling method includes the following steps S10 to S80:
s10: the projection module assembly apparatus 100, the projection module light source assembly 50, and the lens assembly 60 are prepared.
S20: the projection module light source assembly 50 is fixed to the first six-degree-of-freedom stage 30 of the projection module assembly apparatus 100 by the fixing jig 90.
Fig. 3 is a schematic view of an assembly apparatus of a structured light projection module with a projection module light source assembly 50 fixed thereon according to an embodiment of the present invention. Referring to fig. 3, the projection module light source assembly 50 is fixed on the first six-dof stage 30 by a fixing jig 90. Because there are fixed errors and errors in the projection module light source assembly 50, an included angle exists between a projection center line passing through the projection center 501 of the projection module light source assembly 50 and a connection line between the camera lens center 101 and the target center 201, and the connection line between the projection center 501 and the target center 201 is not perpendicular to the target 20, so that the projection module light source assembly 50 needs to be adjusted in translation and inclination, so that the projection center 501 of the projection module light source assembly 50 and the target center 201 of the target 20 are in the same straight line, and the straight line is perpendicular to the target 20.
S30: the camera 10 captures the center of the image of the camera 10 projected by the projection module light source assembly 50, and calculates the translation and tilt of the projection center 501 of the projection module light source assembly 50 relative to the target center 201 by analyzing the image, by projecting the pattern onto the target 20 by the projection module light source assembly 50, and transmitting the image to the computer.
The standard projection module light source assembly is mounted on a first six-degree-of-freedom platform 30 on the projection module assembly device 100, then the standard projection module light source assembly is driven by the first six-degree-of-freedom platform 30 to move towards a specific direction through a specific translation amount, the camera 10 continuously shoots images projected on the target plate 20 and transmits the images to the computer, and the computer establishes a corresponding relation between the translation amount of the standard projection module light source assembly and image change; and driving the standard projection module light source assembly 50 to tilt by a specific tilting amount through the first six-degree-of-freedom platform 30, continuously shooting the image projected on the target 20 by the camera 10, and transmitting the image to the computer, wherein the computer establishes a corresponding relationship between the tilting amount and the image change. Through the above information collecting steps, the computer can calculate the translation and tilt of the projection center 501 of the projection module light source assembly 50 relative to the reticle center 201 according to the pattern of the projection module light source assembly 50 projected on the reticle 20 collected by the camera 10.
S40: according to the translation and tilt of the projection center 501 of the projection module light source assembly 50 relative to the reticle center 201 calculated by the computer, the first six-degree-of-freedom stage 30 adjusts the projection module light source assembly 50 located on the fixing fixture 90, so that the projection center 501 of the projection module light source assembly 50 and the reticle center 201 of the reticle 20 are located on the same straight line, and the straight line is perpendicular to the reticle 20.
Fig. 4 is a schematic view of the assembly apparatus of the structured light projection module with the projection module light source assembly 50 mounted thereon, after adjusting the translation and tilt according to one embodiment of the present invention. Referring to fig. 4, after the adjustment of the translation and tilt of the projection module light source assembly 50, the projection center 501 of the projection module light source assembly 50 and the reticle center 201 of the reticle 20 are located on the same straight line, and the straight line is perpendicular to the reticle 20.
S50: the second six-degree-of-freedom platform 40 arranges the lens assembly 60 above the projection module light source assembly 50 fixed on the first six-degree-of-freedom platform 30 by the robot arm 80.
Fig. 5 shows a schematic view of an assembly apparatus of a structured light projection module with a lens assembly 60 mounted thereon according to one embodiment of the present invention. Referring to fig. 5, the second six-dof platform 40 arranges the lens assembly 60 above the projection module light source assembly 50 fixed on the first six-dof platform 30 by the robot arm 80. Due to the installation error and the error of the lens assembly 60, an included angle exists between the optical axis center line passing through the optical axis center 601 of the lens assembly 60 and the connecting line between the lens center 101 and the target center 201, and the connecting line between the projection center 501 and the target center 201 is not perpendicular to the target 20, so that the lens assembly 60 needs to be adjusted in translation and inclination, so that the optical axis center 601 of the lens assembly 60 and the target center 201 of the target 20 are in the same straight line, and the straight line is perpendicular to the target 20.
S60: the projection module light source assembly 50 projects a pattern onto the target 20 through the lens assembly 60, the camera 10 captures the pattern projected on the target 20 by the projection module light source assembly 50 through the lens assembly 60 and transmits the image to the computer, the computer captures the center of the image captured by the camera 10, and the translation and tilt of the optical axis center 601 of the lens assembly 60 relative to the target center 201 are calculated by analyzing the image.
The standard lens assembly is mounted on a second six-degree-of-freedom platform 40 on the projection module assembly device 100 by mounting the standard projection module light source assembly on a first six-degree-of-freedom platform 30 of the projection module assembly device 100, wherein the projection center of the standard light source assembly and a target center 201 of a target 20 are in the same straight line, and the straight line is perpendicular to the target 20, then the standard lens assembly is driven to move by a specific offset through the second six-degree-of-freedom platform 40, the camera 10 continuously shoots images projected on the target 20 and transmits the images to a computer, and the computer establishes a corresponding relation between the offset and image change; and driving the standard lens assembly to tilt by a specific tilting amount through the second six-degree-of-freedom platform 40, continuously shooting the image projected on the target 20 by the camera 10, and transmitting the image to the computer, wherein the computer establishes a corresponding relationship between the tilting amount and the image change. Through the above information collecting steps, the computer can therefore calculate the amount of translation and tilt of the optical axis center 601 of the lens assembly 60 relative to the reticle center 201 according to the pattern collected by the machine 10 that the projection module light source assembly 50 projects on the reticle 20 through the lens assembly 60.
S70: according to the translation amount and the inclination amount of the optical axis center 601 of the lens assembly 60 relative to the target center 201 calculated by the computer, the second six-degree-of-freedom platform 40 adjusts the lens assembly 60 positioned on the mechanical arm 80 so that the optical axis center 601 of the lens assembly 60 and the target center 201 of the target 20 are in the same straight line, and the straight line is perpendicular to the target 20.
Fig. 6 is a schematic diagram of an assembly apparatus of a structured light projection module with the lens assembly 60 mounted thereon, with the amount of translation and the amount of tilt adjusted, according to an embodiment of the present invention. Referring to fig. 6, after the adjustment of the translation amount and the tilt amount of the lens assembly 60, the optical axis center 601 of the lens assembly 60 and the target center 201 of the target 20 are located on the same straight line, the straight line is perpendicular to the target 20, and the projection center 501 of the projection module light source assembly 50, the optical axis center 601 of the lens assembly 60 and the target center 201 are located on the same straight line, and the straight line is perpendicular to the target 20. Through the above adjustment, the light projected by the projection module light source assembly 50 can form a clear pattern on the receiving curtain of the target 20 after passing through the lens assembly 60.
S80: the projection module light source assembly 50 and the lens assembly 60 are fixed (e.g., adhesively fixed by glue) so that they remain in relative positions determined by the active alignment.
Through the process, the rapid adjustment of each part of the projection module can be realized, and the efficient and rapid assembly of the projection module is realized.
Further, in one embodiment, the projection module light source assembly and the lens assembly are coupled by an adhesive or welding process.
Further, in an embodiment, the method for detecting the structural light projection module by using the above-mentioned structural light projection module assembling device includes the following steps:
fixing a structured light projection module to be detected on the first multi-freedom-degree platform;
starting a light source component of the projection module, and shooting a structured light projection pattern received by the target by using the camera; and
and judging whether the pattern projected by the structured light projection module meets the standard or not according to the quality of the structured light projection pattern shot by the camera.
By the method, the structured light projection module can be rapidly detected.
The above description is only a preferred embodiment of the present application and is illustrative of the principles of the technology employed. It will be appreciated by a person skilled in the art that the scope of the invention as referred to in the present application is not limited to the embodiments with a specific combination of the above-mentioned features, but also covers other embodiments with any combination of the above-mentioned features or their equivalents without departing from the inventive concept. For example, the above features may be replaced with (but not limited to) features having similar functions disclosed in the present application.
Claims (13)
1. A structured light projection module assembly device, comprising:
a camera having a lens center;
a target having a target center;
a first multi-degree-of-freedom platform; and
a second multiple degree of freedom platform;
the camera is mounted at the top of the structured light projection module assembly device, the target is mounted below the camera, and a connecting line between the center of the lens of the camera and the center of the target passes through the center of the target and is perpendicular to the target; the first multi-degree-of-freedom platform is suitable for fixing the projection module light source assembly, and the second multi-degree-of-freedom platform is suitable for fixing the lens assembly; the first multi-degree-of-freedom platform can adjust the translation and the inclination of the projection module light source assembly, and the second multi-degree-of-freedom platform can adjust the translation and the inclination of the lens assembly.
2. The assembly device of claim 1, wherein the multiple degree of freedom platform comprises a plurality of degrees of freedom including a fore-aft translation, a left-right translation, a fore-aft tilt, and a left-right tilt, wherein the multiple degree of freedom platform comprises the first multiple degree of freedom platform or the second multiple degree of freedom platform.
3. The assembly device of claim 2, wherein the multiple degree of freedom platform further comprises degrees of freedom to move up and down and rotate.
4. The assembly device of claim 1, wherein the first multiple degree of freedom platform has a fixture clamp by which the projection module light source assembly is mounted on the first multiple degree of freedom platform.
5. The assembly device of claim 1, wherein the second multiple degree of freedom platform has a robotic arm by which the lens assembly is disposed on the second multiple degree of freedom platform.
6. The assembly device of claim 1, wherein the multiple degree of freedom platform comprises a motion stage, an actuator, and a base;
the motion table is disposed at the top, the actuator is disposed below the motion table, the base is disposed below the actuator, and the actuator can drive a load disposed on the motion table to perform a multi-degree-of-freedom motion and fix the motion table and the load on the motion table when reaching a target position.
7. A method of assembling a structured light projection module using the apparatus of any of claims 1 to 6, comprising:
1) fixing the projection module light source assembly on the first multi-freedom-degree platform on the projection module assembling device through the fixing clamp;
2) projecting patterns to the target plate through the projection module light source assembly, shooting the patterns projected on the target plate by the projection module light source assembly through the camera, and transmitting the images to the computer, wherein the computer calculates the translation amount and the inclination amount of the projection center of the projection module light source assembly relative to the center of the target plate by analyzing the images shot by the camera;
3) according to the translation amount and the inclination amount of the projection center of the projection module light source assembly relative to the center of the target calculated by the computer, the first multi-freedom-degree platform adjusts the projection module light source assembly positioned on the fixed clamp, so that the projection center of the projection module light source assembly and the center of the target are positioned on the same straight line, and the straight line is perpendicular to the target;
4) the second multi-degree-of-freedom platform arranges the lens assembly above the projection module light source assembly fixed on the first multi-degree-of-freedom platform through a mechanical arm;
5) the projection module light source assembly projects patterns to the target plate through the lens assembly, the camera shoots the patterns projected on the target plate by the projection module light source assembly through the lens assembly and transmits images shot by the camera to the computer, and the computer calculates the translation amount and the inclination amount of the optical axis center of the lens assembly relative to the center of the target plate by analyzing the images;
6) according to the translation amount and the inclination amount of the optical axis center of the lens assembly relative to the target plate center calculated by the computer, the second multi-degree-of-freedom platform adjusts the lens assembly on the mechanical arm to enable the optical axis center of the lens assembly and the target plate center of the target plate to be in the same straight line, and the straight line is perpendicular to the target plate; and
7) and fixing the projection module light source assembly and the lens assembly so that the projection module light source assembly and the lens assembly are kept at the relative position determined by active calibration.
8. The assembling method according to claim 7, wherein in the step 3), the translation amount and the tilt amount of the projection center of the projection module light source assembly relative to the target center are obtained by searching a pre-stored relationship between the translation amount and the tilt amount of the projection center of the standard projection module light source assembly relative to the target center.
9. The assembly method according to claim 8, wherein the relationship between the translation and tilt of the projection center of the pre-stored standard projection module light source assembly relative to the reticle center is obtained by:
10) fixing the standard projection module light source assembly to the first multi-degree-of-freedom platform on the projection module assembling device;
20) the standard projection module light source assembly is driven to move through the first multi-degree-of-freedom platform, the camera continuously shoots images projected on the target plate and transmits the images to the computer, and the computer establishes a corresponding relation between the translation amount of the standard projection module light source assembly and image change; and
30) the standard projection module light source assembly is driven to incline by the first multi-degree-of-freedom platform, the camera continuously shoots images projected on the target plate and transmits the images to the computer, and the computer establishes a corresponding relation between the inclination and the image change.
10. The assembling method according to claim 7, wherein in the step 5), the translation amount and the tilt amount of the optical axis center of the lens assembly and the reticle center are obtained by searching a pre-stored relationship between the translation amount and the tilt amount of the optical axis center of the standard lens assembly relative to the reticle center.
11. The assembly method of claim 10, wherein the relationship between the translation and tilt of the pre-stored optical axis center of the standard lens assembly relative to the reticle center is derived by:
100) fixing the standard lens assembly to the second multi-degree-of-freedom platform on the projection module assembly device;
200) the standard lens component is driven to move through the second multi-degree-of-freedom platform, the camera continuously shoots images projected on the target plate and transmits the images to the computer, and the computer establishes a corresponding relation between the translation amount of the standard lens component and the image change; and
300) the standard lens assembly is driven to incline by the second multi-degree-of-freedom platform, the camera continuously shoots images projected on the target plate and transmits the images to the computer, and the computer establishes a corresponding relation between the inclination amount and the image change.
12. The assembly method according to claim 7, wherein in the step 7), the projection module light source assembly and the lens assembly are connected by an adhesive or welding process.
13. A method of inspecting a structured light projection module using the structured light projection module assembly apparatus of any of claims 1 to 6, comprising:
1000) fixing the structured light projection module to the first multi-degree-of-freedom platform;
2000) starting a light source component of the projection module, and shooting a structured light projection pattern received by the target by using the camera; and
3000) and judging whether the pattern projected by the structured light projection module meets the standard or not according to the quality of the structured light projection pattern shot by the camera.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201810890954.2A CN110824722B (en) | 2018-08-07 | 2018-08-07 | Structured light projection module assembly device and projection module assembly and detection method |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201810890954.2A CN110824722B (en) | 2018-08-07 | 2018-08-07 | Structured light projection module assembly device and projection module assembly and detection method |
Publications (2)
Publication Number | Publication Date |
---|---|
CN110824722A CN110824722A (en) | 2020-02-21 |
CN110824722B true CN110824722B (en) | 2021-10-15 |
Family
ID=69534049
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201810890954.2A Active CN110824722B (en) | 2018-08-07 | 2018-08-07 | Structured light projection module assembly device and projection module assembly and detection method |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN110824722B (en) |
Families Citing this family (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
TWI768923B (en) * | 2021-05-24 | 2022-06-21 | 致伸科技股份有限公司 | Alignment method of camera lens and light source |
CN116893513B (en) * | 2023-09-08 | 2023-12-12 | 深圳市安思疆科技有限公司 | Structured light generator and pattern coding projector |
Citations (31)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN1247324A (en) * | 1998-09-08 | 2000-03-15 | 明碁电脑股份有限公司 | Optical lens group detection method |
CN2570732Y (en) * | 2002-10-11 | 2003-09-03 | 徐春云 | Precision measuring instrument |
CN2620297Y (en) * | 2003-01-23 | 2004-06-09 | 浙江大学 | Object surface regulator of optical projecting system based on focus out depth analysis |
FR2857462A3 (en) * | 2003-06-27 | 2005-01-14 | Nucam Corp | Lens adjustment in digital camera, involves horizontally moving translation carrier for eliminating horizontal offset and rotating lens group for eliminating vertical inclination based on position offset of lens and image sensor |
EP1526372A2 (en) * | 2004-11-02 | 2005-04-27 | Agilent Technologies, Inc. | Microfluidic system with adjustment for an optical detection |
CN1624515A (en) * | 2003-12-05 | 2005-06-08 | 鸿富锦精密工业(深圳)有限公司 | Digital camera lens mould set regulating method |
CN1687816A (en) * | 2005-06-17 | 2005-10-26 | 哈尔滨工业大学 | Space aligning method of ultra-precision rotary shaft and direct writing optical axis of laser direct writing apparatus |
CN1866078A (en) * | 2005-05-20 | 2006-11-22 | 奥林巴斯映像株式会社 | Digital camera adjusting method and adjusting device |
CN1888943A (en) * | 2006-04-14 | 2007-01-03 | 上海光谱仪器有限公司 | Micro-full analytical system chip height positioning method |
CN1311274C (en) * | 2001-02-27 | 2007-04-18 | 精工爱普生株式会社 | Device and method for positional adjustment of light mobulator |
KR20080019191A (en) * | 2006-08-24 | 2008-03-03 | 나노시스텍 게엠베하 | Apparatus and method for joining two or more elements |
JP2008051580A (en) * | 2006-08-23 | 2008-03-06 | Nikon Corp | Optical axis adjustment method for stereo camera |
CN101226344A (en) * | 2008-01-31 | 2008-07-23 | 上海微电子装备有限公司 | Apparatus and method for measuring optical system parameter |
CN101276056A (en) * | 2007-12-04 | 2008-10-01 | 中国科学院光电技术研究所 | High-precision splicing sub-mirror relative tilt error photoelectric detection system |
CN101285939A (en) * | 2007-04-09 | 2008-10-15 | 上海远超微纳技术有限公司 | Optical centering instrument possessing double focusing lens and centering method |
CN101387761A (en) * | 2008-10-08 | 2009-03-18 | 上海微电子装备有限公司 | Center aligning assembling and regulating apparatus and method for optical system |
CN201698075U (en) * | 2010-05-20 | 2011-01-05 | 郭其伟 | Optical device convenient for adjusting focal length |
CN102141376A (en) * | 2011-01-06 | 2011-08-03 | 大连理工大学 | Auxiliary reference-based machine vision detection system and method |
CN102538689A (en) * | 2011-12-29 | 2012-07-04 | 中国科学院上海光学精密机械研究所 | Centering and locating device of optical system and using method thereof |
CN202362572U (en) * | 2011-12-14 | 2012-08-01 | 吴江市博众精工科技有限公司 | Adjustment module |
CN202916501U (en) * | 2012-10-19 | 2013-05-01 | 厦门名晶光电科技有限公司 | Lens assembly tool |
CN103402116A (en) * | 2013-08-01 | 2013-11-20 | 北京小米科技有限责任公司 | Optical center calibration method and device |
CN105025290A (en) * | 2014-04-23 | 2015-11-04 | 宁波舜宇光电信息有限公司 | Method for automatically adjusting inclination between image photographing module group sensor and lens |
CN105137561A (en) * | 2015-09-28 | 2015-12-09 | 河南平原光电有限公司 | Adjustment mechanism based on flat glass correction optical axis system |
CN105405115A (en) * | 2014-08-19 | 2016-03-16 | 宁波舜宇光电信息有限公司 | Image module group alignment system and method thereof |
CN205484983U (en) * | 2015-12-31 | 2016-08-17 | 希比希光学(北京)有限公司 | Optical element's aligning device |
CN106338841A (en) * | 2016-10-20 | 2017-01-18 | 深圳市立品光电有限公司 | Method and apparatus for calibrating optical axis of optical lens of photographic module group |
CN106997103A (en) * | 2016-01-25 | 2017-08-01 | 深圳市睿晟自动化技术有限公司 | A kind of device and method of rapid alignment minisize optical lens optical axis |
CN206648614U (en) * | 2017-03-23 | 2017-11-17 | 西安昂科光电有限公司 | A kind of bifocal path technique focal length lens eccentricity measurement apparatus |
CN107505684A (en) * | 2017-08-25 | 2017-12-22 | 南京理工大学 | A kind of Method of Adjustment of microscope group |
CN108088394A (en) * | 2016-11-23 | 2018-05-29 | 中国科学院长春光学精密机械与物理研究所 | The eccentric measuring set and misalignment measurement method and optical centering instrument of a kind of aspherical mirror |
Family Cites Families (15)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20060232784A1 (en) * | 2005-04-19 | 2006-10-19 | Regis Grasser | Interferometers of high resolutions |
CN100547819C (en) * | 2006-08-15 | 2009-10-07 | 中国科学院物理研究所 | Epitaxial material and manufacturing method for GaN-based light-emitting diode chip with low polarization effect |
CN100549807C (en) * | 2006-08-28 | 2009-10-14 | 明基电通股份有限公司 | Shell structure and projector |
CA2690633C (en) * | 2007-06-15 | 2015-08-04 | Historx, Inc. | Method and system for standardizing microscope instruments |
JP2012018292A (en) * | 2010-07-08 | 2012-01-26 | Seiko Epson Corp | Manufacturing method of projection device, manufacturing apparatus of projection device, and projection device |
WO2013063762A1 (en) * | 2011-11-01 | 2013-05-10 | 捷光半导体照明科技(昆山)有限公司 | Lamp structure having moon phase change effect |
CN102628671A (en) * | 2012-03-29 | 2012-08-08 | 中国人民解放军第二炮兵工程学院 | Three-dimensional coordinate measuring method based on single-camera two-color linear structured light |
JP6082560B2 (en) * | 2012-10-09 | 2017-02-15 | 株式会社日立メディアエレクトロニクス | Light source device and projection display device using the same |
CN103226009B (en) * | 2013-03-19 | 2015-06-24 | 陕西科技大学 | Self-imaging structured light projection and phase shift device and method |
CN104155765B (en) * | 2013-05-15 | 2019-09-13 | 北京三星通信技术研究有限公司 | The method and apparatus of revision for 3-D image in spliced integration imaging display |
CN105093802A (en) * | 2014-05-06 | 2015-11-25 | 宁波舜宇光电信息有限公司 | Light-turning three-dimensional imaging apparatus and projection device and method thereof |
CN105572811B (en) * | 2014-10-31 | 2018-04-20 | 高准精密工业股份有限公司 | Surface fixing type laser module |
KR101865624B1 (en) * | 2016-06-10 | 2018-06-11 | 주식회사 토모큐브 | 3D Refractive Index Tomogram and Structured Illumination Microscopy System using Wavefront Shaper and Method thereof |
DE102016224979A1 (en) * | 2016-12-14 | 2018-06-14 | Robert Bosch Gmbh | Electrically controllable adjustment device |
CN106990660A (en) * | 2017-05-09 | 2017-07-28 | 深圳奥比中光科技有限公司 | Structured light projection module |
-
2018
- 2018-08-07 CN CN201810890954.2A patent/CN110824722B/en active Active
Patent Citations (31)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN1247324A (en) * | 1998-09-08 | 2000-03-15 | 明碁电脑股份有限公司 | Optical lens group detection method |
CN1311274C (en) * | 2001-02-27 | 2007-04-18 | 精工爱普生株式会社 | Device and method for positional adjustment of light mobulator |
CN2570732Y (en) * | 2002-10-11 | 2003-09-03 | 徐春云 | Precision measuring instrument |
CN2620297Y (en) * | 2003-01-23 | 2004-06-09 | 浙江大学 | Object surface regulator of optical projecting system based on focus out depth analysis |
FR2857462A3 (en) * | 2003-06-27 | 2005-01-14 | Nucam Corp | Lens adjustment in digital camera, involves horizontally moving translation carrier for eliminating horizontal offset and rotating lens group for eliminating vertical inclination based on position offset of lens and image sensor |
CN1624515A (en) * | 2003-12-05 | 2005-06-08 | 鸿富锦精密工业(深圳)有限公司 | Digital camera lens mould set regulating method |
EP1526372A2 (en) * | 2004-11-02 | 2005-04-27 | Agilent Technologies, Inc. | Microfluidic system with adjustment for an optical detection |
CN1866078A (en) * | 2005-05-20 | 2006-11-22 | 奥林巴斯映像株式会社 | Digital camera adjusting method and adjusting device |
CN1687816A (en) * | 2005-06-17 | 2005-10-26 | 哈尔滨工业大学 | Space aligning method of ultra-precision rotary shaft and direct writing optical axis of laser direct writing apparatus |
CN1888943A (en) * | 2006-04-14 | 2007-01-03 | 上海光谱仪器有限公司 | Micro-full analytical system chip height positioning method |
JP2008051580A (en) * | 2006-08-23 | 2008-03-06 | Nikon Corp | Optical axis adjustment method for stereo camera |
KR20080019191A (en) * | 2006-08-24 | 2008-03-03 | 나노시스텍 게엠베하 | Apparatus and method for joining two or more elements |
CN101285939A (en) * | 2007-04-09 | 2008-10-15 | 上海远超微纳技术有限公司 | Optical centering instrument possessing double focusing lens and centering method |
CN101276056A (en) * | 2007-12-04 | 2008-10-01 | 中国科学院光电技术研究所 | High-precision splicing sub-mirror relative tilt error photoelectric detection system |
CN101226344A (en) * | 2008-01-31 | 2008-07-23 | 上海微电子装备有限公司 | Apparatus and method for measuring optical system parameter |
CN101387761A (en) * | 2008-10-08 | 2009-03-18 | 上海微电子装备有限公司 | Center aligning assembling and regulating apparatus and method for optical system |
CN201698075U (en) * | 2010-05-20 | 2011-01-05 | 郭其伟 | Optical device convenient for adjusting focal length |
CN102141376A (en) * | 2011-01-06 | 2011-08-03 | 大连理工大学 | Auxiliary reference-based machine vision detection system and method |
CN202362572U (en) * | 2011-12-14 | 2012-08-01 | 吴江市博众精工科技有限公司 | Adjustment module |
CN102538689A (en) * | 2011-12-29 | 2012-07-04 | 中国科学院上海光学精密机械研究所 | Centering and locating device of optical system and using method thereof |
CN202916501U (en) * | 2012-10-19 | 2013-05-01 | 厦门名晶光电科技有限公司 | Lens assembly tool |
CN103402116A (en) * | 2013-08-01 | 2013-11-20 | 北京小米科技有限责任公司 | Optical center calibration method and device |
CN105025290A (en) * | 2014-04-23 | 2015-11-04 | 宁波舜宇光电信息有限公司 | Method for automatically adjusting inclination between image photographing module group sensor and lens |
CN105405115A (en) * | 2014-08-19 | 2016-03-16 | 宁波舜宇光电信息有限公司 | Image module group alignment system and method thereof |
CN105137561A (en) * | 2015-09-28 | 2015-12-09 | 河南平原光电有限公司 | Adjustment mechanism based on flat glass correction optical axis system |
CN205484983U (en) * | 2015-12-31 | 2016-08-17 | 希比希光学(北京)有限公司 | Optical element's aligning device |
CN106997103A (en) * | 2016-01-25 | 2017-08-01 | 深圳市睿晟自动化技术有限公司 | A kind of device and method of rapid alignment minisize optical lens optical axis |
CN106338841A (en) * | 2016-10-20 | 2017-01-18 | 深圳市立品光电有限公司 | Method and apparatus for calibrating optical axis of optical lens of photographic module group |
CN108088394A (en) * | 2016-11-23 | 2018-05-29 | 中国科学院长春光学精密机械与物理研究所 | The eccentric measuring set and misalignment measurement method and optical centering instrument of a kind of aspherical mirror |
CN206648614U (en) * | 2017-03-23 | 2017-11-17 | 西安昂科光电有限公司 | A kind of bifocal path technique focal length lens eccentricity measurement apparatus |
CN107505684A (en) * | 2017-08-25 | 2017-12-22 | 南京理工大学 | A kind of Method of Adjustment of microscope group |
Non-Patent Citations (3)
Title |
---|
一种四维光学仪器像面的装调方法;张继超等;《长春理工大学学报》;20161231;第29卷(第4期);第30-32页 * |
光学镜头轴线精确定心校准技术;管伟等;《应用光学》;20180331;第39卷(第2期);第252-256页 * |
子孔径测量系统标定实验及小波边缘提取算法研究;张适琪;《中国优秀硕士学位论文全文数据库 信息科技辑》;20160831;第I138-1275页 * |
Also Published As
Publication number | Publication date |
---|---|
CN110824722A (en) | 2020-02-21 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10578724B2 (en) | LIDAR optics alignment systems and methods | |
CN113074669B (en) | Laser projector with flash alignment | |
CN107407727A (en) | Method and system for optical alignment of light detection and ranging | |
KR20130085434A (en) | Robotic surveying instrument and method for the automated autocollimation of a telescope of a surveying instrument comprising an autocollimation target | |
JP5951793B2 (en) | Image sensor position detector | |
CN107664833B (en) | Machine vision system for aligning substrate and aligning device | |
JP2011133509A (en) | Method for assembling camera module | |
CN110736424B (en) | Calibration method of structured light projection module group equipment and projection module group measurement method | |
CN110824722B (en) | Structured light projection module assembly device and projection module assembly and detection method | |
JP6733895B1 (en) | Camera module manufacturing apparatus and camera module manufacturing method | |
JP7071207B2 (en) | Position detectors, position detection methods, manufacturing systems, manufacturing methods, control programs, and recording media | |
JP5957621B2 (en) | Imaging module manufacturing method and imaging module manufacturing apparatus | |
JP5747396B1 (en) | Optical axis adjustment device | |
JP5775533B2 (en) | System and method for picking up and placing a chip die | |
KR101549139B1 (en) | Method and apparatus for assembling camera module | |
JPWO2013021496A1 (en) | Adjustment device and optical component mounting device | |
US12174445B2 (en) | Camera module manufacturing device | |
JP2011151551A (en) | Method of manufacturing camera module and device | |
KR101958962B1 (en) | Lens element transfer mechanism, controller, optical axis adjustment device, and equipment and method for manufacturing optical module | |
CN113630522B (en) | Camera adjusting method and electronic equipment | |
JPH0545117A (en) | Optical 3D position measurement method | |
JP2014089257A (en) | Lens tilt detection device, lens tilt detection method, and camera module assembly method using lens tilt detection device | |
JP4046627B2 (en) | Optical module assembly method and assembly apparatus | |
CN222049533U (en) | Optical testing equipment | |
US20230400297A1 (en) | Passive alignment of lens module relative to an image sensor for manufacturing a camera module |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |