[go: up one dir, main page]

CN114758440A - Access control system based on mixed recognition of images and characters - Google Patents

Access control system based on mixed recognition of images and characters Download PDF

Info

Publication number
CN114758440A
CN114758440A CN202011588979.0A CN202011588979A CN114758440A CN 114758440 A CN114758440 A CN 114758440A CN 202011588979 A CN202011588979 A CN 202011588979A CN 114758440 A CN114758440 A CN 114758440A
Authority
CN
China
Prior art keywords
image information
mode
information
dimensional
static
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202011588979.0A
Other languages
Chinese (zh)
Other versions
CN114758440B (en
Inventor
王勇
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Chengdu Qiyuan Xipu Technology Co ltd
Original Assignee
Chengdu Qiyuan Xipu Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Chengdu Qiyuan Xipu Technology Co ltd filed Critical Chengdu Qiyuan Xipu Technology Co ltd
Priority to CN202011588979.0A priority Critical patent/CN114758440B/en
Publication of CN114758440A publication Critical patent/CN114758440A/en
Application granted granted Critical
Publication of CN114758440B publication Critical patent/CN114758440B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G07CHECKING-DEVICES
    • G07CTIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE
    • G07C9/00Individual registration on entry or exit
    • G07C9/30Individual registration on entry or exit not involving the use of a pass
    • G07C9/32Individual registration on entry or exit not involving the use of a pass in combination with an identity check
    • G07C9/35Individual registration on entry or exit not involving the use of a pass in combination with an identity check by means of a handwritten signature
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/22Matching criteria, e.g. proximity measures
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/25Fusion techniques
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • GPHYSICS
    • G07CHECKING-DEVICES
    • G07CTIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE
    • G07C9/00Individual registration on entry or exit
    • G07C9/30Individual registration on entry or exit not involving the use of a pass
    • G07C9/32Individual registration on entry or exit not involving the use of a pass in combination with an identity check
    • G07C9/37Individual registration on entry or exit not involving the use of a pass in combination with an identity check using biometric data, e.g. fingerprints, iris scans or voice recognition
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02DCLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Computing Systems (AREA)
  • Molecular Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Human Computer Interaction (AREA)
  • Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

The invention provides an access control system based on mixed recognition of images and characters, aiming at solving the problem that access control is invalid due to the fact that the access control security is possible to have counterfeit biological characteristics in the prior art. The system is based on the matching and fusion technology of two-dimensional and three-dimensional information, the character image information obtained by the on-site signature is combined with the three-dimensional image information obtained by face recognition, the dependence on biological characteristics is reduced, and the interference of counterfeiters on the face recognition by using the three-dimensional printing technology is avoided.

Description

Access control system based on mixed recognition of images and characters
Technical Field
The invention relates to the technical field of security identification, in particular to an access control system based on mixed identification of images and characters.
Background
The entrance guard management system can control the entrance and exit of personnel, can also control the behaviors of the personnel in buildings and sensitive areas, and records and counts the digital entrance and exit control system of management data. With the development of economy and society in China, the access control safety management system has been deeply developed in the aspect of life, and provides important guarantee for personal safety, property safety and information safety of people. The entrance guard safety management system is a modern safety management system, relates to a plurality of new technologies such as electronics, machinery, optics, computer technology, communication technology, biotechnology and the like, is an effective measure for solving the safety precaution management of the entrances and exits of important departments, and is suitable for various occasions such as banks, hotels, parking lot management, machine rooms, military machine storehouses, key rooms, offices, intelligent districts, factories and the like.
Disclosure of Invention
In order to overcome the problem of entrance guard safety caused by the fact that high-tech technology is utilized to imitate biological characteristics possibly existing in entrance guard safety in the prior art, such as the mode of using a 3D printed mask and the like, the invention provides an entrance guard system based on image and character mixed recognition, which comprises:
the system comprises an acquisition unit, a storage unit and a processing unit, wherein the acquisition unit is used for acquiring name character image information and face image information at the same moment, the name character image information and the face image information are associated and establish a pairing relation through the acquisition direction of a sensor for acquiring each information, the character image information is two-dimensional image information obtained by an on-site signature of a person to be detected, and the face image information is three-dimensional image information;
the system comprises a first input unit, a second input unit and a third input unit, wherein the first input unit is used for inputting name character image information and preprocessing the character image information, the name character image information comprises a plurality of groups of static image information of a first mode with the same interesting area and representing shapes and a second mode corresponding to the first mode and representing colors, and the first mode and the second mode are respectively positioned on different layers of a data structure of the static image information;
the second input unit is used for inputting face image information matched with name character image information, wherein the face image information comprises a plurality of groups of dynamic image information of at least shape modes with the same interested area and color modes, speed modes and distance modes corresponding to the shape modes, and the shape modes, the color modes, the speed modes and the distance modes are respectively positioned at different layers of a data structure of the dynamic image information;
the judging unit is used for judging whether the image information of each layer in different modes in each group of static image information is matched with each other; if the image information of each layer in different modes in each group of static image information is matched with each other, dividing the name character image information of each corresponding layer into a plurality of two-dimensional image blocks respectively; if the static image information of each layer in different modes in each group of static image information is not completely matched, performing three-dimensional reconstruction and registration on the static image information of the first mode in each group of data, and then segmenting to obtain a first set containing m layers of static image information of the first mode, wherein m is a natural number greater than 5; cleaning the information of the first set by using a morphological hole filling method, performing information fusion on each layer of sliced static image information in first-mode static image information and corresponding second-mode static image information in the same group by using a frequency domain information fusion method of discrete cosine transform, performing three-dimensional reconstruction and registration to obtain three-dimensional fusion information, wherein the first dimension is the information obtained by fusing the static image information corresponding to the first mode and the second mode, the second dimension is the static image information of the second mode representing color, the third dimension represents distance and is set to be 0, performing information fusion on the reconstructed three-dimensional fusion information and the dynamic image information, and marking the information obtained by fusion as a quasi-identification image sub-block with a direction according to the acquisition direction;
the training unit is used for training the neural network model by utilizing the pre-collected name character image information; setting the third dimension to be 0 by using the prepared image sub-blocks to be recognized in all directions so as to carry out two-dimension to obtain two-dimensional image sub-blocks, inputting the two-dimensional image sub-blocks into a neural network model, and carrying out similarity comparison on the obtained recognition result and one of two-dimensional face image information: if the similarity of the comparison result is smaller than a preset threshold value, the similarity comparison with other two-dimensional face image information is continued, otherwise, the model stops the iterative operation of the similarity comparison, and the model is saved.
Further, the preprocessing includes threshold processing to eliminate the influence of noise possibly existing in the text image information, and/or interpolation processing on the face image information to unify the resolution of different planes of the face image information.
Further, the direction includes three angles of 75 °, +90 °, 105 °.
Further, each set of the first-mode static image information and the second-mode static image information of the name character image information come from the same person to be detected.
Furthermore, each set of the first modality static image information and the second modality static image information of the name text image information come from different persons to be detected and serve as confusion data when the model is trained.
Furthermore, image information of the same modality is acquired by the same equipment.
Further, the device is a three-dimensional camera.
The invention has the beneficial effects that: the character image information obtained by the in-situ signature is combined with the three-dimensional image information obtained by the face recognition, so that the dependence on biological characteristics is reduced, and the interference of counterfeiters on the face recognition by using a three-dimensional printing technology is avoided.
Drawings
Fig. 1 shows a block diagram of the present system.
Detailed Description
An access control system based on mixed discernment of image and characters includes:
the system comprises an acquisition unit, a storage unit and a processing unit, wherein the acquisition unit is used for acquiring name character image information and face image information at the same moment, the name character image information and the face image information are associated and establish a pairing relation through the acquisition direction of a sensor for acquiring each information, the character image information is two-dimensional image information obtained by an on-site signature of a person to be detected, and the face image information is three-dimensional image information;
the system comprises a first input unit, a second input unit and a third input unit, wherein the first input unit is used for inputting name character image information and preprocessing the character image information, the name character image information comprises a plurality of groups of static image information of a first mode with the same interesting area and representing shapes and a second mode corresponding to the first mode and representing colors, and the first mode and the second mode are respectively positioned on different layers of a data structure of the static image information;
the second input unit is used for inputting face image information matched with name character image information, the face image information comprises a plurality of groups of dynamic image information of at least a shape mode with the same region of interest and a color mode, a speed mode and a distance mode corresponding to the shape mode, and the shape mode, the color mode, the speed mode and the distance mode are respectively positioned on different layers of a data structure of the dynamic image information;
the judging unit is used for judging whether the image information of each layer in different modes in each group of static image information is matched with each other; if the image information of each layer in different modes in each group of static image information is matched with each other, dividing the name, character and image information of each corresponding layer into a plurality of two-dimensional image blocks respectively; if the static image information of each layer in different modes in each group of static image information is not completely matched, performing three-dimensional reconstruction and registration on the static image information of the first mode in each group of data, and then segmenting to obtain a first set containing m layers of static image information of the first mode, wherein m is a natural number more than 5; cleaning the information of the first set by using a morphological hole filling method, performing information fusion on each layer of sliced static image information in the static image information of a first mode and the corresponding static image information of a second mode in the same group by using a frequency domain information fusion method of discrete cosine transform, performing three-dimensional reconstruction and registration to obtain three-dimensional fusion information, wherein the first dimension is the information obtained by fusing the static image information corresponding to the first mode and the second mode, the second dimension is the static image information of the second mode representing color, the third dimension represents distance and is set to be 0, the reconstructed three-dimensional fusion information is subjected to information fusion with the dynamic image information, and the information obtained by fusion is marked as a quasi-identification image sub-block with a direction according to the acquisition direction;
the training unit is used for training the neural network model by utilizing the pre-collected name character image information; setting the third dimension to be 0 by using the prepared image sub-blocks to be recognized in all directions so as to carry out two-dimension to obtain two-dimensional image sub-blocks, inputting the two-dimensional image sub-blocks into a neural network model, and carrying out similarity comparison on the obtained recognition result and one of two-dimensional face image information: if the similarity of the comparison result is smaller than the preset threshold value, the similarity comparison with other two-dimensional facial image information is continued, otherwise, the model stops the iteration operation of the similarity comparison, and the model is saved.
Preferably, the preprocessing includes thresholding to remove the effects of noise that may be present in the text image information and/or interpolation of the face image information to unify the resolution of different planes of the face image information.
Preferably, the direction comprises three angles 75 °, +90 °, 105 °.
Preferably, each set of the first-modality static image information and the second-modality static image information of the name text image information is from the same person to be detected.
Preferably, each set of the first-modality static image information and the second-modality static image information of the name text image information comes from different persons to be detected and is used as confusion data when the model is trained.
Preferably, image information of the same modality is acquired using the same apparatus.
Preferably, the device is a three-dimensional camera.
The above-mentioned embodiments only express several embodiments of the present invention, and the description thereof is specific and detailed, but not to be understood as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the inventive concept, which falls within the scope of the present invention. Therefore, the protection scope of the present patent should be subject to the appended claims.

Claims (7)

1. The utility model provides an access control system based on mixed discernment of image and characters which characterized in that includes:
the system comprises an acquisition unit, a display unit and a display unit, wherein the acquisition unit is used for acquiring name character image information and face image information at the same moment, the name character image information and the face image information are associated by the acquisition direction of a sensor for acquiring each piece of information to establish pairing connection, the character image information is two-dimensional image information, and the face image information is three-dimensional image information;
the device comprises a first input unit, a second input unit and a third input unit, wherein the first input unit is used for inputting name character image information and preprocessing the character image information, the name character image information comprises a plurality of groups of static image information of a first mode and a second mode, the first mode and the second mode have the same interesting area and represent shapes, the second mode corresponds to the first mode and represents colors, and the first mode and the second mode are respectively positioned at different layers of a data structure of the static image information;
the second input unit is used for inputting face image information matched with name character image information, the face image information comprises a plurality of groups of dynamic image information of at least a shape mode with the same region of interest and a color mode, a speed mode and a distance mode corresponding to the shape mode, and the shape mode, the color mode, the speed mode and the distance mode are respectively positioned on different layers of a data structure of the dynamic image information;
the judging unit is used for judging whether the image information of each layer in different modes in each group of static image information is matched with each other; if the image information of each layer in different modes in each group of static image information is matched with each other, dividing the name character image information of each corresponding layer into a plurality of two-dimensional image blocks respectively; if the static image information of each layer in different modes in each group of static image information is not completely matched, performing three-dimensional reconstruction and registration on the static image information of the first mode in each group of data, and then segmenting to obtain a first set containing m layers of static image information of the first mode, wherein m is a natural number more than 5; cleaning the information of the first set by using a morphological hole filling method, performing information fusion on each layer of sliced static image information in first-mode static image information and corresponding second-mode static image information in the same group by using a frequency domain information fusion method of discrete cosine transform, performing three-dimensional reconstruction and registration to obtain three-dimensional fusion information, wherein the first dimension is the information obtained by fusing the static image information corresponding to the first mode and the second mode, the second dimension is the static image information of the second mode representing color, the third dimension represents distance and is set to be 0, performing information fusion on the reconstructed three-dimensional fusion information and the dynamic image information, and marking the information obtained by fusion as a quasi-identification image sub-block with a direction according to the acquisition direction;
the training unit is used for training the neural network model by utilizing the pre-collected name character image information; setting the third dimension to be 0 by using the prepared image sub-blocks to be recognized in all directions so as to carry out two-dimension to obtain two-dimensional image sub-blocks, inputting the two-dimensional image sub-blocks into a neural network model, and carrying out similarity comparison on the obtained recognition result and one of two-dimensional face image information: if the similarity of the comparison result is smaller than a preset threshold value, the similarity comparison with other two-dimensional face image information is continued, otherwise, the model stops the iterative operation of the similarity comparison, and the model is saved.
2. The access control system based on mixed recognition of images and characters as claimed in claim 1, wherein: the pre-processing includes thresholding to remove the effects of noise that may be present in the text image information and/or interpolation of the facial image information to unify the resolution of different planes of facial image information.
3. The access control system based on image and text hybrid recognition of claim 1, wherein the direction comprises three angles of 75 °, +90 °, and 105 °.
4. The access control system based on mixed recognition of images and characters as claimed in claim 1, wherein: and each group of the first-mode static image information and the second-mode static image information of the name character image information come from the same person to be detected.
5. The access control system based on mixed recognition of images and characters as claimed in claim 1, wherein: and each group of first-mode static image information and second-mode static image information of the name character image information come from different persons to be detected and serve as confusion data during the training of the model.
6. The access control system based on mixed recognition of images and characters as claimed in claim 1, wherein: and image information of the same modality is acquired by the same equipment.
7. The access control system based on mixed recognition of images and characters as claimed in claim 6, wherein: the device is a three-dimensional camera.
CN202011588979.0A 2020-12-29 2020-12-29 Access control system based on image and text mixed recognition Active CN114758440B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011588979.0A CN114758440B (en) 2020-12-29 2020-12-29 Access control system based on image and text mixed recognition

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011588979.0A CN114758440B (en) 2020-12-29 2020-12-29 Access control system based on image and text mixed recognition

Publications (2)

Publication Number Publication Date
CN114758440A true CN114758440A (en) 2022-07-15
CN114758440B CN114758440B (en) 2023-07-18

Family

ID=82324488

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011588979.0A Active CN114758440B (en) 2020-12-29 2020-12-29 Access control system based on image and text mixed recognition

Country Status (1)

Country Link
CN (1) CN114758440B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114758439A (en) * 2020-12-29 2022-07-15 成都启源西普科技有限公司 Multimode access control system based on artificial intelligence

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2004008379A2 (en) * 2002-06-17 2004-01-22 Mason Ricardo Storm Identity verification
CN1689042A (en) * 2003-01-28 2005-10-26 富士通株式会社 Biometrics information registration apparatus, biometrics information matching apparatus, biometrics information registration/matching system, and biometrics information registration program
CN103426016A (en) * 2013-08-14 2013-12-04 湖北微模式科技发展有限公司 Method and device for authenticating second-generation identity card
CN105631272A (en) * 2016-02-02 2016-06-01 云南大学 Multi-safeguard identity authentication method
CN107507286A (en) * 2017-08-02 2017-12-22 五邑大学 A kind of bi-mode biology feature based on face and handwritten signature is registered system
US20200026908A1 (en) * 2018-07-23 2020-01-23 The Mitre Corporation Name and face matching
CN111599044A (en) * 2020-05-14 2020-08-28 哈尔滨学院 An access control security management system based on multimodal biometric identification
WO2020241987A1 (en) * 2019-05-27 2020-12-03 주식회사 시큐브 Apparatus and method for user authentication based on face recognition and handwritten signature verification
CN114758439A (en) * 2020-12-29 2022-07-15 成都启源西普科技有限公司 Multimode access control system based on artificial intelligence

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2004008379A2 (en) * 2002-06-17 2004-01-22 Mason Ricardo Storm Identity verification
CN1689042A (en) * 2003-01-28 2005-10-26 富士通株式会社 Biometrics information registration apparatus, biometrics information matching apparatus, biometrics information registration/matching system, and biometrics information registration program
CN103426016A (en) * 2013-08-14 2013-12-04 湖北微模式科技发展有限公司 Method and device for authenticating second-generation identity card
CN105631272A (en) * 2016-02-02 2016-06-01 云南大学 Multi-safeguard identity authentication method
CN107507286A (en) * 2017-08-02 2017-12-22 五邑大学 A kind of bi-mode biology feature based on face and handwritten signature is registered system
US20200026908A1 (en) * 2018-07-23 2020-01-23 The Mitre Corporation Name and face matching
WO2020241987A1 (en) * 2019-05-27 2020-12-03 주식회사 시큐브 Apparatus and method for user authentication based on face recognition and handwritten signature verification
CN111599044A (en) * 2020-05-14 2020-08-28 哈尔滨学院 An access control security management system based on multimodal biometric identification
CN114758439A (en) * 2020-12-29 2022-07-15 成都启源西普科技有限公司 Multimode access control system based on artificial intelligence

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114758439A (en) * 2020-12-29 2022-07-15 成都启源西普科技有限公司 Multimode access control system based on artificial intelligence

Also Published As

Publication number Publication date
CN114758440B (en) 2023-07-18

Similar Documents

Publication Publication Date Title
Zhang et al. Face morphing detection using Fourier spectrum of sensor pattern noise
CN107967458A (en) A kind of face identification method
CN101833647A (en) Palmprint image acquisition device and palmprint image processing method
CN107230267A (en) Intelligence In Baogang Kindergarten based on face recognition algorithms is registered method
CN111507320A (en) Detection method, device, equipment and storage medium for kitchen violation behaviors
Reddy et al. Classification of child and adulthood using GLCM based on diagonal LBP
Suresh et al. Deep learning-based image forgery detection system
CN114758440A (en) Access control system based on mixed recognition of images and characters
Long et al. Detection of Face Morphing Attacks Based on Patch‐Level Features and Lightweight Networks
WO2002009024A1 (en) Identity systems
CN114758439B (en) Multi-mode access control system based on artificial intelligence
CN112700576B (en) Multi-modal recognition algorithm based on images and characters
CN111582195B (en) Construction method of Chinese lip language monosyllabic recognition classifier
Gomathy et al. Face recognition based student detail collection using OpenCV
CN112258707A (en) Intelligent access control system based on face recognition
CN204537273U (en) A kind of photographing device of swiping the card for gate control system
Samal Minimum resolution for human face detection and identification
CN113538720A (en) Embedded face recognition attendance checking method based on Haisi intelligent AI chip
JP2008129679A (en) Fingerprint discrimination model construction method, fingerprint discrimination method, identification method, fingerprint discrimination device and identification device
CN116844203A (en) Face privacy protection method and device based on local directional adversarial attack
CN112084857B (en) A face recognition method and recognition system for video stream
Shichkina et al. Synthesis of the method of operative image analysis based on metadata and methods of searching for embedded images
Li et al. Crowd density estimation: An improved approach
Chen et al. Broad learning with uniform local binary pattern for fingerprint liveness detection
CN112927602A (en) Anti-counterfeiting random marker and printing method thereof

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant