CN111337050B - Zero-speed judgment condition and step counting method based on multi-condition fusion - Google Patents
Zero-speed judgment condition and step counting method based on multi-condition fusion Download PDFInfo
- Publication number
- CN111337050B CN111337050B CN202010201694.0A CN202010201694A CN111337050B CN 111337050 B CN111337050 B CN 111337050B CN 202010201694 A CN202010201694 A CN 202010201694A CN 111337050 B CN111337050 B CN 111337050B
- Authority
- CN
- China
- Prior art keywords
- condition
- particle
- zero
- acceleration
- angular velocity
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 32
- 230000004927 fusion Effects 0.000 title claims abstract description 19
- 230000001133 acceleration Effects 0.000 claims abstract description 35
- 239000002245 particle Substances 0.000 claims description 56
- 238000005457 optimization Methods 0.000 claims description 20
- 238000005070 sampling Methods 0.000 claims description 15
- 238000004364 calculation method Methods 0.000 claims description 4
- 238000012804 iterative process Methods 0.000 claims description 4
- 239000011541 reaction mixture Substances 0.000 claims description 4
- 238000001914 filtration Methods 0.000 claims description 3
- 238000001514 detection method Methods 0.000 description 3
- 230000003068 static effect Effects 0.000 description 3
- 238000006073 displacement reaction Methods 0.000 description 2
- 238000007796 conventional method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C22/00—Measuring distance traversed on the ground by vehicles, persons, animals or other moving solid bodies, e.g. using odometers, using pedometers
- G01C22/006—Pedometers
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
Abstract
The invention discloses a zero-speed judgment condition and step counting method based on multi-condition fusion, which comprises the steps of firstly wearing a portable acquisition device for a sporter to acquire accelerations in three directions of x, y and z and angular velocities in three directions; then calibrating zero-speed judgment condition thresholds of different moving personnel; and finally, judging the zero speed and finishing step counting based on the zero speed judgment method of the multi-condition fusion. The zero-speed judgment method introduces different zero-speed judgment conditions to jointly judge the zero speed, so that the zero-speed judgment is more accurate; and the number of the moving steps is calculated based on the proposed accurate zero-speed judgment method, so that the step counting is more accurate.
Description
Technical Field
The invention relates to a step counting method, in particular to a step counting strategy of a portable motion acquisition device for rehabilitation motion, and belongs to the technical field of motion analysis.
Background
The portable motion parameter acquisition equipment brings new development for rehabilitation exercises. The rehabilitation crowd can train at home, and the doctor can analyze the motion data from the internet. The key to the motion analysis of the rehabilitation population is the accurate detection of foot contact time periods during walking, i.e., zero velocity time periods.
The portable motion parameter acquisition equipment comprises an acceleration sensor and an angular velocity sensor. Due to the inaccuracy of the sensors themselves, the speed and position of the wearer gradually diverge. The zero velocity detection method updates the velocity within a step by detecting the zero velocity of each step of the wearer, thereby reducing errors. Typically, each person has a zero speed interval of 0.2-0.4 seconds in normal walking. This interval begins with full foot contact and ends with the beginning of foot lift advancement. Zero velocity detection is critical to accurately measuring the velocity and displacement of the wearer. In the conventional method, a condition that the difference between the acceleration and the angular velocity is lower than a threshold is regarded as a static condition, a condition that a certain component value of the acceleration and the angular velocity is lower than the threshold is regarded as a static condition, or a condition that the angular velocity modulo is lower than a certain threshold is regarded as a static judgment condition. These quiescent conditions do not identify the zero speed interval one hundred percent.
Disclosure of Invention
The invention provides a zero-speed judgment condition with multi-condition fusion, and provides a step counting algorithm on the basis of the zero-speed judgment condition, aiming at the problem that the current portable motion acquisition equipment needs to judge the motion zero speed and the current zero-speed judgment condition cannot be used for judging the zero speed in a hundred percent.
The invention provides a zero-speed judgment condition and step counting method based on multi-condition fusion, which comprises the following steps:
step 2, calibrating zero-speed judgment condition thresholds of different sports personnel;
and 3, judging the zero speed and finishing step counting based on the proposed zero speed judgment method of multi-condition fusion.
The invention has the advantages that:
1. according to the zero speed judgment condition and the step counting method based on multi-condition fusion, different zero speed judgment conditions are introduced to jointly judge the zero speed, so that the zero speed judgment is more accurate;
2. the zero speed judgment condition and the step counting method based on multi-condition fusion calculate the number of the moving steps based on the proposed accurate zero speed judgment method, so that the step counting is more accurate.
Drawings
FIG. 1 is an overall flowchart of the zero-speed judgment condition and step counting method based on multi-condition fusion.
FIG. 2 is a flow chart of an objective function in the zero-speed judgment condition and step counting method based on multi-condition fusion according to the present invention.
FIG. 3 is a flow chart of a particle group optimization algorithm in the zero-speed judgment condition and step counting method based on multi-condition fusion.
FIG. 4 is a flow chart of the method for judging zero speed and counting steps by multi-condition fusion according to the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings.
The zero-speed judgment condition and the step counting method based on multi-condition fusion firstly realize a practical and effective zero-speed identification method, and count steps on the basis of the method, and the embodiment of the invention is described by the following three steps as shown in figure 1.
The method comprises the following steps: and acquiring acceleration and angular speed signals measured by the portable measuring equipment.
The portable measuring equipment is worn on the instep of the person to be measured, and no relative displacement exists between the portable measuring equipment and the person to be measured. Measuring the accelerations [ ax, ay, az ] of N measured personnel in three directions of space x, y and z and the angular velocities [ gx, gy and gz ] of the three directions by measuring equipment at a certain sampling period;
step two: and (4) calibrating thresholds under different conditions.
A. The different conditions include a modulus condition of acceleration and angular velocity, a difference condition of acceleration and angular velocity, and a duration condition. Specifically, the acceleration signals acquired by the kth sampling are set as a (k) x, a (k) y and a (k) z, and the angular velocity signals are set as g (k) x, g (k) y and g (k) z; then:
a. the acceleration mode conditions are:
in the formula (1), a th The threshold, representing the acceleration mode condition, is the quantity to be sought.
b. The angular velocity mode conditions were:
in the formula (2), g th The threshold value representing the angular velocity mode condition is the quantity to be sought.
c. The acceleration difference condition is:
in the formula (3), the reaction mixture is,
threshold representing acceleration differential condition, a (k-1) x ,a(k-1) y ,a(k-1) z Representing acceleration signals in three directions of x, y and z acquired by the k-1 th sampling.
d. The angular velocity difference condition is:
in the formula (4), the reaction mixture is,
threshold value representing angular velocity difference condition, g (k-1) x ,g(k-1) y ,g(k-1) z Representing the angular velocity signals of the x, y and z directions acquired by the k-1 th sampling.
e. The time length condition is as follows:
if the acquired acceleration signal and the acquired angular velocity signal start to satisfy the modulus condition and the difference condition from the sampling of the s-th time, and one of the modulus condition and the difference condition is not satisfied until the sampling of the e-th time, the time duration condition is as follows:
(e-s)T s ＞t th (5)
wherein, T s Representing the sampling period, t th Representing a time condition threshold.
The foot contact time is determined using a logical and combination of three conditions. Identification of optimal a by threshold calibration th ,g th ,
And t th And the motion data meeting the mold condition, the differential condition and the duration condition are all the data acquired when the foot touches the ground, and the unsatisfied data are all the motion data generated when the foot does not touch the ground.
B. Before the threshold value is calibrated, an objective function of the optimization algorithm needs to be defined. Specifically, as shown in fig. 2, from the data collected in step 1, the motion data of N persons are filteredThe maximum value a of the modulus of all the acceleration data and angular velocity data is obtained max Minimum value a min And maximum differential acceleration
Minimum value>
Differential maximum angular velocity->
Minimum value>
When each person walks, the longest period of the minimum touchdown period is T. The range of the mode is [ a ] min ,a max ]，[g min ,g max ](ii) a Has a difference range of->
The duration range is [0, T ]]. Within these different ranges, the variable a to be optimized is given th ,g th ,
t th
Starting from the first person's movement, let the initial optimization objective cost be zero. Let i =1, for the movement of the ith person, the number of time periods satisfying the modulus condition, the duration condition and the difference condition is obtained, that is, the number of times of touchdown is calculated by the algorithm
Judging whether the contact time of the ith individual is step according to the walking periodicity i . Local optimization objectives for the ith individual are derived:
and updating the optimization objective:
adding 1 to i, and repeating the calculation steps to calculate Qi; until i is greater than N. The optimization objective is
The final value of (c).
C. Carrying out threshold calibration by utilizing a particle swarm optimization algorithm, specifically, solving the optimal a by utilizing the particle swarm optimization algorithm th ,g th ,
t th The objective function is minimized. The particle swarm optimization algorithm has an optimization variable dimension of 5, and is shown in fig. 3, specifically as follows:
(a) Setting the number of population particles as m and the particle dimension as 5;
(b) Randomly initializing the speed and position of each particle in the population to obtain the position of the initial population
The lower corner marks the particle number u m Is a five-dimensional vector representing the position of the m-th particle, which is a possible solution in the optimization process, namely a th ,g th ,
t th The upper corner mark "1" indicates that the 1 st iteration is currently performed; setting an initial population velocity v (1);
(c) In an iterative process, the objective function values for each possible solution are compared. Setting the optimal position (the objective function value is minimum) of each particle in the iterative process as the optimal position of the particle, wherein the optimal position of the ith particle is p best,i (ii) a The optimal positions in all the particles are obtained through comparison and are set as the optimal positions g of the particle swarm best ；
(d) And updating the particle speed and the position of each particle, taking the ith particle as an example, and expressing the following expression:
where v is the particle velocity, u is the particle position, w is the inertial weight, r 1 And r 2 Is distributed in the interval [0,1]The random number in the table, s is the current iteration number, the initial value is 1,
for the individual optimal particle position of the ith particle at the s-th iteration,
is the global optimum particle position at the s-th generation, c 1 And c 2 Is a constant. />
And further obtaining the s +1 generation population position:
(e) Calculating the objective function value of each particle in the step s +1 and comparing the objective function value with the previous optimal position
Comparing the obtained objective function values, and if the current position is better (the value of the standard function is smaller), taking the current position as the optimal position (or greater or lesser value) of the particle>
Combining the objective function value of each particle with the optimal particle position of the particle swarm>
In comparison, if the current position is better (the objective function value is smaller), the optimal particle position is updated->
(f) Checking the final value condition, if the precision (the size of the objective function value) meets a preset condition or the iteration times exceed the limit, stopping iteration, and otherwise, repeating the steps (c) to (f);
(g) Outputting the optimal position of the particle swarm
I.e. the optimum a th ,g th 
t th 。
Step four: using the solved optimal a th ,g th ,
t th The touchdown period determination is made and steps are counted, as shown in fig. 4. Specifically, for acceleration and angular velocity data acquired by a section of motion, firstly, filtering the data, then calculating an acceleration mode, an angular velocity mode, an acceleration difference and an angular velocity difference of each sampling moment in the section of motion, searching a data section which continuously meets a mode condition and a difference condition in the filtered data, and judging whether the duration of the data section meets a duration condition t th If yes, the time interval is a touchdown time interval, the absolute speed is 0, the times of the time interval meeting the conditions are counted, and the obtained times are the walking steps; if not, the time period is not a touchdown period.
The threshold condition calculation method avoids complex attempts of a traditional method for searching and judging the threshold, provides a new idea for threshold condition calculation, integrates the module conditions of angular velocity and acceleration, difference conditions and duration conditions by the threshold condition and the step counting method, and can effectively reduce step counting errors.
Claims (4)
1. A zero-speed judgment condition and step counting method based on multi-condition fusion is characterized in that:
step 1: acquiring acceleration [ ax, ay, az ] in three directions of x, y and z and angular velocity [ gx, gy and gz ] in three directions by a portable acquisition device worn by a sportsman;
and 2, step: calibrating zero-speed judgment condition thresholds of different sports persons;
and step 3: judging the zero speed and finishing step counting based on a zero speed judgment method of multi-condition fusion;
the specific method of the step 2 comprises the following steps:
A. the different conditions comprise the acceleration and angular velocity module conditions, the acceleration and angular velocity difference conditions and the duration conditions; specifically, the acceleration signals acquired by the kth sampling are set as a (k) x, a (k) y and a (k) z, and the angular velocity signals are set as g (k) x, g (k) y and g (k) z; then:
a. the acceleration mode conditions are:
in the formula (1), a th A threshold value representing an acceleration mode condition is a quantity to be solved;
b. the angular velocity mode conditions were:
in the formula (2), g th A threshold value representing the angular velocity mode condition is a quantity to be solved;
c. the acceleration difference condition is:
in the formula (3), the reaction mixture is,
threshold representing acceleration differential condition, a (k-1) x ,a(k-1) y ,a(k-1) z Representing acceleration signals in x, y and z directions acquired by sampling at the (k-1) th time;
d. the angular velocity difference condition is:
in the formula (4), the reaction mixture is,
threshold value representing angular velocity difference condition, g (k-1) x ,g(k-1) y ,g(k-1) z Representing angular speed signals in the x direction, the y direction and the z direction acquired by the (k-1) th sampling;
e. the time length condition is as follows:
if the acceleration signal and the angular velocity signal which are collected from the s-th sampling start to satisfy the modulus condition and the difference condition, and one of the modulus condition and the difference condition is not satisfied from the e-th sampling, the time length condition is as follows:
(e-s)T s ＞t th
wherein, T s Representing the sampling period, t th A representative duration condition threshold;
judging the foot contact time by using the logic and combination of the three conditions; identification of optimality by thresholding
And t th The motion data meeting the mold condition, the differential condition and the duration condition are all data acquired when the foot touches the ground, and the unsatisfied data are all motion data generated when the foot does not touch the ground;
B. solving optimality using particle swarm optimization
Minimizing the objective function.
2. The method for zero-speed judgment and step counting based on multi-condition fusion as claimed in claim 1, wherein: before the threshold value is calibrated, an objective function of an optimization algorithm needs to be defined; filtering the motion data of N persons from the data collected in the step 1, and then obtaining the maximum value a of the modulus of all the acceleration data and the angular velocity data max Minimum value a min And maximum differential acceleration
Minimum value->
Angular velocity differential maximum>
Minimum value->
When each person walks, the longest time duration in the minimum touchdown time period is T; a mode range of [ a ] min ,a max ]，[g min ,g max ](ii) a Differential range of->
The duration range is [0, T](ii) a Within these different ranges, the variable to be optimized is given>
Starting from the movement of the first person, making the initial optimization target cost zero; let i =1, for the motion of the ith person, the number of time periods satisfying the model condition, the duration condition and the difference condition is obtained, that is, the number of times of touchdown is calculated for the algorithm
Judging whether the contact time of the ith individual is step according to the walking periodicity i (ii) a Local optimization objectives for the ith individual are derived:
and updating the optimization objective:
adding 1 to i, and repeating the calculation steps to calculate Qi; until i is greater than N; the optimization objective is that
The final value of (c).
3. The method for zero-speed judgment and step counting based on multi-condition fusion as claimed in claim 1, wherein: the specific method of the step B comprises the following steps:
the particle swarm optimization algorithm optimizes the variable dimension to be 5, and the optimization algorithm is as follows:
(a) Setting the number of population particles as m and the particle dimension as 5;
(b) Randomly initializing the speed and position of each particle in the population to obtain the position of the initial population
The lower corner marks the particle number u m Is a five-dimensional vector representing the position of the mth particle, which is a possible solution in the optimization process, i.e., -greater than or equal to->
The upper corner mark "1" indicates that the 1 st iteration is currently performed; setting an initial population velocity v (1);
(c) In the iterative process, comparing objective function values of all possible solutions; setting the optimal position of each particle in the iterative process and the minimum objective function value as the optimal position of the particle, wherein the optimal position of the ith particle is p best,i (ii) a The optimal position in all the particles is set as the optimal position g of the particle swarm through comparison best ；
(d) And updating the particle speed and the position of each particle, taking the ith particle as an example, and expressing the following expression:
where v is the particle velocity, u is the particle position, w is the inertial weight, r 1 And r 2 Is distributed in the interval [0,1]The random number in the table, s is the current iteration number, the initial value is 1,
for the individual optimal particle position of the ith particle at the s-th iteration,
is the global optimum particle position at the s-th generation, c 1 And c 2 Is a constant;
and further obtaining the s +1 generation population position:
(e) Calculating the objective function value of each particle in the step s +1 and comparing the objective function value with the previous optimal position
Comparing the obtained objective function values, and if the current position is better and the objective function value is smaller, taking the current position as the optimal position of the particle
Combining the value of the objective function for each particle with the optimum particle position for the particle swarm>
Comparing, if the current position is better and the objective function value is smaller, updating the optimal particle position->
(f) Checking the final value condition, if the precision and the size of the objective function value meet the preset condition or the iteration times exceed the limit, stopping iteration, and otherwise, repeating the steps (c) to (f);
(g) Outputting the optimal position of the particle swarm
Is optimally->
4. The method for zero-speed judgment and step counting based on multi-condition fusion as claimed in claim 1, wherein: the specific method of the step 3 comprises the following steps: for acceleration and angular velocity data acquired by a section of motion, firstly filtering the data, then calculating an acceleration module, an angular velocity module, an acceleration difference and an angular velocity difference of each sampling moment in the section of motion, searching a data section which continuously meets a module condition and a difference condition in the filtered data, judging whether the duration of the data section meets a duration condition, if so, the duration is a touchdown time period, the absolute velocity is 0, counting the times of the time period meeting the condition, and the obtained times are walking steps; if not, the time period is not a touchdown period.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010201694.0A CN111337050B (en) | 2020-03-20 | 2020-03-20 | Zero-speed judgment condition and step counting method based on multi-condition fusion |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010201694.0A CN111337050B (en) | 2020-03-20 | 2020-03-20 | Zero-speed judgment condition and step counting method based on multi-condition fusion |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111337050A CN111337050A (en) | 2020-06-26 |
| CN111337050B true CN111337050B (en) | 2023-03-28 |
Family
ID=71180367
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010201694.0A Active CN111337050B (en) | 2020-03-20 | 2020-03-20 | Zero-speed judgment condition and step counting method based on multi-condition fusion |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111337050B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114019182B (en) * | 2021-11-04 | 2024-02-02 | 苏州挚途科技有限公司 | Zero-speed state detection method and device and electronic equipment |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106225786A (en) * | 2016-08-15 | 2016-12-14 | 北京理工大学 | A kind of adaptive pedestrian navigation system zero-speed section detecting method |
| WO2017124809A1 (en) * | 2016-01-21 | 2017-07-27 | 上海斐讯数据通信技术有限公司 | Particle swarm optimization method and system based on gpu operation of mobile terminal |
| WO2019109757A1 (en) * | 2017-12-06 | 2019-06-13 | 华南理工大学 | Method for using particle swarm algorithm to optimize power electronic circuit |
| CN110274592A (en) * | 2019-07-18 | 2019-09-24 | 北京航空航天大学 | A kind of zero-speed section of waist foot Inertial Measurement Unit information fusion determines method |
| CN110553643A (en) * | 2019-09-17 | 2019-12-10 | 电子科技大学 | pedestrian self-adaptive zero-speed updating point selection method based on neural network |
| CN110715659A (en) * | 2019-10-25 | 2020-01-21 | 高新兴物联科技有限公司 | Zero-speed detection method, pedestrian inertial navigation method, device and storage medium |
-
2020
- 2020-03-20 CN CN202010201694.0A patent/CN111337050B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017124809A1 (en) * | 2016-01-21 | 2017-07-27 | 上海斐讯数据通信技术有限公司 | Particle swarm optimization method and system based on gpu operation of mobile terminal |
| CN106225786A (en) * | 2016-08-15 | 2016-12-14 | 北京理工大学 | A kind of adaptive pedestrian navigation system zero-speed section detecting method |
| WO2019109757A1 (en) * | 2017-12-06 | 2019-06-13 | 华南理工大学 | Method for using particle swarm algorithm to optimize power electronic circuit |
| CN110274592A (en) * | 2019-07-18 | 2019-09-24 | 北京航空航天大学 | A kind of zero-speed section of waist foot Inertial Measurement Unit information fusion determines method |
| CN110553643A (en) * | 2019-09-17 | 2019-12-10 | 电子科技大学 | pedestrian self-adaptive zero-speed updating point selection method based on neural network |
| CN110715659A (en) * | 2019-10-25 | 2020-01-21 | 高新兴物联科技有限公司 | Zero-speed detection method, pedestrian inertial navigation method, device and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111337050A (en) | 2020-06-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| AU781848B2 (en) | Pedestrian navigation method and apparatus operative in a dead reckoning mode | |
| US11896368B2 (en) | Systems and methods for determining swimming metrics | |
| EP2165203B1 (en) | Method and device for measuring the progress of a moving person | |
| US20050033200A1 (en) | Human motion identification and measurement system and method | |
| US20140172361A1 (en) | Multi-posture stride length calibration system and method for indoor positioning | |
| CN109297485B (en) | Indoor personal inertial navigation elevation precision improving method based on height self-observation algorithm | |
| Wang et al. | Swimming motion analysis and posture recognition based on wearable inertial sensors | |
| Tumkur et al. | Modeling human walking for step detection and stride determination by 3-axis accelerometer readings in pedometer | |
| CN103630147B (en) | A kind of individual autonomy navigation system zero-speed detection method based on HMM | |
| CN106225786A (en) | A kind of adaptive pedestrian navigation system zero-speed section detecting method | |
| US20210093917A1 (en) | Detecting outdoor walking workouts on a wearable device | |
| US12249185B2 (en) | System and method for exercise type recognition using wearables | |
| Ryu et al. | Adaptive step detection algorithm for wireless smart step counter | |
| CN109211229A (en) | A kind of personnel's indoor orientation method based on mobile phone sensor and WiFi feature | |
| Yang et al. | Symmetrical-Net: Adaptive zero velocity detection for ZUPT-aided pedestrian navigation system | |
| AU2015200031A1 (en) | Estimation of direction of motion of users on mobile devices | |
| CN111337050B (en) | Zero-speed judgment condition and step counting method based on multi-condition fusion | |
| US12215975B2 (en) | Methods and systems for adaptive pedestrian inertial navigation | |
| CN108537101A (en) | A kind of pedestrian's localization method based on state recognition | |
| Wu et al. | Indoor positioning system based on inertial MEMS sensors: Design and realization | |
| CN105953794B (en) | A kind of meter step air navigation aid based on MEMS sensor | |
| KR101609813B1 (en) | Apparatus and method for counting step in smartphone | |
| Chen et al. | Pedestrian positioning with physical activity classification for indoors | |
| CN109084765B (en) | A pedestrian indoor walking positioning method, device and storage medium | |
| CN106225803A (en) | A Step Counting Method Based on Triaxial Accelerator |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |