[go: up one dir, main page]

CN116736288B - Indoor human body target tracking method and system based on millimeter wave radar - Google Patents

Indoor human body target tracking method and system based on millimeter wave radar

Info

Publication number
CN116736288B
CN116736288B CN202310447818.7A CN202310447818A CN116736288B CN 116736288 B CN116736288 B CN 116736288B CN 202310447818 A CN202310447818 A CN 202310447818A CN 116736288 B CN116736288 B CN 116736288B
Authority
CN
China
Prior art keywords
point
points
partition
target
distance
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202310447818.7A
Other languages
Chinese (zh)
Other versions
CN116736288A (en
Inventor
邓维波
左鑫
吴小川
董英凝
菅宇鹏
李文隆
马占超
杨松岩
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhongyi Zhilian Wuxi Industrial Automation Technology Co ltd
Harbin Institute of Technology Shenzhen
Original Assignee
Zhongyi Zhilian Wuxi Industrial Automation Technology Co ltd
Harbin Institute of Technology Shenzhen
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhongyi Zhilian Wuxi Industrial Automation Technology Co ltd, Harbin Institute of Technology Shenzhen filed Critical Zhongyi Zhilian Wuxi Industrial Automation Technology Co ltd
Priority to CN202310447818.7A priority Critical patent/CN116736288B/en
Publication of CN116736288A publication Critical patent/CN116736288A/en
Application granted granted Critical
Publication of CN116736288B publication Critical patent/CN116736288B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G01MEASURING; TESTING
    • G01SRADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S13/00Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
    • G01S13/66Radar-tracking systems; Analogous systems
    • G01S13/72Radar-tracking systems; Analogous systems for two-dimensional tracking, e.g. combination of angle and range tracking, track-while-scan radar
    • G01S13/723Radar-tracking systems; Analogous systems for two-dimensional tracking, e.g. combination of angle and range tracking, track-while-scan radar by using numerical data
    • G01S13/726Multiple target tracking

Landscapes

  • Engineering & Computer Science (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Radar Systems Or Details Thereof (AREA)

Abstract

The application discloses an indoor human body target tracking method and system based on millimeter wave radar, which relate to the technical field of radar positioning and aim at the problem of low tracking efficiency and accuracy of the human body target tracking method in the prior art, and the application enables an algorithm to cluster according to the proportion of the distance between data instead of a fixed value by specifying coefficients, and the clustering can be adaptively adjusted according to the distance between the target and the radar; through the contour coefficients, the clustering mode with the best effect can be selected, and then the accuracy and the efficiency of indoor human body target detection are improved.

Description

Indoor human body target tracking method and system based on millimeter wave radar
Technical Field
The invention relates to the technical field of radar positioning, in particular to an indoor human body target tracking method and system based on millimeter wave radar.
Background
Lei Daqun target tracking refers to monitoring and tracking the motion of multiple targets in a radar system. In general, radar systems can detect a large number of indoor human targets, which if not classified and tracked, can easily lead to information confusion and target tracking errors, thereby reducing the efficiency and accuracy of the detection system. The clustering algorithm is widely applied to Lei Daqun target tracking, can classify and analyze targets detected by the radar, and improves the accuracy and instantaneity of target tracking. Compared with other target tracking algorithms, the clustering algorithm can better identify the behavior patterns of a plurality of targets, separate the data sets better, and reduce the possibility of misjudgment as much as possible while ensuring the tracking efficiency. The clustering algorithm can effectively group a large number of radar-detected targets and further analyze the behavior patterns and motion laws of each group. This information helps us better grasp the motion changes of the target, thus achieving more accurate, real-time target tracking. Another important application scenario of the clustering algorithm is data preprocessing for target tracking. For example, redundant information in radar data can be deleted by using a clustering technology, so that subsequent target tracking is more accurate. In addition, in the process of target tracking, different grouping strategies can be adopted in real time and applied to a density-based clustering algorithm, so that targets with different modes can be tracked and predicted. In an indoor environment, the human body target tracking is performed by using the prior art, and the problems of low tracking efficiency and low accuracy can occur.
Disclosure of Invention
The invention aims to provide an indoor human body target tracking method and system based on millimeter wave radar, aiming at the problems of low tracking efficiency and accuracy of the human body target tracking method in the prior art.
The technical scheme adopted by the invention for solving the technical problems is as follows:
an indoor human body target tracking method based on millimeter wave radar comprises the following steps:
Firstly, acquiring echo time domain original data of an indoor human body target by utilizing a millimeter wave radar, reconstructing the original data to form a matrix, wherein the matrix size is ADCNum x NumChirp (NTx NRx), and then respectively carrying out two-dimensional Fourier transform on each channel in the matrix to obtain an RD (remote data) spectrum matrix;
Wherein NumChirp denotes the number of chirp, ADCNum denotes the number of sampling points contained in each chirp, NTx denotes the number of transmitting antennas, and NRx denotes the number of receiving antennas;
Step two, static clutter filtering and constant false alarm detection are carried out on the RD spectrum matrix, and then the distance between a target point and the radar is obtained;
measuring angle information of a target point through an angle measurement algorithm, and obtaining the abscissa of the target point through the distance between the target point and the radar and the angle information, so as to construct a data set;
step four, obtaining an reachable distance sequence by utilizing the horizontal coordinates and the vertical coordinates of the target points in the data set;
Step five, clustering and dividing each reachable distance in the reachable distance sequence to obtain all clustering and dividing of the target point;
Step six, calculating the contour coefficients of each cluster partition in all cluster partitions of the target point, and selecting the partition with the largest contour coefficient as a clustering Result, namely the state of the target at the moment;
and seventhly, repeating the second step to the sixth step to track the indoor human body target.
Further, the specific steps of the fourth step are as follows:
Step four, core distances of all the target points in the data set are obtained according to the abscissa of the target points, then the target point corresponding to the maximum core distance is selected as an initial observation point, the rest target points are all set as unobserved points, and then the initial reachable distances of all the unobserved points are set as the core distances of the initial observation points;
Step four, respectively obtaining a distance value sequence A between the observation point and all the unobserved points and a core distance value B of the observation point, respectively comparing each value in the distance value sequence A with B according to the sequence order, sequentially reserving the maximum value in each comparison to obtain an M sequence, respectively comparing each value in the current reachable distance of the unobserved point with the value of the corresponding position in the M sequence, and if the value in the current reachable distance of the unobserved point is larger than the corresponding value in the M sequence, replacing the corresponding value in the reachable distance of the unobserved point with the corresponding value in the M sequence, otherwise, not replacing the value;
And fourthly, finding out the point with the smallest reachable distance in the unobserved points, taking the point as a new observation point, marking the previous observation point as observed, and then not comparing the points, repeating the fourth and the second steps until all the points are observed, finally recording the target points which are taken as the observation points each time, sorting the target points, and constructing a reachable distance table according to the reachable distance corresponding to each target point in the sorting.
Further, the specific steps of the fifth step are as follows:
Traversing an reachable distance table, wherein a target point with the reachable distance change exceeding a specified coefficient xi is a mutation point, and representing a division by every two mutation points, if the mutation point is one, only one division is needed by the data set, namely the data set itself, and the division comprises a starting point and an ending point;
Step five, sorting the partitions obtained through the mutation points according to the starting points in the partitions from small to large, defaulting that all the partitions do not conflict, taking all the partitions except the partition corresponding to the first starting point as undetermined partitions, and taking the partition corresponding to the first starting point as reference partitions;
Step five, starting with the partition corresponding to the minimum starting point in the ordering without judgment, judging whether the partition is overlapped with the reference partition from small to large, if so, taking the partition overlapped with the reference partition in the ordering as conflict partition, if not, merging the partition with the reference partition as new reference partition,
Repeating the fifth step until the last division in the sequence is judged, and taking the finally obtained reference division as a clustering division result;
and step five, deleting the last conflict-causing partition from the finally obtained reference partitions to obtain new reference partitions, taking all partitions after the last conflict-causing partition as undetermined partitions, and executing step five three until all partitions are judged, wherein all cluster partition results are obtained as final cluster partition results.
Further, the specific steps of the step six are as follows:
Calculating the contour coefficients of all cluster partitions of the target point, selecting the partition with the largest contour coefficient as a clustering Result, and determining the state of the target at the moment according to the clustering Result and the radial speed of the target point relative to the radar, namely the state of the target at the moment;
and the radial speed of the target point relative to the radar is obtained by carrying out static clutter filtering and constant false alarm detection on the RD spectrum matrix in the second step.
Further, the contour coefficients are expressed as:
Where a i represents the average distance of the i-th point to other points in the same cluster and b i represents the minimum average distance of the point to other points in different clusters.
Further, the minimum point number in the cluster division is 5.
An indoor human body target tracking system based on millimeter wave radar comprises an indoor human body target echo time domain original data acquisition module, a data set construction module and an indoor human body target tracking module;
The indoor human body target echo time domain original data acquisition module is used for acquiring echo time domain original data of an indoor human body target by utilizing a millimeter wave radar, reconstructing the original data to form a matrix, and then respectively carrying out two-dimensional Fourier transform on each channel in the matrix to obtain an RD spectrum matrix, wherein NumChirp represents the number of chirp, ADCNum represents the number of sampling points contained in each chirp, NTx represents the number of transmitting antennas, and NRx represents the number of receiving antennas;
the data set construction module is used for obtaining the distance between the target point and the radar after static clutter filtering and constant false alarm detection are carried out on the RD spectrum matrix, measuring the angle information of the target point through an angle measurement algorithm, and obtaining the abscissa of the target point through the distance between the target point and the radar and the angle information, so as to construct a data set;
The indoor human body target tracking module is used for obtaining an reachable distance sequence by utilizing the horizontal and vertical coordinates of the target points in the data set, then clustering and dividing each reachable distance in the reachable distance sequence to obtain all cluster divisions of the target points, then calculating the contour coefficients of each cluster division in all cluster divisions of the target points, and selecting the division with the largest contour coefficient as a clustering Result, namely the state of the target at the moment;
the indoor human body target is tracked by the indoor human body target echo time domain original data acquisition module, the data set construction module and the indoor human body target tracking module.
Further, the specific steps of obtaining the reachable distance sequence by utilizing the horizontal and vertical coordinates of the target point in the data set in the indoor human body target tracking module are as follows:
step 1, obtaining core distances of all target points in a data set according to the abscissa of the target points, then selecting the target point corresponding to the maximum core distance as an initial observation point, setting the rest target points as unobserved points, and setting the initial reachable distances of all the unobserved points as the core distances of the initial observation points;
Step 2, respectively solving a distance value sequence A between the observation point and all the unobserved points and a core distance value B of the observation point, respectively comparing each value in the distance value sequence A with B according to the sequence order, sequentially reserving the maximum value in each comparison to obtain an M sequence, respectively comparing each value in the current reachable distance of the unobserved point with the value of the corresponding position in the M sequence, and if the value in the current reachable distance of the unobserved point is larger than the corresponding value in the M sequence, replacing the corresponding value in the reachable distance of the unobserved point with the corresponding value in the M sequence, otherwise, not replacing the value;
And 3, finding out the point with the smallest reachable distance in the unobserved points, taking the point as a new observation point, marking the previous observation point as observed, and then not comparing the points, repeating the step 2 until all the points are observed, finally recording the target points which are taken as the observation points each time, sorting the target points, and constructing a reachable distance table according to the reachable distance corresponding to each target point in the sorting.
Further, the specific steps of clustering and dividing each reachable distance in the reachable distance sequence in the indoor human body target tracking module to obtain all the clustered and divided target points are as follows:
step A, traversing an reachable distance table, wherein a target point with the reachable distance change exceeding a specified coefficient xi is a mutation point, and representing a division by every two mutation points, if the mutation point is one, only one division is performed by the data set, namely the data set itself, and the division comprises a starting point and an ending point;
Step B, sorting the partitions obtained through the mutation points according to the starting points in the partitions from small to large, defaulting that all the partitions do not conflict, taking all the partitions except the partition corresponding to the first starting point as undetermined partitions, and taking the partition corresponding to the first starting point as reference partitions;
step C, starting with the partition corresponding to the minimum starting point in the ordering without judgment, judging whether the partition is overlapped with the reference partition from small to large, if so, taking the partition overlapped with the reference partition in the ordering as conflict partition, if not, merging the partition with the reference partition as new reference partition,
Step D, repeating the step C until the last division in the sequence is judged, and taking the finally obtained reference division as a clustering division result;
And E, deleting the last conflict-causing partition from the finally obtained reference partitions to obtain new reference partitions, taking all partitions after the last conflict-causing partition as undetermined partitions, and then executing the step C until all partitions are judged, wherein all cluster partition results obtained are final cluster partition results.
Further, the contour coefficients are expressed as:
Where a i represents the average distance of the i-th point to other points in the same cluster and b i represents the minimum average distance of the point to other points in different clusters.
The beneficial effects of the invention are as follows:
the method and the system enable the algorithm to cluster according to the ratio of the distance between the data instead of a fixed value through the specified coefficients, enable the clustering to be adaptively adjusted according to the distance between the target and the radar, select the clustering mode with the best effect through the contour coefficients, and further improve the accuracy and the efficiency of indoor human body target detection.
Under the indoor tracking condition, the specific number of targets cannot be known in advance, so that different target numbers need to be clustered respectively by applying a K-means clustering algorithm, and then the optimal target number is judged according to additional judging parameters. In order to reduce the dependence of the K-means algorithm on initial values, the random values can be taken for clustering, so that the algorithm is complicated and redundant, the hierarchical clustering algorithm can lead to a plurality of obtained clustering results, the algorithm is long in operation time, in the density clustering method, the DBSCAN can overcome the defect that the K-means clustering center has a large influence on the clustering results to a certain extent, but the clustering effect also depends on the preset clustering radius, and under the indoor environment, the discrete degrees of the millimeter wave echo points at a far and near distance are different, and the fixed clustering radius can lead to the clustering result being difficult to meet the indoor tracking requirement.
Drawings
FIG. 1 is a flow chart of the present application;
FIG. 2 is a graph of reachable distances;
FIG. 3 is a diagram of raw data;
FIG. 4 is a schematic diagram of a clustering result calculated by the method of the present application;
FIG. 5 is a schematic diagram of a clustering result of a conventional DBSCAN algorithm;
fig. 6 is a schematic diagram of a fourth step.
Detailed Description
It should be noted that, in particular, the various embodiments of the present disclosure may be combined with each other without conflict.
The first embodiment specifically describes an indoor human body target tracking method based on millimeter wave radar according to the present embodiment with reference to fig. 1, and includes the following steps:
Firstly, acquiring echo time domain original data of an indoor human body target by utilizing a millimeter wave radar, reconstructing the original data to form a matrix, wherein the matrix size is ADCNum x NumChirp (NTx NRx), and then respectively carrying out two-dimensional Fourier transform on each channel in the matrix to obtain an RD (remote data) spectrum matrix;
Wherein NumChirp denotes the number of chirp, ADCNum denotes the number of sampling points contained in each chirp, NTx denotes the number of transmitting antennas, and NRx denotes the number of receiving antennas;
Step two, static clutter filtering and constant false alarm detection are carried out on the RD spectrum matrix, and then the distance between a target point and the radar is obtained;
Measuring angle information of a target point through an angle measurement algorithm, and then combining the distance between the target point and the radar with the angle information to obtain the abscissa of the target point, so as to construct a data set;
step four, obtaining an reachable distance sequence by utilizing the horizontal coordinates and the vertical coordinates of the target points in the data set;
Step five, clustering and dividing each reachable distance in the reachable distance sequence to obtain all clustering and dividing of the target point;
Step six, calculating the contour coefficient S of each cluster partition in all cluster partitions of the target point, and selecting the partition with the largest contour coefficient as a clustering Result, namely the state of the target at the moment;
and seventhly, repeating the second step to the sixth step to track the indoor human body target.
Clustering algorithms are a common machine learning technique that can divide a data set into several categories, each of which contains a set of highly similar samples. In Lei Daqun target tracking, a clustering algorithm is mainly used for distinguishing the motion laws and behavior patterns of targets of different categories. The most common algorithms in the clustering algorithm include a K-means clustering algorithm, a hierarchical clustering algorithm, a density clustering algorithm and the like. These algorithms have respective advantages and application ranges.
The K-means clustering algorithm is a clustering algorithm based on a central point. In this algorithm, the parameter k is first entered, representing the classification of the data set into k different categories. Then, k samples are randomly selected in the dataset as the center of k categories. Then, all samples in the dataset are assigned to the k different categories, respectively, according to the distances between the samples in the dataset and the k samples. The sum of the distances between samples assigned to the same class is minimal, and the assignment results satisfy the minimal distance principle. Finally, the center point of each category is recalculated according to the distribution result. This process is repeated until the center point is no longer changed or other stop conditions are met.
Hierarchical clustering algorithms are a top-down or bottom-up clustering method. In the algorithm, for a data set, it is first treated as a single cluster, then each cluster is split into several clusters in turn, and this process is repeated until a predetermined condition is met. The distance between samples in the hierarchical clustering process can be defined differently, and single connectivity, full connectivity, average connectivity and the like are common. Wherein, single connectivity refers to the shortest distance between two clusters, full connectivity refers to the longest distance between two clusters, and average connectivity refers to calculating the average distance between all samples in two clusters.
The density clustering algorithm is a density-based clustering algorithm. In this algorithm, clusters are defined as dense areas of sample points, and distances between clusters are defined as distances of sample points between clusters. By calculating the density of sample points, the data set can be divided into clusters. Wherein the density of a sample point represents the number of samples contained within a circle centered at that point and having a radius. In the density clustering algorithm, two parameters, namely a radius and a density threshold, are adopted when clustering and discriminating clusters. The density clustering algorithm can overcome the problem of strong initial value dependence in the K-means algorithm, so that the density clustering algorithm is widely applied to the field of target tracking.
Under the indoor human body tracking condition, the specific number of targets cannot be known in advance, so that different target numbers need to be clustered respectively by applying a K-means clustering algorithm, and then the optimal target number is judged according to additional judgment parameters. In order to reduce the dependence of the K-means algorithm on initial values, the random values can be taken for clustering, so that the algorithm is complicated and redundant, the hierarchical clustering algorithm can lead to a plurality of obtained clustering results, the algorithm is long in operation time, in the density clustering method, the DBSCAN can overcome the defect that the K-means clustering center has a large influence on the clustering results to a certain extent, but the clustering effect also depends on the preset clustering radius, and under the indoor environment, the discrete degrees of the millimeter wave echo points at a far and near distance are different, and the fixed clustering radius can lead to the clustering result being difficult to meet the indoor tracking requirement. The present application improves upon the shortcomings of these clustering algorithms.
The technical scheme of the application specifically comprises three stages of core distance and reachable distance calculation, clustering division and contour coefficient calculation.
Stage one, core distance and reachable distance calculation
The data set D is assumed to need clustering, and parameters and conditions are set as follows, wherein each cluster at least comprises MinPts points, and the distance adopts Euclidean distance. The neighborhood of a point q is denoted as N ε (q), where epsilon denotes the neighborhood radius.
The core distance of a data point q is defined as the minimum neighborhood radius epsilon 0 for the neighborhood of that point, which contains the MinPts points. Data points with smaller core distances are typically more dense and relatively concentrated data points. For each data point q belonging to the data set D, only the distances of other points need to be calculated, the distances are ordered from small to large, and the distance with the MinPts small is taken out, namely the core distance corresponding to the point.
The reachable distance is the distance between one data point p and another data point q, and also the core distance of q and the core distance of p need to be considered. If p to q have a connected path and the core distance maximum of all data points on the path is not higher than the core distance of q, then the reachable distance between p and q is the core distance of q.
The calculation of the reachable distance is carried out according to the following steps:
(1) Finding out the point corresponding to the maximum core distance of all points in the data set D as an observation point, which means that the point is the most likely noise point;
(2) Setting the reachable distances of points contained in the core area of the observation point as the core distance of the observation point, and setting the reachable distances of points not in the core area as the distance between the point and the observation point, wherein the distance is denoted as m;
(3) Finding out points with the reachable distance larger than m in the points in the core area, and setting the points as m;
(4) Finding out the point with the smallest current reachable distance from the rest points, taking the point as a new observation point, repeating the steps (2) - (4) until the rest points in the step (4) are zero, ending, and taking note that in the processing process, the points which are taken as the observation points each time are required to be recorded and ordered, and constructing a reachable distance table according to the reachable distance corresponding to each point.
Stage two, cluster partitioning
In stage one, the reachable distances and their corresponding sequences are obtained, and it is first clear that the reachable distance corresponding to each point represents the close relationship between the point and the point before it, so that the smaller the reachable distance, the closer the point to the point before it, and vice versa. According to the characteristics, it can be known that a region with a mutation of a reachable distance exists between two clusters, and the region between the two mutation regions is one cluster.
In the cluster division stage, a coefficient ζ needs to be specified, which determines in what case a region can be defined as "mutation". After the value of ζ is given according to the actual situation, all mutation points are found out by traversing the reachable distance table, and every two mutation points are combined to obtain a cluster. The clusters obtained by this method are usually coincident with each other and therefore require further processing for cluster division. The specific method comprises the steps of finding out sub-clusters which cover all data sets as far as possible and are not overlapped with each other, and listing the sub-clusters as one of the clustering methods, wherein the steps are as follows:
(1) Setting a specified coefficient xi, and traversing an reachable distance table, wherein the reachable distance table is used as a mutation point when the reachable distance is changed beyond the specified coefficient xi, and each two mutation points represent a division, and if the mutation point is one, only one division is used for the data set, namely the data set per se.
The partitions obtained through mutation comprise a starting point and an ending point, the partitions are ordered from small to large according to the starting point, all the partitions are defaulted without conflict, and all the partitions except the partition corresponding to the first starting point are used as undetermined partitions. And the partition corresponding to the first starting point is taken as a reference partition,
(2) Starting with the partition corresponding to the minimum starting point in the sequence without judgment, judging whether the partition coincides with the reference partition from small to large, if so, taking the partition coinciding with the reference partition in the sequence as the conflict partition, if not, merging the partition with the reference partition as the new reference partition, and repeating the steps until the last partition in the sequence is judged. And taking the finally obtained reference partition as a clustering partition result.
Deleting the last conflict-causing partition from the finally obtained reference partitions to obtain new reference partitions, taking all the conflict-causing partitions with the last conflict-causing partition as undetermined partitions, and repeating the steps until all the partitions are judged, wherein all the obtained clustering partition results are final clustering partition results.
Stage three, contour coefficient calculation
In the second stage, a plurality of different clustering modes of a data set can be obtained, and the most suitable clustering mode is required to be selected according to the contour coefficients of different clusters.
The contour coefficient (silhouette value) of a point in a cluster is defined as
Wherein a i represents the average distance of the i-th point to other points in the same cluster;
b i represents the minimum average distance of the point to other points in different clusters.
The contour coefficients of all points in the clusters are obtained, and the contour coefficients are averaged to be used as the contour coefficients of the whole clustering mode, so that the contour coefficients of different clustering modes can be obtained, and the clustering mode with the largest contour coefficient is the clustering mode which is most suitable for the data set.
Taking one frame of radar actual data as an example, it is known that there are 3 targets in the current frame, as shown in fig. 2. The minpts=5 is set and,
Specific flow
(1) The reachable distance table of all points was found as in fig. 3.
(2) The small block clusters are obtained according to the mutation values and are respectively [13,19], [13,100], [34,98], [46,98], [57,62],
[81,100],[106,120],[106,208],[106,218],[123,131],[208,218],[232,253];
(3) The small block clusters are combined to obtain different classification results as follows:
{[13,19],[34,98],[106,120],[123,131],[208,218],[232,253]};
{[13,19],[34,98],[106,208],[208,218],[232,253]};
{[13,19],[34,98],[106,218],[232,253]};
{[13,19],[34,98],[123,131],[208,218],[232,253]};
{[13,19],[46,98],[106,120],[123,131],[208,218],[232,253]};
{[13,19],[57,62],[81,100],[106,120],[123,131],[208,218],[232,253]};
{[13,100],[106,120],[123,131],[208,218],[232,253]};
{[13,100],[106,208],[208,218],[232,253]};
{[13,100],[106,218],[232,253]};
{[13,100],[123,131],[208,218],[232,253]};
(4) Respectively solving contour coefficients for the small blocks of clusters, and taking out the clusters corresponding to the largest contour coefficients as
{[13,100],[106,218],[232,253]};
(5) The final cluster is { [13,100], [106,218], [232,253] };
Compared with the DBSCAN clustering method, the method can consider a plurality of clustering modes and obtain the cluster with the best performance.
Assuming that the radar emits FMCW continuous waves, a total of frames are emitted throughout the measurement, and for each frame its parameters are as follows, a total of NumChirp chirp, each chirp having ADCNum sampling points, a total of NTx transmit antennas and NRx receive antennas. In the clustering process, the minimum point number in one cluster is set as MinPts, and the threshold coefficient is zeta.
It should be noted that the detailed description is merely for explaining and describing the technical solution of the present invention, and the scope of protection of the claims should not be limited thereto. All changes which come within the meaning and range of equivalency of the claims and the specification are to be embraced within their scope.

Claims (10)

1.一种基于毫米波雷达的室内人体目标跟踪方法,其特征在于包括以下步骤:1. An indoor human target tracking method based on millimeter-wave radar, characterized by comprising the following steps: 步骤一:利用毫米波雷达获取室内人体目标的回波时域原始数据,并对原始数据进行重新构造,形成矩阵,矩阵大小为ADCNum*NumChirp*(NTx*NRx),然后分别对矩阵中每个通道进行二维傅里叶变换,得到RD谱矩阵;Step 1: Use millimeter-wave radar to acquire raw time-domain echo data of indoor human targets, and reconstruct the raw data to form a matrix with size ADCNum*NumChirp*(NTx*NRx). Then, perform a two-dimensional Fourier transform on each channel in the matrix to obtain the RD spectrum matrix. 其中,NumChirp表示chirp的数量,ADCNum表示每个chirp所包含的采样点的数量,NTx表示发射天线的数量,NRx表示接收天线的数量;Where NumChirp represents the number of chirps, ADCNum represents the number of sampling points contained in each chirp, NTx represents the number of transmit antennas, and NRx represents the number of receive antennas; 步骤二:对RD谱矩阵进行静态杂波滤除及恒虚警检测后,得到目标点距雷达的距离;Step 2: After static clutter filtering and constant false alarm rate detection on the RD spectrum matrix, the distance from the target point to the radar is obtained; 步骤三:通过测角算法测量目标点的角度信息,然后通过目标点距雷达的距离和角度信息,得到目标点的横纵坐标,以此构建数据集;Step 3: Measure the angle information of the target point using an angle measurement algorithm, and then obtain the horizontal and vertical coordinates of the target point using the distance and angle information from the radar, thereby constructing a dataset; 步骤四:利用数据集中目标点的横纵坐标,得到可达距离序列;Step 4: Use the x and y coordinates of the target points in the dataset to obtain the reachability distance sequence; 步骤五:将可达距离序列中的各个可达距离进行聚类划分,得到目标点的所有聚类划分;Step 5: Cluster the reachable distances in the reachable distance sequence to obtain all clusters of the target point; 步骤六:计算目标点的所有聚类划分中的各个聚类划分的轮廓系数,选出具有最大轮廓系数的划分作为聚类结果Result,即当前时刻目标的状态;Step 6: Calculate the silhouette coefficient of each cluster partition in all cluster partitions of the target point, and select the partition with the largest silhouette coefficient as the clustering result Result, which is the state of the target at the current time. 步骤七:重复步骤二至步骤六,实现对室内人体目标的跟踪。Step 7: Repeat steps 2 through 6 to track human targets indoors. 2.根据权利要求1所述的一种基于毫米波雷达的室内人体目标跟踪方法,其特征在于所述步骤四的具体步骤为:2. The indoor human target tracking method based on millimeter-wave radar according to claim 1, characterized in that the specific steps of step four are as follows: 步骤四一:根据目标点的横纵坐标得到数据集中所有目标点的核心距离,然后选取最大核心距离对应的目标点作为初始观测点,并将其余目标点均设置为未观测点,之后将所有未观测点的初始可达距离设置为初始观测点的核心距离;Step 41: Obtain the core distance of all target points in the dataset based on the x and y coordinates of the target point. Then select the target point with the maximum core distance as the initial observation point and set the remaining target points as unobserved points. After that, set the initial reachable distance of all unobserved points as the core distance of the initial observation point. 步骤四二:分别求出观测点距所有未观测点间的距离值序列A以及观测点自身的核心距离值B,将距离值序列A中每一个值按照序列的顺序分别与B进行比较,并将每次比较中的最大值按顺序保留,得到M序列,之后将未观测点当前的可达距离中的每个值分别与M序列中对应位置的值进行比较,若未观测点当前的可达距离中的值比M序列中对应的值大,则将M序列中对应的值替换未观测点的可达距离中对应的值,否则不替换;Step 42: Calculate the distance value sequence A between the observation point and all unobserved points, and the core distance value B of the observation point itself. Compare each value in the distance value sequence A with B in the order of the sequence, and keep the maximum value in each comparison in order to obtain the M sequence. Then, compare each value in the current reachable distance of the unobserved point with the value at the corresponding position in the M sequence. If the value in the current reachable distance of the unobserved point is greater than the corresponding value in the M sequence, replace the corresponding value in the reachable distance of the unobserved point with the corresponding value in the M sequence; otherwise, do not replace it. 步骤四三:找出未观测点中可达距离最小的点,并将其作为新的观测点,然后将之前的观测点标记为已观测,之后不再进行比较,重复步骤四二,直至所有的点均完成观测,最后记录每次作为观测点的目标点,并进行排序,并以排序中每个目标点对应的可达距离构建可达距离表。Step 43: Find the point with the smallest reachable distance among the unobserved points and use it as the new observation point. Then mark the previous observation points as observed and do not make any further comparisons. Repeat Step 42 until all points have been observed. Finally, record the target points used as observation points each time, sort them, and construct a reachable distance table based on the reachable distance of each target point in the sort. 3.根据权利要求2所述的一种基于毫米波雷达的室内人体目标跟踪方法,其特征在于所述步骤五的具体步骤为:3. The indoor human target tracking method based on millimeter-wave radar according to claim 2, characterized in that the specific steps of step five are as follows: 步骤五一:遍历可达距离表,其中可达距离变化超过规定系数的目标点,即为突变点,并以每两个突变点代表一个划分,若突变点为一个,则以该数据集只有一个划分,即数据集本身,所述划分包括起始点和结束点;Step 51: Traverse the reachability table, where the reachability changes by a specified coefficient. The target point is the mutation point, and every two mutation points represent a partition. If there is only one mutation point, then the dataset has only one partition, namely the dataset itself. The partition includes the start point and the end point. 步骤五二:将通过突变点得到的划分,按照划分中起始点从小到大进行排序,默认所有划分均不冲突,并将除第一个起始点对应的划分以外的所有划分作为未判断划分,然后将第一个起始点对应的划分作为参考划分;Step 52: Sort the partitions obtained through mutation points in ascending order of their starting points. By default, all partitions are non-conflicting. All partitions except the one corresponding to the first starting point are treated as unjudged partitions. Then, the partition corresponding to the first starting point is used as the reference partition. 步骤五三:以未判断划分的排序中最小起始点对应的划分开始,由小到大,分别判断该划分与参考划分是否重合,若重合,则将排序中与参考划分重合的划分作为冲突划分,若不重合,则将该划分与参考划分合并后作为新的参考划分,Step 53: Starting with the smallest starting point in the unresolved sort, proceed from smallest to largest, checking if each of these partitions overlaps with the reference partition. If they overlap, the partition that overlaps with the reference partition in the sort is considered a conflicting partition. If they do not overlap, merge the conflicting partition with the reference partition to create a new reference partition. 步骤五四:重复步骤五三,直至排序中最后一个划分判断完毕,将最后得到的参考划分作为一个聚类划分结果;Step 54: Repeat step 53 until the last partition in the sorting process is determined, and take the last reference partition as a clustering result; 步骤五五:从最后得到的参考划分中删除最后一个造成冲突的划分,得到新的参考划分,并将最后一个造成冲突的划分之后的所有划分作为未判断划分,然后执行步骤五三,直到所有划分均已判断,得到的所有聚类划分结果为最终聚类划分结果。Step 55: Delete the last conflicting partition from the final reference partition, obtain a new reference partition, and treat all partitions after the last conflicting partition as unresolved partitions. Then execute Step 53 until all partitions have been resolved. The resulting clustering partitions are the final clustering partitions. 4.根据权利要求1所述的一种基于毫米波雷达的室内人体目标跟踪方法,其特征在于所述步骤六的具体步骤为:4. The indoor human target tracking method based on millimeter-wave radar according to claim 1, characterized in that the specific steps of step six are as follows: 步骤六:计算目标点的所有聚类划分中的各个聚类划分的轮廓系数,选出具有最大轮廓系数的划分作为聚类结果Result,并根据聚类结果Result以及目标点相对雷达的径向速度确定该时刻目标的状态,即该时刻目标的状态;Step 6: Calculate the silhouette coefficient of each cluster partition in all cluster partitions of the target point, select the partition with the largest silhouette coefficient as the cluster result Result, and determine the state of the target at that moment based on the cluster result Result and the radial velocity of the target point relative to the radar. 所述目标点相对雷达的径向速度通过步骤二中对RD谱矩阵进行静态杂波滤除及恒虚警检测得到。The radial velocity of the target point relative to the radar is obtained by performing static clutter filtering and constant false alarm rate detection on the RD spectrum matrix in step two. 5.根据权利要求3或4所述的一种基于毫米波雷达的室内人体目标跟踪方法,其特征在于所述轮廓系数表示为:5. The indoor human target tracking method based on millimeter-wave radar according to claim 3 or 4, characterized in that the contour coefficient is expressed as: 其中,表示第点到同一聚类中其它点的平均距离,表示该点到不同聚类中其他点的最小平均距离。in, Indicates the first The average distance from a point to other points in the same cluster. This represents the minimum average distance from this point to other points in different clusters. 6.根据权利要求5所述的一种基于毫米波雷达的室内人体目标跟踪方法,其特征在于所述聚类划分中最小点数为5。6. The indoor human target tracking method based on millimeter-wave radar according to claim 5, wherein the minimum number of points in the clustering partition is 5. 7.一种基于毫米波雷达的室内人体目标跟踪系统,其特征在于包括:室内人体目标回波时域原始数据获取模块、数据集构建模块以及室内人体目标跟踪模块;7. An indoor human target tracking system based on millimeter-wave radar, characterized in that it includes: an indoor human target echo time-domain raw data acquisition module, a dataset construction module, and an indoor human target tracking module; 所述室内人体目标回波时域原始数据获取模块用于利用毫米波雷达获取室内人体目标的回波时域原始数据,并对原始数据进行重新构造,形成矩阵,矩阵大小为ADCNum*NumChirp*(NTx*NRx),然后分别对矩阵中每个通道进行二维傅里叶变换,得到RD谱矩阵;其中,NumChirp表示chirp的数量,ADCNum表示每个chirp所包含的采样点的数量,NTx表示发射天线的数量,NRx表示接收天线的数量;The indoor human target echo time-domain raw data acquisition module is used to acquire the indoor human target echo time-domain raw data using millimeter-wave radar, and reconstruct the raw data into a matrix with a size of ADCNum*NumChirp*(NTx*NRx). Then, a two-dimensional Fourier transform is performed on each channel in the matrix to obtain the RD spectrum matrix. Here, NumChirp represents the number of chirps, ADCNum represents the number of sampling points contained in each chirp, NTx represents the number of transmitting antennas, and NRx represents the number of receiving antennas. 所述数据集构建模块用于对RD谱矩阵进行静态杂波滤除及恒虚警检测后,得到目标点距雷达的距离,并通过测角算法测量目标点的角度信息,然后通过目标点距雷达的距离和角度信息,得到目标点的横纵坐标,以此构建数据集;The dataset construction module is used to perform static clutter filtering and constant false alarm detection on the RD spectrum matrix, obtain the distance between the target point and the radar, measure the angle information of the target point through an angle measurement algorithm, and then obtain the horizontal and vertical coordinates of the target point through the distance and angle information between the target point and the radar, thereby constructing the dataset. 所述室内人体目标跟踪模块用于利用数据集中目标点的横纵坐标,得到可达距离序列,之后将可达距离序列中的各个可达距离进行聚类划分,得到目标点的所有聚类划分,然后计算目标点的所有聚类划分中的各个聚类划分的轮廓系数,选出具有最大轮廓系数的划分作为聚类结果Result,即当前时刻目标的状态;The indoor human target tracking module is used to obtain a reachable distance sequence by using the horizontal and vertical coordinates of the target points in the dataset. Then, it clusters each reachable distance in the reachable distance sequence to obtain all clusters of the target points. Then, it calculates the contour coefficient of each cluster in all clusters of the target points and selects the cluster with the largest contour coefficient as the clustering result Result, which is the state of the target at the current time. 通过室内人体目标回波时域原始数据获取模块、数据集构建模块以及室内人体目标跟踪模块进而实现室内人体目标的跟踪。The tracking of indoor human targets is achieved through an indoor human target echo time-domain raw data acquisition module, a dataset construction module, and an indoor human target tracking module. 8.根据权利要求7所述的一种基于毫米波雷达的室内人体目标跟踪系统,其特征在于所述室内人体目标跟踪模块中利用数据集中目标点的横纵坐标,得到可达距离序列的具体步骤为:8. The indoor human target tracking system based on millimeter-wave radar according to claim 7, characterized in that the specific steps in the indoor human target tracking module for obtaining the reachable distance sequence using the horizontal and vertical coordinates of the target points in the data set are as follows: 步骤1:根据目标点的横纵坐标得到数据集中所有目标点的核心距离,然后选取最大核心距离对应的目标点作为初始观测点,并将其余目标点均设置为未观测点,之后将所有未观测点的初始可达距离设置为初始观测点的核心距离;Step 1: Obtain the core distance of all target points in the dataset based on the x and y coordinates of the target point. Then, select the target point with the maximum core distance as the initial observation point and set the remaining target points as unobserved points. After that, set the initial reachable distance of all unobserved points as the core distance of the initial observation point. 步骤2:分别求出观测点距所有未观测点间的距离值序列A以及观测点自身的核心距离值B,将距离值序列A中每一个值按照序列的顺序分别与B进行比较,并将每次比较中的最大值按顺序保留,得到M序列,之后将未观测点当前的可达距离中的每个值分别与M序列中对应位置的值进行比较,若未观测点当前的可达距离中的值比M序列中对应的值大,则将M序列中对应的值替换未观测点的可达距离中对应的值,否则不替换;Step 2: Calculate the distance value sequence A between the observation point and all unobserved points, and the core distance value B of the observation point itself. Compare each value in the distance value sequence A with B in the order of the sequence, and keep the maximum value in each comparison in order to obtain the M sequence. Then, compare each value in the current reachable distance of the unobserved point with the value at the corresponding position in the M sequence. If the value in the current reachable distance of the unobserved point is greater than the corresponding value in the M sequence, replace the corresponding value in the reachable distance of the unobserved point with the corresponding value in the M sequence; otherwise, do not replace it. 步骤3:找出未观测点中可达距离最小的点,并将其作为新的观测点,然后将之前的观测点标记为已观测,之后不再进行比较,重复步骤2,直至所有的点均完成观测,最后记录每次作为观测点的目标点,并进行排序,并以排序中每个目标点对应的可达距离构建可达距离表。Step 3: Find the point with the smallest reachable distance among the unobserved points and use it as the new observation point. Then mark the previous observation points as observed and do not make any further comparisons. Repeat Step 2 until all points have been observed. Finally, record the target points used as observation points each time, sort them, and construct a reachable distance table based on the reachable distance of each target point in the sort. 9.根据权利要求8所述的一种基于毫米波雷达的室内人体目标跟踪系统,其特征在于所述室内人体目标跟踪模块中将可达距离序列中的各个可达距离进行聚类划分,得到目标点的所有聚类划分的具体步骤为:9. The indoor human target tracking system based on millimeter-wave radar according to claim 8, characterized in that the specific steps in the indoor human target tracking module to cluster each reachable distance in the reachable distance sequence to obtain all clustering divisions of the target point are as follows: 步骤A:遍历可达距离表,其中可达距离变化超过规定系数的目标点,即为突变点,并以每两个突变点代表一个划分,若突变点为一个,则以该数据集只有一个划分,即数据集本身,所述划分包括起始点和结束点;Step A: Traverse the reachability table, where the reachability distance changes by a specified coefficient. The target point is the mutation point, and every two mutation points represent a partition. If there is only one mutation point, then the dataset has only one partition, namely the dataset itself. The partition includes the start point and the end point. 步骤B:将通过突变点得到的划分,按照划分中起始点从小到大进行排序,默认所有划分均不冲突,并将除第一个起始点对应的划分以外的所有划分作为未判断划分,然后将第一个起始点对应的划分作为参考划分;Step B: Sort the partitions obtained through mutation points in ascending order of their starting points. By default, all partitions are non-conflicting. All partitions except the one corresponding to the first starting point are considered unjudged partitions. The partition corresponding to the first starting point is then used as the reference partition. 步骤C:以未判断划分的排序中最小起始点对应的划分开始,由小到大,分别判断该划分与参考划分是否重合,若重合,则将排序中与参考划分重合的划分作为冲突划分,若不重合,则将该划分与参考划分合并后作为新的参考划分,Step C: Starting with the smallest starting point in the unresolved sort, proceed from smallest to largest, checking if each of these partitions overlaps with the reference partition. If they overlap, the partition that overlaps with the reference partition in the sort is considered a conflicting partition. If they do not overlap, merge the conflicting partition with the reference partition to create a new reference partition. 步骤D:重复步骤C,直至排序中最后一个划分判断完毕,将最后得到的参考划分作为一个聚类划分结果;Step D: Repeat step C until the last partition in the sorting is determined, and take the last reference partition as a clustering result; 步骤E:从最后得到的参考划分中删除最后一个造成冲突的划分,得到新的参考划分,并将最后一个造成冲突的划分之后的所有划分作为未判断划分,然后执行步骤C,直到所有划分均已判断,得到的所有聚类划分结果为最终聚类划分结果。Step E: Delete the last conflicting partition from the final reference partition, obtain a new reference partition, and treat all partitions after the last conflicting partition as unresolved partitions. Then execute Step C until all partitions have been resolved. The resulting clustering partitions are the final clustering partitions. 10.根据权利要求9所述的一种基于毫米波雷达的室内人体目标跟踪系统,其特征在于所述轮廓系数表示为:10. The indoor human target tracking system based on millimeter-wave radar according to claim 9, wherein the contour coefficient is expressed as: 其中,表示第点到同一聚类中其它点的平均距离,表示该点到不同聚类中其他点的最小平均距离。in, Indicates the first The average distance from a point to other points in the same cluster. This represents the minimum average distance from this point to other points in different clusters.
CN202310447818.7A 2023-04-24 2023-04-24 Indoor human body target tracking method and system based on millimeter wave radar Active CN116736288B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310447818.7A CN116736288B (en) 2023-04-24 2023-04-24 Indoor human body target tracking method and system based on millimeter wave radar

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310447818.7A CN116736288B (en) 2023-04-24 2023-04-24 Indoor human body target tracking method and system based on millimeter wave radar

Publications (2)

Publication Number Publication Date
CN116736288A CN116736288A (en) 2023-09-12
CN116736288B true CN116736288B (en) 2026-01-16

Family

ID=87901895

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310447818.7A Active CN116736288B (en) 2023-04-24 2023-04-24 Indoor human body target tracking method and system based on millimeter wave radar

Country Status (1)

Country Link
CN (1) CN116736288B (en)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110361725A (en) * 2018-04-11 2019-10-22 英飞凌科技股份有限公司 The system and method for carrying out mankind's detection and identification using millimetre-wave radar
CN110361727A (en) * 2019-07-22 2019-10-22 浙江大学 A Multi-target Tracking Method for Millimeter-Wave Radar

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP7623106B2 (en) * 2019-05-12 2025-01-28 オリジン リサーチ ワイヤレス, インコーポレイテッド Method, Apparatus and System for Wireless Tracking, Scanning and Surveillance - Patent application
GB2590115B (en) * 2019-09-13 2023-12-06 Motional Ad Llc Extended object tracking using radar
CN111239728B (en) * 2020-02-26 2021-06-15 深圳雷研技术有限公司 Passenger counting method and system based on millimeter wave radar
CN112114309B (en) * 2020-08-10 2023-04-07 西安电子科技大学 JPDA multi-target tracking method based on optimal contour coefficient self-adaptive K-means clustering
CN112444805A (en) * 2020-11-01 2021-03-05 复旦大学 Distributed multi-target detection, positioning tracking and identity recognition system based on radar
CN112505648B (en) * 2020-11-19 2023-06-30 西安电子科技大学 Target Feature Extraction Method Based on Millimeter Wave Radar Echo
CN114994656A (en) * 2021-11-16 2022-09-02 上海昉恩智能科技有限公司 Indoor personnel tracking method based on millimeter wave radar

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110361725A (en) * 2018-04-11 2019-10-22 英飞凌科技股份有限公司 The system and method for carrying out mankind's detection and identification using millimetre-wave radar
CN110361727A (en) * 2019-07-22 2019-10-22 浙江大学 A Multi-target Tracking Method for Millimeter-Wave Radar

Also Published As

Publication number Publication date
CN116736288A (en) 2023-09-12

Similar Documents

Publication Publication Date Title
CN112036074B (en) Radar signal sorting method and system in high pulse density environment
CN107728140B (en) A Multi-Target Multi-Channel Parallel Tracking Processing Method for Warning Radar
CN109581312A (en) A multi-target clustering method for high-resolution millimeter-wave radar
CN106851571A (en) WiFi localization methods in a kind of quick KNN rooms based on decision tree
CN111929653B (en) Target detection and tracking method and system based on unmanned ship navigation radar
CN115062683B (en) 4D millimeter wave radar clustering method and storage medium based on DBSCAN
CN110049549A (en) More fusion indoor orientation methods and its system based on WiFi fingerprint
CN118091572A (en) A method for clustering dense target point cloud based on millimeter wave radar
CN113608193A (en) Radar multi-target distance and speed estimation method based on UNet
CN119087357A (en) A parameter adaptive multi-density clustering radar signal sorting method
CN116736288B (en) Indoor human body target tracking method and system based on millimeter wave radar
CN113177097B (en) Track start judging method based on attribute clustering and space-time constraint
CN113269242B (en) Target detection trace aggregation method based on peak clustering
CN119414341B (en) Doppler velocity filtering multipath false target suppression method, device and medium
CN119939286A (en) Adaptive radar signal sorting and clustering optimization method based on improved density peak clustering
CN118692264B (en) A method for extracting and eliminating incomplete routes based on ADS-B data
CN120314928A (en) Target tracking method and system based on DBSCAN and Kmeans
CN117194951B (en) HRRP target identification method based on joint feature extraction and Stacking model classification
CN108445443A (en) A kind of fingerprint point clustering method based on KNN
Dai et al. Clustering of DOA data in radar pulse based on SOFM and CDbw
CN115147385B (en) Intelligent detection and judgment method for repeated damage in aviation hole exploration video
CN113381827B (en) A Spectrum Sensing Method for Electromagnetic Signals Based on Deep Clustering Network
CN119513627B (en) A radar pulse sorting method based on inverse distance weighted online clustering
CN120779388B (en) A method and system for multi-target tracking of radar signals based on dynamic graph convolutional networks
CN115032692B (en) River phase reservoir sediment microphase partition method and device based on seismic waveform clustering

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant