[go: up one dir, main page]

CN115607108B - Multi-target reconstruction method based on surface measurement signal blind source separation - Google Patents

Multi-target reconstruction method based on surface measurement signal blind source separation Download PDF

Info

Publication number
CN115607108B
CN115607108B CN202211207887.2A CN202211207887A CN115607108B CN 115607108 B CN115607108 B CN 115607108B CN 202211207887 A CN202211207887 A CN 202211207887A CN 115607108 B CN115607108 B CN 115607108B
Authority
CN
China
Prior art keywords
light
separation
matrix
target
reconstruction
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202211207887.2A
Other languages
Chinese (zh)
Other versions
CN115607108A (en
Inventor
郭红波
张力之
贺小伟
侯榆青
易黄建
何雪磊
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Northwest University
Original Assignee
Northwest University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Northwest University filed Critical Northwest University
Priority to CN202211207887.2A priority Critical patent/CN115607108B/en
Publication of CN115607108A publication Critical patent/CN115607108A/en
Application granted granted Critical
Publication of CN115607108B publication Critical patent/CN115607108B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/0059Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence
    • A61B5/0071Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence by measuring fluorescence emission
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/0059Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence
    • A61B5/0073Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence by tomography, i.e. reconstruction of 3D images from 2D projections

Landscapes

  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Medical Informatics (AREA)
  • Biophysics (AREA)
  • Pathology (AREA)
  • Engineering & Computer Science (AREA)
  • Biomedical Technology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Physics & Mathematics (AREA)
  • Molecular Biology (AREA)
  • Surgery (AREA)
  • Animal Behavior & Ethology (AREA)
  • General Health & Medical Sciences (AREA)
  • Public Health (AREA)
  • Veterinary Medicine (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Investigating, Analyzing Materials By Fluorescence Or Luminescence (AREA)

Abstract

一种基于表面测量信号盲源分离的多目标重建方法,步骤一:基于光传输模型和有限元理论,将重建荧光目标的结构信息和光学特征参数作为先验信息,对多目标测量信号产生的物理过程进行系统分析,说明其满足可进行盲源分离的条件。步骤二:通过多激发点激发,构建符合盲源分离条件的表面叠加测量信号。步骤三:采用盲源分离的方法构建分离矩阵,将叠加的测量信号分离为各目标在生物组织表面的源测量信号,建立各个源测量信号与体内光源分布之间的线性关系。步骤四:利用重建算法对每个目标分别进行重建,并将各个目标的重建结果放在一个坐标系中合并展示。该方法在提供准确的定位和形态恢复能力的同时保证了重建问题对参数和算法的鲁棒性,这是在有关癌症分期的预临床研究中利用光学分子技术进行多源重建的突破性工作。

A multi-target reconstruction method based on blind source separation of surface measurement signals. Step 1: Based on the light transmission model and finite element theory, the structural information and optical characteristic parameters of the reconstructed fluorescent target are used as prior information, and the physical process of the multi-target measurement signal generation is systematically analyzed to show that it meets the conditions for blind source separation. Step 2: Through multi-excitation point excitation, a surface superposition measurement signal that meets the blind source separation conditions is constructed. Step 3: The separation matrix is constructed using the blind source separation method to separate the superimposed measurement signal into the source measurement signal of each target on the surface of the biological tissue, and a linear relationship between each source measurement signal and the distribution of the light source in the body is established. Step 4: Each target is reconstructed separately using the reconstruction algorithm, and the reconstruction results of each target are combined and displayed in a coordinate system. This method provides accurate positioning and morphological restoration capabilities while ensuring the robustness of the reconstruction problem to parameters and algorithms. This is a breakthrough work in multi-source reconstruction using optical molecular technology in pre-clinical research on cancer staging.

Description

一种基于表面测量信号盲源分离的多目标重建方法A multi-target reconstruction method based on blind source separation of surface measurement signals

技术领域Technical Field

本发明属于荧光分子断层成像领域,具体涉及一种基于表面测量信号盲源分离的多目标重建方法。The invention belongs to the field of fluorescent molecular tomography, and in particular relates to a multi-target reconstruction method based on blind source separation of surface measurement signals.

背景技术Background Art

荧光分子断层成像(以下简称FMT)是近年来发展起来的一种新型的光学分子影像技术,其由于具有灵敏度高、时间分辨率高、无电离、无放射性、成本低以及成像快捷方便等诸多优点,成为分子影像技术的重要分支并广泛应用于预临床研究。该模态使用外部光源激发荧光探针使其发射光子,然后利用荧光采集装置(高灵敏度的CCD相机)收集生物体表的荧光信号,结合数学模型,重建出目标内部荧光源的三维分布和荧光产额,以此实现对活体状态下的生物过程进行细胞和分子水平的定性和定量的研究。考虑到目标本身对光的吸收和散射作用,准确而快速的重建出荧光标记物在目标内部的三维分布一直是FMT中的关键问题。近十年来人们在重建算法和成像系统方面做了许多努力,提出l2-范数正则化用于求解逆问题,并结合一些迭代技术进行图像重建;受压缩感知理论的启发,具有l1-范数惩罚函数的稀疏正则化方法由于其在FMT中的优良性能也受到了广泛的关注;研究发现非凸lp(0<p<1)-正则化方法在稀疏重构中比l1-范数方法能产生更稀疏、更好的解;此外,生物组织的光学特性和可行域等先验信息也被纳入重建中。Fluorescence molecular tomography (hereinafter referred to as FMT) is a new type of optical molecular imaging technology developed in recent years. It has become an important branch of molecular imaging technology and is widely used in pre-clinical research due to its many advantages such as high sensitivity, high temporal resolution, non-ionization, non-radioactivity, low cost, and fast and convenient imaging. This modality uses an external light source to excite the fluorescent probe to emit photons, and then uses a fluorescence acquisition device (a high-sensitivity CCD camera) to collect the fluorescence signal on the surface of the biological body. Combined with mathematical models, the three-dimensional distribution and fluorescence yield of the fluorescence source inside the target are reconstructed, thereby achieving qualitative and quantitative research on biological processes at the cellular and molecular levels in vivo. Considering the absorption and scattering of light by the target itself, accurately and quickly reconstructing the three-dimensional distribution of fluorescent markers inside the target has always been a key issue in FMT. In the past decade, people have made a lot of efforts in reconstruction algorithms and imaging systems. The l 2 -norm regularization was proposed to solve the inverse problem and combined with some iterative techniques for image reconstruction. Inspired by the theory of compressed sensing, the sparse regularization method with l 1 -norm penalty function has also attracted widespread attention due to its excellent performance in FMT. Studies have found that the non-convex l p (0<p<1)-regularization method can produce sparser and better solutions than the l 1 -norm method in sparse reconstruction. In addition, prior information such as the optical properties and feasible domain of biological tissues is also incorporated into the reconstruction.

在实际肿瘤研究中,由于肿瘤细胞的扩散与转移,需要对多个肿瘤区域进行综合分析来进行分期研究。因此,需要得到高精度的FMT多光源重建结果进行量化分析。然而,目前有许多应对方法在重建单个荧光目标时表现良好,但FMT逆问题的严重不适定性导致在多目标重建时,各目标的重建浓度存在较大差异,使得重建结果的处理以及可行域的选取都非常困难。对于FMT的多目标重建,多点激发和多角度投影能够通过增加测量数据在一定程度上缓解问题的不适定性,但由此会带来了新的问题,如增加数据采集时间和测量数据中的冗余数据等,不利于荧光分子断层成像的精确和高效重建,这是FMT实际应用中的一大障碍。In actual tumor research, due to the spread and metastasis of tumor cells, it is necessary to conduct a comprehensive analysis of multiple tumor areas for staging research. Therefore, it is necessary to obtain high-precision FMT multi-light source reconstruction results for quantitative analysis. However, there are many coping methods that perform well in reconstructing a single fluorescent target, but the serious ill-posedness of the FMT inverse problem leads to large differences in the reconstructed concentrations of each target during multi-target reconstruction, making the processing of reconstruction results and the selection of feasible domains very difficult. For FMT multi-target reconstruction, multi-point excitation and multi-angle projection can alleviate the ill-posedness of the problem to a certain extent by increasing the measurement data, but this will bring new problems, such as increasing data acquisition time and redundant data in the measurement data, which is not conducive to the accurate and efficient reconstruction of fluorescence molecular tomography, which is a major obstacle in the practical application of FMT.

发明内容Summary of the invention

针对临床中的多肿瘤三维检测问题,本发明提出了一种基于表面测量信号盲源分离的多目标重建方法,解决了多个光源重建困难的问题。Aiming at the problem of three-dimensional detection of multiple tumors in clinical practice, the present invention proposes a multi-target reconstruction method based on blind source separation of surface measurement signals, which solves the problem of difficulty in reconstruction of multiple light sources.

为了实现上述目的,本发明采用的技术方案如下:In order to achieve the above object, the technical solution adopted by the present invention is as follows:

一种基于表面测量信号盲源分离的多目标重建方法,包括以下步骤:A multi-target reconstruction method based on blind source separation of surface measurement signals comprises the following steps:

步骤一:基于光传输模型和有限元理论,将重建荧光目标的结构信息和光学特征参数作为先验信息,对多目标测量信号产生的物理过程进行系统分析,说明其满足可进行盲源分离的条件。Step 1: Based on the light transmission model and finite element theory, the structural information and optical characteristic parameters of the reconstructed fluorescent target are used as prior information, and the physical process of the multi-target measurement signal generation is systematically analyzed to show that it meets the conditions for blind source separation.

步骤二:通过多激发点激发,构建符合盲源分离条件的表面叠加测量信号。Step 2: Construct a surface superposition measurement signal that meets the blind source separation conditions through multi-excitation point excitation.

步骤三:采用盲源分离的方法构建分离矩阵,将叠加的测量信号分离为各目标在生物组织表面的源测量信号,建立各个源测量信号与体内光源分布之间的线性关系。Step 3: Use the blind source separation method to construct a separation matrix, separate the superimposed measurement signals into source measurement signals of each target on the surface of the biological tissue, and establish a linear relationship between each source measurement signal and the distribution of the light source in the body.

步骤四:利用重建算法对每个目标分别进行重建,并将各个目标的重建结果放在一个坐标系中合并展示。Step 4: Use the reconstruction algorithm to reconstruct each target separately, and combine the reconstruction results of each target in a coordinate system for display.

进一步,所述步骤一具体包括:Further, the step 1 specifically includes:

在荧光分子断层成像中,光的传输包括两个相关联的过程,即激发过程和发射过程。激发过程是指用外部的激发光照射成像物体的某些特定点或特定区域,这部分光通过成像物体表面进入到内部,并在物体内部形成光强分布,在本发明中我们考虑稳态的成像模式,即连续波成像方式。发射过程是物体内部的荧光团吸收了一定的激发光光能,一部分转化成光子释放出去,释放出去的这部分光叫做发射光,其波长比激发光波长要大,其光子能量小于激发光光子。激发光和发射光的传输过程可用下面两个耦合的扩散方程来描述:In fluorescence molecular tomography, the transmission of light includes two related processes, namely the excitation process and the emission process. The excitation process refers to irradiating certain specific points or specific areas of the imaging object with external excitation light. This part of light enters the interior through the surface of the imaging object and forms a light intensity distribution inside the object. In the present invention, we consider the steady-state imaging mode, that is, the continuous wave imaging mode. The emission process is that the fluorophore inside the object absorbs a certain amount of excitation light energy, and a part of it is converted into photons and released. The released part of light is called emission light, and its wavelength is larger than the wavelength of the excitation light, and its photon energy is less than the excitation light photon. The transmission process of excitation light and emission light can be described by the following two coupled diffusion equations:

其中Ω表示成像物体所占据的三维空间,下标x,m分别表示激发光和发射光,η是量子产能,是苂光团对激发光的吸收系数,它与苂光团浓度成正比,就是所要求的苂光产额分布。是狄拉克函数,Θ表示光源的强度。Where Ω represents the three-dimensional space occupied by the imaging object, the subscripts x and m represent the excitation light and the emission light respectively, and η is the quantum yield. is the absorption coefficient of the fluorophore to the excitation light, which is proportional to the concentration of the fluorophore. This is the required light yield distribution. is the Dirac function, Θ represents the intensity of the light source.

采用基于Galerkin变分法求解,并结合Robin边界条件:The solution is based on the Galerkin variational method and combined with the Robin boundary condition:

其中,是Ω的边界,v(r)是r的法向量,q是一个常数。设成像区域被离散成Np个点和Ne个四面体单元,并采用离散化网格上的基函数作为测试函数,因为任意函数都可表示为基函数的线性组合,所以基函数可以作为测试函数。于是可以表示为in, is the boundary of Ω, v(r) is the normal vector of r, and q is a constant. Assume that the imaging area is discretized into Np points and Ne tetrahedral units, and the basis function on the discretized grid is used as the test function. Since any function can be expressed as a linear combination of basis functions, the basis function can be used as the test function. So and It can be expressed as

其中,φxk,mk和(ημaf)k表示节点k上的函数值,表示节点k处的基函数。Among them, φ xk,mk and (ημ af ) k represent the function value at node k, represents the basis function at node k.

对每一个单元,将(4)式和(5)式代入扩散方程,形成每个单元的刚度矩阵,将所有单元刚度矩阵进行叠加和组装,获得如下方程:For each unit, substitute equations (4) and (5) into the diffusion equation to form the stiffness matrix of each unit. Superimpose and assemble all unit stiffness matrices to obtain the following equation:

KxΦx=Tx (6)K x Φ x =T x (6)

KmΦm=Fx (7)K m Φ m =Fx (7)

其中,激发光在成像物体内的分布向量Φx可以通过直接求解方程(6)得到,再将其代入(7)中,则得到发射光在成像物体内部和表面的分布向量Φmin, make The distribution vector Φ x of the excitation light in the imaging object can be obtained by directly solving equation (6), and then substituting it into (7), the distribution vector Φ m of the emission light inside and on the surface of the imaging object is obtained:

Φm=Km -1Fx=Ax=AΦxS (8)Φ m =K m -1 Fx=Ax=AΦ x S (8)

由于任意信号都可以用冲激信号的组合表示,我们将上式表示为:Since any signal can be represented by a combination of impulse signals, we express the above formula as:

其中AΦx为M×N的向量,表示系统矩阵;S为N×1向量,表示内部荧光探针的分布,Φ为M×1的向量,表示生物体表面光子分布的测量值。Where AΦ x is an M×N vector, representing the system matrix; S is an N×1 vector, representing the distribution of internal fluorescent probes; Φ is an M×1 vector, representing the measured value of the photon distribution on the surface of the organism.

对于FMT多目标重建,当一个激发点激发多个光源时,发出的光是多个光源发射光的叠加,成像物体内每个光源的δ(r-ri)不同,故不同光源荧光产额分布的权重不同,ωi=Φxiδ(r-ri),每个光源的发射光在成像物体内部和表面的分布向量可表示为:For FMT multi-target reconstruction, when one excitation point excites multiple light sources, the emitted light is the superposition of the light emitted by multiple light sources. The δ(r-ri) of each light source in the imaging object is different, so the weights of the fluorescence yield distribution of different light sources are different, ω ixi δ(r-ri). The distribution vector of the emitted light of each light source inside and on the surface of the imaging object can be expressed as:

Φmi=ωiASi (10)Φ mi =ω i AS i (10)

假设光源数目和激发光数目为p,探测器探测到的p个混叠信号的表面光分布为:Assuming that the number of light sources and the number of excitation lights are p, the surface light distribution of p aliased signals detected by the detector is:

其中ω=[ωij]p×p∈Rp×p为混合矩阵,如果可以分离表面光分布信息,即得到解混后的就可以实现每个光源的单独重建,这一步是多目标重建策略的核心。Where ω=[ω ij ] p×p ∈R p×p is the mixing matrix. If the surface light distribution information can be separated, the unmixed Then, the individual reconstruction of each light source can be achieved. This step is the core of the multi-objective reconstruction strategy.

盲源分离算法(以下简称BSS)中唯一可用的信息来自于混合观测矩阵,BSS模型可表示为下式:The only available information in the blind source separation algorithm (hereinafter referred to as BSS) comes from the mixed observation matrix. The BSS model can be expressed as follows:

Φ[n]=Ws[n] (n=1,2...,M) (12)Φ[n]=Ws[n] (n=1,2...,M) (12)

其中,Φ[n]=(Φ1[n],...,Φp[n])T是一个p×M的混叠观测矩阵,W=[ωij]p×k∈Rp×k是一个未知的混合矩阵,s[n]=(s1[n],...,sk[n])T指分离后的k×M的源矩阵。通常进一步假设Φ和s的维数相等,即p=k,我们在本发明的其余部分做出这个假设。Φ[n]=(Φ1[n],...,Φp[n])T中每一行均包含p个源信息s1,s2…sp,这与FMT多目标信号混叠情况一致。故多目标测量信号产生的物理过程满足可进行盲源分离的条件。Wherein, Φ[n]=(Φ 1 [n],...,Φ p [n]) T is a p×M aliasing measurement matrix, W=[ω ij ] p×k ∈R p×k is an unknown mixing matrix, and s[n]=(s 1 [n],...,s k [n]) T refers to the separated k×M source matrix. It is usually further assumed that the dimensions of Φ and s are equal, that is, p=k, and we make this assumption in the rest of the present invention. Each row in Φ[n]=(Φ 1 [n],...,Φ p [n]) T contains p source information s 1 ,s 2 …s p , which is consistent with the aliasing of FMT multi-target signals. Therefore, the physical process of generating multi-target measurement signals satisfies the conditions for blind source separation.

进一步,所述步骤二具体包括:Further, the step 2 specifically includes:

在FMT中,可以将多个光源发射光的叠加作为BSS的混合观测值,即将表面光分布信息Φm作为混合观测值Φ。其中Φ为p×M的矩阵,我们将其作为盲源分离算法的输入数据矩阵,使其满足盲源分离的条件:In FMT, the superposition of light emitted by multiple light sources can be used as the mixed observation value of BSS, that is, the surface light distribution information Φm is used as the mixed observation value Φ. Where Φ is a p×M matrix, we use it as the input data matrix of the blind source separation algorithm to satisfy the conditions of blind source separation:

进一步,所述步骤三具体包括:Further, the step three specifically includes:

对于混合的观测值,我们的目标在于通过分离算法得到每个光源的表面光分布信息,进而分别重建每个光源,p=k的假设在此处表示分离光源的数量应与激发点的数量相一致。盲源分离后p个独立信号的表面光分布为:For mixed observations, our goal is to obtain the surface light distribution information of each light source through the separation algorithm, and then reconstruct each light source separately. The assumption of p = k here means that the number of separated light sources should be consistent with the number of excitation points. The surface light distribution of p independent signals after blind source separation is:

具体分离过程中,我们通过设计分离矩阵B={bij}p×p∈Rp×p使其满足In the specific separation process, we design the separation matrix B = {b ij } p×p ∈R p×p to satisfy

z[n]=BΦ[n]=BWs[n]=Ps[n] (15)z[n]=BΦ[n]=BWs[n]=Ps[n] (15)

其中,z[n]=(z1[n],...,zp[n])T表示分离或提取出的源信号,P是一个置换矩阵,意味着提取的源点z[n]可以等价于实际的源信号s[n]。Among them, z[n]=(z 1 [n], ..., z p [n]) T represents the separated or extracted source signal, and P is a permutation matrix, which means that the extracted source point z[n] can be equivalent to the actual source signal s[n].

本发明采用的算法通过最小化估计的非负源之间的联合相关函数来对FMT数据进行分解,分离矩阵B可以通过最大化单纯形cov{0,z1,...zp}(分离后)的体积来获得,该单纯形体积与cov{0,Φ1,...,Φp}(分离前)的体积相关:The algorithm used in the present invention decomposes the FMT data by minimizing the joint correlation function between the estimated non-negative sources. The separation matrix B can be obtained by maximizing the volume of the simplex cov{0,z 1 ,...z p } (after separation), which is related to the volume of cov{0,Φ 1 ,...,Φ p } (before separation):

vol(cov{0,z1,...zp})=|det(B)|vol(cov{0,Φ1,...,Φp}) (16)vol(cov{0,z 1 ,...z p })=|det(B)|vol(cov{0,Φ 1 ,...,Φ p }) (16)

由相关文献可知,通过最大化由解混信源向量形成的固体区域的体积可以得到最优分离矩阵,该矩阵列满秩且矩阵各行元素和为1,故解决体积最大化问题就可以得到最优解:It can be seen from relevant literature that the optimal separation matrix can be obtained by maximizing the volume of the solid region formed by the demixing source vector. The matrix is full rank and the sum of the elements in each row of the matrix is 1. Therefore, solving the volume maximization problem can obtain the optimal solution:

{zi,i=1,...,p}={si,i=1,...,p} (17){z i ,i=1,...,p}={s i ,i=1,...,p} (17)

因此,盲源分离问题可以转化为下面的非凸优化问题:Therefore, the blind source separation problem can be transformed into the following non-convex optimization problem:

(1)p=k=2(1) p = k = 2

此情况下可以使用解析法求闭式解,将公式(18)中的等式约束整合到目标函数中,得到:In this case, we can use the analytical method to find the closed-form solution and integrate the equality constraint in formula (18) into the objective function to obtain:

假设det(B)≥0,公式(18)转化为Assuming det(B)≥0, formula (18) is transformed into

max(B11-B21) (20)max(B 11 -B 21 ) (20)

s.t.B11Φ1[n]+(1-B11Φ2[n])≥0stB 11 Φ 1 [n]+(1-B 11 Φ 2 [n])≥0

B21Φ1[n]+(1-B21Φ2[n])≥0B 21 Φ 1 [n]+(1-B 21 Φ 2 [n])≥0

上述约束条件可等价为:β≤B11,B21≤αThe above constraints can be equivalent to: β≤B 11 ,B 21 ≤α

其中:in:

因此,公式(20)最终转化为以下形式:Therefore, formula (20) is finally transformed into the following form:

最优解为B11 *=α且B21 *=β,因此,最优值为det(B*)=α-β>0。The optimal solution is B 11 * = α and B 21 * = β, so the optimal value is det(B * ) = α - β > 0.

同理,可以证明det(B)≤0的情况,最优值为det(B*)=β-α<0。Similarly, it can be proved that when det(B)≤0, the optimal value is det(B * )=β-α<0.

(2)p=k>2(2) p = k > 2

对B的任意一行进行代数余子式展开,以第i行为例,记Bi T=[Bi1,Bi2,...,BiN],则行列式为:Expand any row of B by algebraic co-factors. Take the i-th row as an example, let B i T = [B i1 ,B i2 ,...,B iN ], then the determinant is:

其中,Bij是删去B的第i行和第j列的子矩阵。显然,当Bij关于j=1,2,...,N固定时,行列式det(B)变为关于Bi的线性函数。因此,通过固定B的其他行向量,公式(18)就转化为如下非凸最大化问题:Where Bij is the submatrix with the i-th row and j-th column of B deleted. Obviously, when Bij is fixed with respect to j=1,2,...,N, the determinant det(B) becomes a linear function with respect to Bi . Therefore, by fixing the other row vectors of B, formula (18) is transformed into the following non-convex maximization problem:

公式(23)中目标函数仍然时非凸的,通过求解以下两个线性规划问题,可以得到该局部最大化问题的全局最优解The objective function in formula (23) is still non-convex. By solving the following two linear programming problems, the global optimal solution of the local maximization problem can be obtained:

在算法实施过程中,我们通过每次迭代更新矩阵B直到收敛,从而选取最优解。During the algorithm implementation, we update the matrix B in each iteration until convergence, thereby selecting the optimal solution.

经分离后,线性模型中的Φ被重新定义为z,故分离后表面测量值与内部荧光目标分布的线性关系,采用如下公式表示:After separation, Φ in the linear model is redefined as z, so the linear relationship between the surface measurement value and the internal fluorescent target distribution after separation is expressed by the following formula:

xS=Ax=z (26)x S=Ax=z (26)

其中,A是系统权重矩阵,z是盲源分离后各个光源在生物体表面光子分布的测量值,S是重建目标的三维分布。Among them, A is the system weight matrix, z is the measured value of the photon distribution of each light source on the surface of the organism after blind source separation, and S is the three-dimensional distribution of the reconstructed target.

由于盲源分离后每个光源的表面测量信号仍然是多个激发光联合作用的结果,对于第j个光源,FMT模型可表示如下:Since the surface measurement signal of each light source after blind source separation is still the result of the combined action of multiple excitation lights, for the jth light source, the FMT model can be expressed as follows:

到目前为止,我们成功地将多光源重建问题转化为多个单光源重建问题。So far, we have successfully transformed the multi-light source reconstruction problem into multiple single-light source reconstruction problems.

进一步,所述步骤四具体包括:Further, the step 4 specifically includes:

在FMT中,目标重建等价于找到等式Ax=z的近似解,故利用混合构成的矩阵与经BSS分离得到的矩阵z对每个光源进行单独重建。为了获得一个更稀疏的解决方案,受压缩感知理论启发,FMT模型可变换为以下具有l1正则化项的最小化问题:In FMT, target reconstruction is equivalent to finding an approximate solution to the equation Ax = z, so we use The matrix formed by the mixture and the matrix z obtained by BSS separation are used to reconstruct each light source separately. In order to obtain a sparser solution, inspired by the theory of compressed sensing, the FMT model can be transformed into the following minimization problem with an l 1 regularization term:

利用重建算法对每个光源分别进行重构,我们得到:Using the reconstruction algorithm to reconstruct each light source separately, we get:

Sj=(Sj[1],...,Sj[N])T j=1,2...,p (29)S j =(S j [1],...,S j [N]) T j=1,2...,p (29)

由于盲源分离建立的是一个通用的多光源重建模型,对算法没有依赖性,故重建算法不是本发明的重点,在接下来的实施例中,我们使用经典的快速迭代收缩阈值算法(以下简称FISTA)作为具体的重建算法。FISTA是一种十分受关注的基于梯度的算法,它由梯度下降法改进得到,对于经典的LASSO问题,基于软阈值函数的FISTA算法可以快速的迭代求解,具有时间复杂度,利用不同的近似函数起始点使收敛速度大大提升。Since blind source separation establishes a general multi-light source reconstruction model and has no algorithm dependency, the reconstruction algorithm is not the focus of the present invention. In the following embodiments, we use the classic fast iterative shrinkage threshold algorithm (hereinafter referred to as FISTA) as a specific reconstruction algorithm. FISTA is a very popular gradient-based algorithm, which is improved by the gradient descent method. For the classic LASSO problem, the FISTA algorithm based on the soft threshold function can quickly iterate and solve it. Time complexity, using different approximate function starting points can greatly improve the convergence speed.

最后,我们将各个目标重建的结果放在一个坐标系中合并展示。Finally, we combine the reconstruction results of each target in one coordinate system for display.

与现有技术相比,本发明具有以下优点:Compared with the prior art, the present invention has the following advantages:

第一,本发明从临床角度出发,专门针对实际肿瘤研究中由于肿瘤细胞的扩散与转移,需要对多个肿瘤区域进行综合分析和三维检测的问题。First, from a clinical perspective, the present invention specifically addresses the problem of comprehensive analysis and three-dimensional detection of multiple tumor regions due to the spread and metastasis of tumor cells in actual tumor research.

第二,本发明将基于表面测量信号进行盲源分离的思想引入多光源重建,将多目标重建问题转化为多个单目标重建问题,在一定程度上解决了FMT逆问题的严重不适定性导致的多目标重建精度低的问题。Second, the present invention introduces the idea of blind source separation based on surface measurement signals into multi-light source reconstruction, transforming the multi-target reconstruction problem into multiple single-target reconstruction problems, which to a certain extent solves the problem of low multi-target reconstruction accuracy caused by the serious ill-posedness of the FMT inverse problem.

附图说明BRIEF DESCRIPTION OF THE DRAWINGS

图1为本发明实施例提供的基于表面测量信号盲源分离的多目标重建方法的流程图;FIG1 is a flow chart of a multi-target reconstruction method based on blind source separation of surface measurement signals provided by an embodiment of the present invention;

图2为南加州大学数字鼠仿真模型,所有的器官和组织均已标记;Figure 2 shows the USC Digital Mouse simulation model, with all organs and tissues labeled;

图3为通过Amira软件剖分出来的不含光源的数字鼠仿真模型的.grid.am文件;FIG3 is a .grid.am file of a digital mouse simulation model without light source segmented by Amira software;

图4为通过tecplot软件展示的双目标混叠的表面测量信号的前向仿真结果,光源大小和形状为半径1.2mm的点光源,位置分别为(23.5,8,16.5)mm和(16.1,8,16.5)mm;其中(a)图为激发点1激发得到的表面光斑分布图,(b)图为激发点2激发得到的表面光斑分布图;FIG4 is a forward simulation result of the surface measurement signal of dual-target aliasing displayed by tecplot software. The size and shape of the light source is a point light source with a radius of 1.2 mm, and the positions are (23.5, 8, 16.5) mm and (16.1, 8, 16.5) mm respectively; (a) is the surface spot distribution diagram obtained by excitation point 1, and (b) is the surface spot distribution diagram obtained by excitation point 2;

图5为通过tecplot软件展示的上述双目标经盲源分离后表面测量信号的前向仿真结果,其中(a)图为目标1的表面光斑分布图,(b)图为目标2的表面光斑分布图;FIG5 is a forward simulation result of the surface measurement signal of the above dual targets after blind source separation displayed by tecplot software, wherein (a) is the surface spot distribution diagram of target 1, and (b) is the surface spot distribution diagram of target 2;

图6为盲源分离后使用FISTA算法重建的实验结果,其中(a)图为结果三维展示图,真实光源位置用箭头标记;(b)图为(a)图沿着XOY平面在Z=16.5mm处的切面图,圆圈代表真实光源位置,用箭头标记;FIG6 is an experimental result of reconstruction using the FISTA algorithm after blind source separation, where (a) is a three-dimensional display of the result, and the real light source position is marked with an arrow; (b) is a cross-sectional view of (a) along the XOY plane at Z=16.5 mm, and the circle represents the real light source position, which is marked with an arrow;

具体实施实例Specific implementation examples

下面将结合本发明实施例中的附图,对本发明实施例中的技术方案进行清楚、完整地描述,应指出的是,所描述的实施例仅仅是本发明一部分实施例,而不是全部的实施例。基于本发明中的实施例,本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例,都属于本发明保护的范围。The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. It should be noted that the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

本发明的重建方法,处理对象基于荧光分子断层成像技术,以生物组织内光传播的物理模型为基础,使用外部光源激发荧光探针使其发射光子,然后利用荧光采集装置(高灵敏度的CCD相机)收集生物体表的荧光信号,随后结合预先知道的生物组织内部的解剖结构以及相应各个组织的光学参数(如吸收系数、散射系数等),通过使用重建算法反向计算内部的光源目标的空间位置以及信号强度分布,进而定量的展示生物组织内部的早期病灶变化情况。具体涉及的是荧光分子断层成像技术领域中的一种基于表面测量信号盲源分离的多目标重建方法,该方法可用于精确寻找病变区域中多个肿瘤的位置,解决了由于FMT逆问题的严重不适定性导致的多目标重建结果不佳的问题。The reconstruction method of the present invention is based on the fluorescence molecular tomography technology, which is based on the physical model of light propagation in biological tissues. An external light source is used to excite the fluorescent probe to emit photons, and then a fluorescence acquisition device (a high-sensitivity CCD camera) is used to collect the fluorescence signal on the biological surface. Then, the spatial position of the internal light source target and the signal intensity distribution are reversely calculated by using a reconstruction algorithm in combination with the previously known anatomical structure inside the biological tissue and the corresponding optical parameters of each tissue (such as absorption coefficient, scattering coefficient, etc.), thereby quantitatively displaying the changes in the early lesions inside the biological tissue. Specifically, it involves a multi-target reconstruction method based on blind source separation of surface measurement signals in the field of fluorescence molecular tomography technology. The method can be used to accurately find the locations of multiple tumors in the lesion area, solving the problem of poor multi-target reconstruction results due to the serious ill-posedness of the FMT inverse problem.

下面结合附图,对本发明的应用原理做详细的描述。The application principle of the present invention is described in detail below in conjunction with the accompanying drawings.

参见图1,本发明以目标数目为2的情况为例,重建方法的流程如下:Referring to FIG. 1 , the present invention takes the case where the number of targets is 2 as an example, and the process of the reconstruction method is as follows:

S101:获取有限个角度的测量数据;S101: Acquire measurement data of a limited number of angles;

S102:获得重建目标的解剖结构信息以及光学特性参数,得到双目标叠加的表面测量信号的前向结果;S102: Obtain anatomical structure information and optical characteristic parameters of the reconstructed target, and obtain a forward result of a surface measurement signal of dual-target superposition;

S103:利用盲源分离算法,分别得到双目标经盲源分离后在生物组织表面的源测量信号的前向结果;S103: using a blind source separation algorithm, respectively obtaining forward results of source measurement signals of dual targets on the surface of biological tissue after blind source separation;

S104:建立各个源测量信号与体内光源分布之间的线性关系,利用FISTA算法分别重建各个目标,得到分离后的重建结果;S104: establishing a linear relationship between each source measurement signal and the distribution of the light source in the body, and reconstructing each target separately using the FISTA algorithm to obtain a separated reconstruction result;

所述步骤S101的实现过程:The implementation process of step S101 is as follows:

将FMT成像系统(激光和冷却CCD)和XCT成像系统(X射线管和X射线探测器)与小鼠中心呈直线,这两个系统相互垂直。激光产生的激发光照射小鼠表面,激发小鼠的体内的荧光目标产生发射荧光并传播到小鼠表面,然后由冷却的CCD相机收集这些荧光信号,同时,XCT系统获取小鼠的结构图像。The FMT imaging system (laser and cooled CCD) and the XCT imaging system (X-ray tube and X-ray detector) are placed in a straight line with the center of the mouse, and the two systems are perpendicular to each other. The excitation light generated by the laser illuminates the surface of the mouse, stimulating the fluorescent targets in the mouse's body to generate emitted fluorescence and propagate to the surface of the mouse, and then these fluorescent signals are collected by the cooled CCD camera. At the same time, the XCT system obtains the structural image of the mouse.

步骤S102具体包括:Step S102 specifically includes:

(1)用3Dmed软件对成像目标的计算机断层成像数据进行三维重建;(1) Perform three-dimensional reconstruction of the computed tomography data of the imaging target using 3Dmed software;

(2)采用Amira分割体数据得到成像目标的解剖结构信息;(2) Using Amira to segment the volume data, we can obtain the anatomical structure information of the imaging target;

(3)利用光通量密度和成像目标的解剖结构信息获得成像目标的光学特性参数;(3) Obtaining optical characteristic parameters of the imaging target using the light flux density and the anatomical structure information of the imaging target;

如前所述,发射光在成像物体内部和表面的分布向量Φm可表示为:As mentioned above, the distribution vector Φm of the emitted light inside and on the surface of the imaging object can be expressed as:

Φm=AΦxSΦ m = AΦ x S

对于FMT多目标重建,当一个激发点激发多个光源时,发出的光是多个光源发射光的叠加,成像物体内每个光源的δ(r-ri)不同,故不同光源荧光产额分布的权重不同,ωi=Φxiδ(r-ri),每个光源的发射光在成像物体内部和表面的分布向量可表示为:For FMT multi-target reconstruction, when one excitation point excites multiple light sources, the emitted light is the superposition of the light emitted by multiple light sources. The δ(r-ri) of each light source in the imaging object is different, so the weights of the fluorescence yield distribution of different light sources are different, ω ixi δ(r-ri). The distribution vector of the emitted light of each light source inside and on the surface of the imaging object can be expressed as:

Φmi=ωiASi Φ mi =ω i AS i

BSS模型可表示为下式:The BSS model can be expressed as follows:

Φ[n]=Ws[n] (n=1,2...,M)Φ[n]=Ws[n] (n=1,2...,M)

其中,Φ[n]=(Φ1[n],...,Φp[n])T是一个p×M的混叠观测矩阵,W=[ωij]p×k∈Rp×p是一个未知的混合矩阵,s[n]=(s1[n],...,sp[n])T指分离后的p×M的源矩阵。Among them, Φ[n]=(Φ 1 [n],...,Φ p [n]) T is a p×M aliasing measurement matrix, W=[ω ij ] p×k ∈R p×p is an unknown mixing matrix, and s[n]=(s 1 [n],...,s p [n]) T refers to the separated p×M source matrix.

在FMT中,可以将多个光源发射光的叠加作为BSS的混合观测值,即将表面光分布信息Φm作为混合观测值Φ。其中Φ为p×M的矩阵,我们将其作为盲源分离算法的输入数据矩阵,使其满足盲源分离的条件:In FMT, the superposition of light emitted by multiple light sources can be used as the mixed observation value of BSS, that is, the surface light distribution information Φm is used as the mixed observation value Φ. Where Φ is a p×M matrix, we use it as the input data matrix of the blind source separation algorithm to satisfy the conditions of blind source separation:

步骤S103的具体实现是:The specific implementation of step S103 is:

盲源分离后p个独立信号的表面光分布为:The surface light distribution of p independent signals after blind source separation is:

具体分离过程中,我们通过设计分离矩阵B={bij}p×p∈Rp×p使其满足In the specific separation process, we design the separation matrix B = {b ij } p×p ∈R p×p to satisfy

z[n]=BΦ[n]=BWs[n]=Ps[n]z[n]=BΦ[n]=BWs[n]=Ps[n]

其中,z[n]=(z1[n],...,zp[n])T表示分离或提取出的源信号,P是一个置换矩阵,意味着提取的源点z[n]可以等价于实际的源信号s[n]。Among them, z[n]=(z 1 [n], ..., z p [n]) T represents the separated or extracted source signal, and P is a permutation matrix, which means that the extracted source point z[n] can be equivalent to the actual source signal s[n].

本发明采用的算法通过最小化估计的非负源之间的联合相关函数来对FMT数据进行分解。分离矩阵B可以通过最大化单纯形cov{0,z1,...zp}(分离后)的体积来获得,该单纯形体积与cov{0,Φ1,...,Φp}(分离前)的体积相关,即The algorithm used in the present invention decomposes the FMT data by minimizing the estimated joint correlation function between non-negative sources. The separation matrix B can be obtained by maximizing the volume of the simplex cov{0,z 1 ,...z p } (after separation), which is related to the volume of cov{0,Φ 1 ,...,Φ p } (before separation), that is,

vol(cov{0,z1,...zp})=|det(B)|vol(cov{0,Φ1,...,Φp})vol(cov{0,z 1 ,...z p })=|det(B)|vol(cov{0,Φ 1 ,...,Φ p })

解决体积最大化问题就可以得到最优解:Solving the volume maximization problem gives us the optimal solution:

{zi,i=1,...,p}={si,i=1,...,p}{z i ,i=1,...,p}={s i ,i=1,...,p}

经分离后,线性模型中的Φ被重新定义为z,故分离后表面测量值与内部荧光目标分布的线性关系,采用如下公式表示:After separation, Φ in the linear model is redefined as z, so the linear relationship between the surface measurement value and the internal fluorescent target distribution after separation is expressed by the following formula:

xS=Ax=zx S=Ax=z

其中,A是系统权重矩阵,z是盲源分离后各个光源在生物体表面光子分布的测量值,S是重建目标的三维分布。Among them, A is the system weight matrix, z is the measured value of the photon distribution of each light source on the surface of the organism after blind source separation, and S is the three-dimensional distribution of the reconstructed target.

由于盲源分离后每个光源的表面测量信号仍然是多个激发光联合作用的结果,对于第j个光源,FMT模型可表示如下:Since the surface measurement signal of each light source after blind source separation is still the result of the combined action of multiple excitation lights, for the jth light source, the FMT model can be expressed as follows:

到目前为止,我们已经成功得到每个独立光源在生物体表面的光分布信息。So far, we have successfully obtained the light distribution information of each independent light source on the surface of the organism.

所述步骤S104为:The step S104 is:

在FMT中,目标重建等价于找到等式Ax=z的近似解,故利用混合构成的矩阵与经BSS分离得到的矩阵z对每个光源进行单独重建。为了获得一个更稀疏的解决方案,受压缩感知理论启发,FMT模型可变换为以下具有l1正则化项的最小化问题:In FMT, target reconstruction is equivalent to finding an approximate solution to the equation Ax = z, so we use The matrix formed by the mixture and the matrix z obtained by BSS separation are used to reconstruct each light source separately. In order to obtain a sparser solution, inspired by the theory of compressed sensing, the FMT model can be transformed into the following minimization problem with an l 1 regularization term:

利用逆算法对每个光源分别进行重构,我们得到:Using the inverse algorithm to reconstruct each light source separately, we get:

Sj=(Sj[1],...,Sj[N])T j=1,2...,pS j =(S j [1],...,S j [N]) T j=1,2...,p

我们使用经典的FISTA算法作为具体的重建算法分别重建分离后的两个目标,最后将各个目标重建的结果放在一个坐标系中合并展示。We use the classic FISTA algorithm as the specific reconstruction algorithm to reconstruct the two separated targets separately, and finally combine the reconstruction results of each target in a coordinate system for display.

下面结合仿真对本发明的应用效果做详细的描述。步骤如下:The following is a detailed description of the application effect of the present invention in combination with simulation. The steps are as follows:

一、获得含有光源的.raw文件。1. Get the .raw file containing the light source.

图2是用于仿真实验的数字鼠模型,由真实试验动物通过CT技术获取的成像切片数据中所提取的组织信息,经过断层成像重建手段获得去数字鼠模型,模型是尺寸为38.0mm×20.8mm×35.0mm的.raw文件,其内部不含有光源。为了降低计算复杂度,节约系统资源,去除小鼠的头部和尾部区域,只保留躯干部分。根据其内部组织结构,可划分为六个器官,及其肌体剩余组织,从上到下分别为心脏、肺、肝、胃、肾和肌肉。Figure 2 is a digital mouse model used in simulation experiments. The digital mouse model is obtained by tomographic reconstruction using tissue information extracted from imaging slice data obtained from real experimental animals through CT technology. The model is a .raw file with a size of 38.0mm×20.8mm×35.0mm, and does not contain a light source. In order to reduce computational complexity and save system resources, the head and tail regions of the mouse are removed, leaving only the trunk. According to its internal tissue structure, it can be divided into six organs and the remaining tissues of the body, from top to bottom, namely, heart, lungs, liver, stomach, kidneys and muscles.

二、获得整个仿真模型和光源的.grid.am文件。2. Obtain the .grid.am file of the entire simulation model and light source.

图3是通过Amira软件对I中获得的.raw文件进行剖分,获得整个仿真模型的网格结构,文件格式为.grid.am文件。FIG3 is a grid structure of the entire simulation model obtained by segmenting the .raw file obtained in FIG1 using Amira software. The file format is .grid.am file.

三、获得双目标混叠表面测量信号的仿真结果。3. Obtain simulation results of dual-target aliased surface measurement signals.

图4为双光源在生物组织表面光分布信息的仿真结果。生物组织内部探针受激辐射发出光子,光子在由内到外的传输过程中经历了反射、吸收、散射等多种物理光学过程,传输到生物体表面形成光分布信息。生物表面光分布信息与光源信息之间具有一对一的非线性映射关系。将II中获得的整个仿真模型的.grid.am文件通过仿真获得生物组织表面光分布信息。Figure 4 shows the simulation results of the light distribution information of the dual light sources on the surface of biological tissue. The probe inside the biological tissue emits photons by stimulated radiation. The photons undergo reflection, absorption, scattering and other physical optical processes during the transmission from the inside to the outside, and are transmitted to the surface of the organism to form light distribution information. There is a one-to-one nonlinear mapping relationship between the light distribution information on the biological surface and the light source information. The .grid.am file of the entire simulation model obtained in II is used to simulate the light distribution information on the surface of biological tissue.

四、利用混叠表面测量信号,生成预运行数据。4. Use the aliasing surface to measure the signal and generate pre-run data.

使用Matlab编程语言从III中获得的前向仿真结果文件中提取出生物组织表面光分布信息,并相应的读入II中所生成的.grid.am网格文件。根据网格文件和仿真结果文件,生成表面能量分布向量、网格节点坐标(X、Y和Z)矩阵、网格内部四面体矩阵、网格内部四面体索引矩阵和系统矩阵,将这些结果保存成Matlab对应的.mat文件。The light distribution information on the biological tissue surface is extracted from the forward simulation result file obtained in step III using the Matlab programming language, and the .grid.am grid file generated in step II is read in accordingly. Based on the grid file and the simulation result file, the surface energy distribution vector, the grid node coordinate (X, Y and Z) matrix, the grid internal tetrahedron matrix, the grid internal tetrahedron index matrix and the system matrix are generated, and these results are saved as the corresponding .mat file of Matlab.

五、将步骤IV得到的.mat文件进行盲源分离,分离后得到新的.mat文件。5. Perform blind source separation on the .mat file obtained in step IV to obtain a new .mat file after separation.

对于FMT多目标重建,当一个激发点激发多个光源时,发出的光是多个光源发射光的叠加。选择与光源数目相同的p个激发点进行多角度激发,探测器探测到的p个混叠信号的表面光分布,如果可以分离表面光分布信息,即得到解混后的源信号的表面光分布,就可以实现每个光源的单独重建,我们利用BSS算法将上一步得到的.mat文件进行盲源分离。For FMT multi-target reconstruction, when one excitation point excites multiple light sources, the emitted light is the superposition of the light emitted by multiple light sources. Select p excitation points with the same number of light sources for multi-angle excitation. If the surface light distribution information of the p aliased signals detected by the detector can be separated, that is, the surface light distribution of the unmixed source signal can be obtained, then each light source can be reconstructed separately. We use the BSS algorithm to perform blind source separation on the .mat file obtained in the previous step.

图5分别为分离后的两个目标表面测量信号的仿真结果。FIG5 shows the simulation results of the measurement signals of the two target surfaces after separation.

六、将步骤V得到的分离后的.mat文件作为重建算法的输入进行重建。6. Use the separated .mat file obtained in step V as the input of the reconstruction algorithm for reconstruction.

将数字鼠模型离散为由13064个节点和67649个四面体元素组成的四面体网格。The digital mouse model was discretized into a tetrahedral mesh consisting of 13064 nodes and 67649 tetrahedral elements.

图6为使用FISTA重建的实验结果,其中(a)图为三维结果展示图,真实光源是两个半径为1.2mm的球形荧光源,放置在肝脏中,中心分别为(16.1、8.0、16.5)和(23.5、8.0、16.5),两个光源之间的距离为5mm,位置用箭头标记;(b)图为(a)图沿着XOY平面在Z=16.5mm处的切面图,圆圈代表真实光源位置,用箭头标记;FIG6 is the experimental result reconstructed using FISTA, where (a) is a three-dimensional result display diagram. The real light source is two spherical fluorescent sources with a radius of 1.2 mm, which are placed in the liver. The centers are (16.1, 8.0, 16.5) and (23.5, 8.0, 16.5), respectively. The distance between the two light sources is 5 mm, and the positions are marked with arrows; (b) is a cross-sectional view of (a) along the XOY plane at Z = 16.5 mm. The circle represents the position of the real light source, which is marked with an arrow;

为了进一步定量评价BBS策略在源定位和形状恢复方面的能力,我们采用位置误差(LE)和Dice系数作为定量指标。To further quantitatively evaluate the ability of the BBS strategy in source localization and shape recovery, we use the location error (LE) and Dice coefficient as quantitative indicators.

LE测量了真实光源位置的中心与重建区域的中心之间的距离。LE定义为:LE measures the distance between the center of the true light source position and the center of the reconstructed area. LE is defined as:

LE=||Lr-L0||2 LE=||L r -L 0 || 2

其中Lr为重建区域的中心,L0为真实荧光区域的重心。||.||2是欧几里得距离的算子。LE指数越低,说明重建效果越好.Where L r is the center of the reconstructed area, L 0 is the center of gravity of the real fluorescence area. ||.|| 2 is the operator of Euclidean distance. The lower the LE index, the better the reconstruction effect.

本发明的实施例中,两个光源LE分别为0.56mm和0.53mm,其位置误差小,说明本发明提出的策略对多目标有很好的位置复原能力。In the embodiment of the present invention, the two light sources LE are 0.56 mm and 0.53 mm respectively, and their position errors are small, which shows that the strategy proposed in the present invention has good position restoration capability for multiple targets.

Dice系数用于评估重建区域与真实荧光区域的形态相似性,其定义为:The Dice coefficient is used to evaluate the morphological similarity between the reconstructed area and the real fluorescence area, which is defined as:

其中,Sr和S0分别表示重建区域面积和实际荧光区域面积。Dice系数越高,表明这两个区域在位置和形态上相似性越高。Among them, S r and S 0 represent the area of the reconstructed region and the area of the actual fluorescence region, respectively. The higher the Dice coefficient, the higher the similarity between the two regions in position and morphology.

本发明的实施例中,Dice系数为72%,其相似性较高,说明本发明提出的策略对多目标有很好的形态复原能力。In the embodiment of the present invention, the Dice coefficient is 72%, and the similarity is relatively high, which indicates that the strategy proposed in the present invention has good morphological restoration capability for multiple targets.

Claims (2)

1.一种基于表面测量信号盲源分离的多目标重建方法,其特征在于,包括以下步骤:1. A multi-target reconstruction method based on blind source separation of surface measurement signals, characterized in that it comprises the following steps: 步骤一:基于光传输模型和有限元理论,将重建荧光目标的结构信息和光学特征参数作为先验信息,对多目标测量信号产生的物理过程进行系统分析,说明其满足可进行盲源分离的条件;Step 1: Based on the light transmission model and finite element theory, the structural information and optical characteristic parameters of the reconstructed fluorescent target are used as prior information to systematically analyze the physical process of the multi-target measurement signal generation to show that it meets the conditions for blind source separation; 在荧光分子断层成像中,光的传输包括两个相关联的过程,即激发过程和发射过程,采用连续波成像方式,发射过程是物体内部的荧光团吸收了一定的激发光光能,一部分转化成光子释放出去,释放出去的这部分光叫做发射光,其波长比激发光波长要大,其光子能量小于激发光光子,激发光和发射光的传输过程可用下面两个耦合的扩散方程来描述:In fluorescence molecular tomography, the transmission of light includes two related processes, namely the excitation process and the emission process. Continuous wave imaging is used. In the emission process, the fluorophore inside the object absorbs a certain amount of excitation light energy, and part of it is converted into photons and released. The released part of light is called emission light, whose wavelength is larger than the wavelength of the excitation light, and its photon energy is less than the excitation light photon. The transmission process of excitation light and emission light can be described by the following two coupled diffusion equations: 其中Ω表示成像物体所占据的三维空间,下标x,m分别表示激发光和发射光,η是量子产能,是苂光团对激发光的吸收系数,它与苂光团浓度成正比,就是所要求的苂光产额分布,是狄拉克函数,Θ表示光源的强度;Where Ω represents the three-dimensional space occupied by the imaging object, the subscripts x and m represent the excitation light and the emission light respectively, and η is the quantum yield. is the absorption coefficient of the fluorophore to the excitation light, which is proportional to the concentration of the fluorophore. is the required distribution of light yield, is the Dirac function, Θ represents the intensity of the light source; 采用基于Galerkin变分法求解,并结合Robin边界条件:The solution is based on the Galerkin variational method and combined with the Robin boundary condition: 其中,是Ω的边界,v(r)是r的法向量,q是一个常数;设成像区域被离散成Np个点和Ne个四面体单元,并采用离散化网格上的基函数作为测试函数,因为任意函数都可表示为基函数的线性组合,所以基函数可以作为测试函数,于是可以表示为in, is the boundary of Ω, v(r) is the normal vector of r, and q is a constant; suppose the imaging area is discretized into Np points and Ne tetrahedral units, and the basis function on the discretized grid is used as the test function. Since any function can be expressed as a linear combination of basis functions, the basis function can be used as the test function, so and It can be expressed as 其中,Φxk,mk和(ημaf)k表示节点k上的函数值,表示节点k处的基函数;Among them, Φ xk,mk and (ημ af ) k represent the function value at node k, represents the basis function at node k; 对每一个单元,将(4)式和(5)式代入扩散方程,形成每个单元的刚度矩阵,将所有单元刚度矩阵进行叠加和组装,获得如下方程:For each unit, substitute equations (4) and (5) into the diffusion equation to form the stiffness matrix of each unit. Superimpose and assemble all unit stiffness matrices to obtain the following equation: KxΦx=Tx (6)K x Φ x =T x (6) KmΦm=Fx (7)K m Φ m =Fx (7) 其中,激发光在成像物体内的分布向量Φx可以通过直接求解方程(6)得到,再将其代入(7)中,则得到发射光在成像物体内部和表面的分布向量Φmin, make The distribution vector Φ x of the excitation light in the imaging object can be obtained by directly solving equation (6), and then substituting it into (7), the distribution vector Φ m of the emission light inside and on the surface of the imaging object is obtained: Φm=Km -1Fx=Ax=AΦxS (8)Φ m =K m -1 Fx=Ax=AΦ x S (8) 由于任意信号都可以用冲激信号的组合表示,将上式表示为:Since any signal can be represented by a combination of impulse signals, the above formula can be expressed as: 其中AΦx为M×N的向量,表示系统矩阵;S为N×1向量,表示内部荧光探针的分布,Φ为M×1的向量,表示生物体表面光子分布的测量值;Where AΦ x is an M×N vector, representing the system matrix; S is an N×1 vector, representing the distribution of internal fluorescent probes; Φ is an M×1 vector, representing the measured value of the photon distribution on the surface of the organism; 对于FMT多目标重建,当一个激发点激发多个光源时,发出的光是多个光源发射光的叠加,成像物体内每个光源的δ(r-ri)不同,故不同光源荧光产额分布的权重不同,ωi=Φxiδ(r-ri),每个光源的发射光在成像物体内部和表面的分布向量可表示为:For FMT multi-target reconstruction, when one excitation point excites multiple light sources, the emitted light is the superposition of the light emitted by multiple light sources. The δ(r-ri) of each light source in the imaging object is different, so the weights of the fluorescence yield distribution of different light sources are different, ω ixi δ(r-ri). The distribution vector of the emitted light of each light source inside and on the surface of the imaging object can be expressed as: Φmi=ωiASi (10)Φ mi =ω i AS i (10) 假设光源数目和激发光数目为p,探测器探测到的p个混叠信号的表面光分布为:Assuming that the number of light sources and the number of excitation lights are p, the surface light distribution of p aliased signals detected by the detector is: 其中ω=[ωij]p×p∈Rp×p为混合矩阵,如果可以分离表面光分布信息,即得到解混后的就可以实现每个光源的单独重建,这一步是多目标重建策略的核心;Where ω=[ω ij ] p×p ∈R p×p is the mixing matrix. If the surface light distribution information can be separated, the unmixed It is then possible to achieve individual reconstruction of each light source, which is the core of the multi-objective reconstruction strategy. 盲源分离算法,以下简称BSS,中唯一可用的信息来自于混合观测矩阵,BSS模型可表示为下式:In the blind source separation algorithm, hereinafter referred to as BSS, the only available information comes from the mixed observation matrix. The BSS model can be expressed as follows: Φ[n]=Ws[n] (n=1,2...,M) (12)Φ[n]=Ws[n] (n=1,2...,M) (12) 其中,Φ[n]=(Φ1[n],...,Φp[n])T是一个p×M的混叠观测矩阵,W=[ωij]p×k∈Rp×k是一个未知的混合矩阵,s[n]=(s1[n],...,sk[n])T指分离后的k×M的源矩阵;通常进一步假设Φ和s的维数相等,即p=k,假设:Φ[n]=(Φ1[n],...,Φp[n])T中每一行均包含p个源信息s1,s2…sp,这与FMT多目标信号混叠情况一致,故多目标测量信号产生的物理过程满足可进行盲源分离的条件;Among them, Φ[n]=(Φ 1 [n],...,Φ p [n]) T is a p×M aliasing observation matrix, W=[ω ij ] p×k ∈R p×k is an unknown mixing matrix, s[n]=(s 1 [n],...,s k [n]) T refers to the separated k×M source matrix; it is usually further assumed that the dimensions of Φ and s are equal, that is, p=k, and it is assumed that: each row in Φ[n]=(Φ 1 [n],...,Φ p [n]) T contains p source information s 1 ,s 2 …s p , which is consistent with the aliasing of FMT multi-target signals, so the physical process generated by the multi-target measurement signals meets the conditions for blind source separation; 步骤二:通过多激发点激发,构建符合盲源分离条件的表面叠加测量信号;Step 2: Construct a surface superposition measurement signal that meets the blind source separation conditions through multi-excitation point excitation; 在FMT中,可以将多个光源发射光的叠加作为BSS的混合观测值,即将表面光分布信息Φm作为混合观测值Φ;其中Φ为p×M的矩阵,将其作为盲源分离算法的输入数据矩阵,使其满足盲源分离的条件:In FMT, the superposition of light emitted by multiple light sources can be used as the mixed observation value of BSS, that is, the surface light distribution information Φm is used as the mixed observation value Φ; where Φ is a p×M matrix, which is used as the input data matrix of the blind source separation algorithm to meet the conditions of blind source separation: 步骤三:采用盲源分离的方法构建分离矩阵,将叠加的测量信号分离为各目标在生物组织表面的源测量信号,建立各个源测量信号与体内光源分布之间的线性关系;Step 3: Use the blind source separation method to construct a separation matrix, separate the superimposed measurement signals into source measurement signals of each target on the surface of the biological tissue, and establish a linear relationship between each source measurement signal and the distribution of the light source in the body; 对于混合的观测值,通过分离算法得到每个光源的表面光分布信息,进而分别重建每个光源,p=k的假设在此处表示分离光源的数量应与激发点的数量相一致,盲源分离后p个独立信号的表面光分布为:For mixed observations, the surface light distribution information of each light source is obtained through the separation algorithm, and then each light source is reconstructed separately. The assumption of p = k here means that the number of separated light sources should be consistent with the number of excitation points. The surface light distribution of p independent signals after blind source separation is: 具体分离过程中,通过设计分离矩阵B={bij}p×p∈Rp×p使其满足In the specific separation process, the separation matrix B = {b ij } p×p ∈R p×p is designed to satisfy z[n]=BΦ[n]=BWs[n]=Ps[n] (15)z[n]=BΦ[n]=BWs[n]=Ps[n] (15) 其中,z[n]=(z1[n],...,zp[n])T表示分离或提取出的源信号,P是一个置换矩阵,意味着提取的源点z[n]可以等价于实际的源信号s[n];Wherein, z[n]=(z 1 [n],...,z p [n]) T represents the separated or extracted source signal, and P is a permutation matrix, which means that the extracted source point z[n] can be equivalent to the actual source signal s[n]; 通过最小化估计的非负源之间的联合相关函数来对FMT数据进行分解,分离矩阵B可以通过最大化单纯形cov{0,z1,...zp},分离后的体积来获得,该单纯形体积与cov{0,Φ1,...,Φp},分离前的体积相关:The FMT data is decomposed by minimizing the estimated joint correlation function between non-negative sources. The separation matrix B can be obtained by maximizing the volume of the simplex cov{0,z 1 ,...z p }, after separation, which is related to the volume of cov{0,Φ 1 ,...,Φ p }, before separation: vol(cov{0,z1,...zp})=|det(B)|vol(cov{0,Φ1,...,Φp}) (16)vol(cov{0,z 1 ,...z p })=|det(B)|vol(cov{0,Φ 1 ,...,Φ p }) (16) 通过最大化由解混信源向量形成的固体区域的体积可以得到最优分离矩阵,该矩阵列满秩且矩阵各行元素和为1,故解决体积最大化问题就可以得到最优解:The optimal separation matrix can be obtained by maximizing the volume of the solid region formed by the demixing source vector. The matrix has full rank and the sum of the elements in each row of the matrix is 1. Therefore, the optimal solution can be obtained by solving the volume maximization problem: {zi,i=1,...,p}={si,i=1,...,p} (17){z i ,i=1,...,p}={s i ,i=1,...,p} (17) 因此,盲源分离问题可以转化为下面的非凸优化问题:Therefore, the blind source separation problem can be transformed into the following non-convex optimization problem: 在算法实施过程中,通过每次迭代更新矩阵B直到收敛,从而选取最优解;During the algorithm implementation, the matrix B is updated in each iteration until convergence, thereby selecting the optimal solution; 经分离后,线性模型中的Φ被重新定义为z,故分离后表面测量值与内部荧光目标分布的线性关系,采用如下公式表示:After separation, Φ in the linear model is redefined as z, so the linear relationship between the surface measurement value and the internal fluorescent target distribution after separation is expressed by the following formula: xS=Ax=z (19)其中,A是系统权重矩阵,z是盲源分离后各个光源在生物体表面光子分布的测量值,S是重建目标的三维分布;x S=Ax=z (19)where A is the system weight matrix, z is the measured value of the photon distribution of each light source on the surface of the organism after blind source separation, and S is the three-dimensional distribution of the reconstructed target; 由于盲源分离后每个光源的表面测量信号仍然是多个激发光联合作用的结果,对于第j个光源,FMT模型可表示如下:Since the surface measurement signal of each light source after blind source separation is still the result of the combined action of multiple excitation lights, for the jth light source, the FMT model can be expressed as follows: 为止将多光源重建问题转化为多个单光源重建问题;So far, the multi-light source reconstruction problem is transformed into multiple single-light source reconstruction problems; 步骤四:利用重建算法对每个目标分别进行重建;在FMT中,目标重建等价于找到等式Ax=z的近似解,故利用混合构成的矩阵与经BSS分离得到的矩阵z对每个光源进行单独重建,为了获得一个更稀疏的解决方案,FMT模型可变换为以下具有l1正则化项的最小化问题:Step 4: Use the reconstruction algorithm to reconstruct each target separately; in FMT, target reconstruction is equivalent to finding an approximate solution to the equation Ax = z, so use The matrix formed by the mixture and the matrix z obtained by BSS separation are used to reconstruct each light source separately. In order to obtain a sparser solution, the FMT model can be transformed into the following minimization problem with an l 1 regularization term: 利用重建算法对每个光源分别进行重构,得到:Use the reconstruction algorithm to reconstruct each light source separately and get: Sj=(Sj[1],...,Sj[N])T j=1,2...,p (22)S j =(S j [1],...,S j [N]) T j=1,2...,p (22) 由于盲源分离建立的是一个通用的多光源重建模型,对重建算法没有依赖性;因此,在接下来的实施例中,采用经典的快速迭代收缩阈值算法,以下简称FISTA,作为具体的重建算法,基于软阈值函数的FISTA算法可以快速的迭代求解,具有时间复杂度,利用不同的近似函数起始点使收敛速度提升;最后,将各个目标重建的结果放在一个坐标系中合并展示。Since blind source separation establishes a universal multi-light source reconstruction model, it has no dependence on the reconstruction algorithm; therefore, in the following embodiments, the classic fast iterative shrinkage threshold algorithm, hereinafter referred to as FISTA, is used as a specific reconstruction algorithm. The FISTA algorithm based on the soft threshold function can be quickly iterated and solved. Time complexity, using different approximate function starting points to improve the convergence speed; finally, the reconstruction results of each target are combined and displayed in a coordinate system. 2.根据权利要求1所述的基于表面测量信号盲源分离的多目标重建方法,其特征在于,所述的步骤三中盲源分离问题转化如下非凸优化问题:2. The multi-objective reconstruction method based on blind source separation of surface measurement signals according to claim 1, characterized in that the blind source separation problem in step 3 is transformed into the following non-convex optimization problem: (1)p=k=2(1) p = k = 2 此情况下可以使用解析法求闭式解,将公式(18)中的等式约束整合到目标函数中,得到:In this case, we can use the analytical method to find the closed-form solution and integrate the equality constraint in formula (18) into the objective function to obtain: 假设det(B)≥0,公式(18)转化为max(B11-B21)(25)Assuming det(B)≥0, formula (18) is transformed into max(B 11 -B 21 )(25) s.t.B11Φ1[n]+(1-B11Φ2[n])≥0stB 11 Φ 1 [n]+(1-B 11 Φ 2 [n])≥0 B21Φ1[n]+(1-B21Φ2[n])≥0B 21 Φ 1 [n]+(1-B 21 Φ 2 [n])≥0 上述约束条件可等价为:β≤B11,B21≤αThe above constraints can be equivalent to: β≤B 11 ,B 21 ≤α 其中:in: 因此,公式(20)最终转化为以下形式:Therefore, formula (20) is finally transformed into the following form: 最优解为B11 *=α且B21 *=β,因此,最优值为det(B*)=α-β>0;The optimal solution is B 11 * = α and B 21 * = β, so the optimal value is det(B * ) = α-β>0; 同理,可以证明det(B)≤0的情况,最优值为det(B*)=β-α<0;Similarly, it can be proved that when det(B)≤0, the optimal value is det(B * )=β-α<0; (2)p=k>2(2) p = k > 2 对B的任意一行进行代数余子式展开,以第i行为例,记Bi T=[Bi1,Bi2,...,BiN],则行列式为:Expand any row of B by algebraic co-factors. Take the i-th row as an example, let B i T = [B i1 ,B i2 ,...,B iN ], then the determinant is: 其中,Bij是删去B的第i行和第j列的子矩阵;显然,当Bij关于j=1,2,...,N固定时,行列式det(B)变为关于Bi的线性函数;因此,通过固定B的其他行向量,公式(20)就转化为如下非凸最大化问题:Where Bij is the submatrix of B with the i-th row and j-th column deleted. Obviously, when Bij is fixed with respect to j=1,2,...,N, the determinant det(B) becomes a linear function with respect to Bi . Therefore, by fixing the other row vectors of B, formula (20) is transformed into the following non-convex maximization problem: 公式(23)中目标函数仍然时非凸的,通过求解一下两个线性规划问题,可以得到该局部最大化问题的全局最优解The objective function in formula (23) is still non-convex. By solving the following two linear programming problems, we can obtain the global optimal solution to the local maximization problem:
CN202211207887.2A 2022-09-30 2022-09-30 Multi-target reconstruction method based on surface measurement signal blind source separation Active CN115607108B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211207887.2A CN115607108B (en) 2022-09-30 2022-09-30 Multi-target reconstruction method based on surface measurement signal blind source separation

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211207887.2A CN115607108B (en) 2022-09-30 2022-09-30 Multi-target reconstruction method based on surface measurement signal blind source separation

Publications (2)

Publication Number Publication Date
CN115607108A CN115607108A (en) 2023-01-17
CN115607108B true CN115607108B (en) 2024-10-29

Family

ID=84860010

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211207887.2A Active CN115607108B (en) 2022-09-30 2022-09-30 Multi-target reconstruction method based on surface measurement signal blind source separation

Country Status (1)

Country Link
CN (1) CN115607108B (en)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117830565B (en) * 2024-03-06 2024-06-14 华中科技大学 Multi-operator self-supervision fluorescent molecular tomography method and system
WO2025251227A1 (en) * 2024-06-05 2025-12-11 合肥锐视医疗科技有限公司 Optical imaging-based image reconstruction method and apparatus, storage medium, and optical imaging device
CN118731608B (en) * 2024-07-08 2025-09-05 保定华创电气有限公司 Partial discharge detection method and device
CN119073923A (en) * 2024-09-30 2024-12-06 天津大学 Breast dynamic optical tomography system based on fluorescence image guidance

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102137618A (en) * 2008-07-25 2011-07-27 健康与环境慕尼黑德国研究中心赫姆霍茨中心(有限公司) Quantitative multi-spectral opto-acoustic tomography (MSOT) of tissue biomarkers
CN105548952A (en) * 2015-12-03 2016-05-04 中国电子科技集团公司第五十四研究所 Common-frequency multi-signal direction-finding method based on blind source separation

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6535666B1 (en) * 1995-06-02 2003-03-18 Trw Inc. Method and apparatus for separating signals transmitted over a waveguide
WO2013159250A1 (en) * 2012-04-28 2013-10-31 清华大学 4d fluorescence molecular tomography image reconstruction method
CN107220961A (en) * 2017-06-14 2017-09-29 西北大学 A kind of fluorescent molecule tomography rebuilding method based on half threshold value tracing algorithm

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102137618A (en) * 2008-07-25 2011-07-27 健康与环境慕尼黑德国研究中心赫姆霍茨中心(有限公司) Quantitative multi-spectral opto-acoustic tomography (MSOT) of tissue biomarkers
CN105548952A (en) * 2015-12-03 2016-05-04 中国电子科技集团公司第五十四研究所 Common-frequency multi-signal direction-finding method based on blind source separation

Also Published As

Publication number Publication date
CN115607108A (en) 2023-01-17

Similar Documents

Publication Publication Date Title
CN115607108B (en) Multi-target reconstruction method based on surface measurement signal blind source separation
CN102488493B (en) Small animal living body multi-mode molecule imaging system and imaging method
US8280124B2 (en) Methods of view selection for radioactive emission measurements
CN103271723B (en) Bioluminescence tomography reconstruction method
CN103300829B (en) Biological autofluorescence tomography method based on iteration reweighting
Jiang et al. Reconstruction of fluorescence molecular tomography via a fused LASSO method based on group sparsity prior
CN108451508A (en) Archebiosis fluorescent three-dimensional imaging method based on multi-layer perception (MLP)
CN107146261B (en) Quantitative reconstruction of bioluminescence tomography based on a priori region of interest in magnetic resonance imaging
CN114581553B (en) Fluorescent molecular tomography reconstruction method based on magnetic particle imaging prior guidance
CN102988026A (en) Auto-fluorescence tomography re-establishing method based on multiplier method
CN107220961A (en) A kind of fluorescent molecule tomography rebuilding method based on half threshold value tracing algorithm
CN108095686B (en) Fluorescence molecular tomography target feasible region selection method
CN119108096A (en) A method for establishing a risk prediction model for malignant pulmonary nodules based on comprehensive imaging indicators and exhaled breath VOCs data
Zhao et al. A robust elastic net-ℓ 1 ℓ 2 reconstruction method for x-ray luminescence computed tomography
Wu et al. Detection of mouse liver cancer via a parallel iterative shrinkage method in hybrid optical/microcomputed tomography imaging
Cheng et al. Maximum likelihood activity and attenuation estimation using both emission and transmission data with application to utilization of Lu‐176 background radiation in TOF PET
CN112089434A (en) A method and system for multispectral bioluminescence tomography
CN115494437B (en) Hand-held low-gradient single-side nuclear magnetic resonance device for detecting full-layer skin
Yavuz et al. Calculation of recovery coefficients for partial volume effect correction in PET/CT imaging using a customized anthropomorphic body phantom
CN107374588B (en) Multi-light-source fluorescent molecular tomography reconstruction method based on synchronous clustering
Yi et al. A permissible region extraction based on a knowledge priori for x-ray luminescence computed tomography
Du et al. MSCN-NET: multi-stage cascade neural network based on attention mechanism for Čerenkov luminescence tomography
CN108309340B (en) Excited Fluorescence Tomography Reconstruction Based on Correlation Entropy Matching Tracking
Zhang et al. Adaptive permissible region based random Kaczmarz reconstruction method for localization of carotid atherosclerotic plaques in fluorescence molecular tomography
Yi et al. Variable Step-Size Fast Iterative Shrinkage Thresholding Algorithm for Fluorescence Molecular Tomography

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant