[go: up one dir, main page]

CN114646606A - Spectrum water quality detection method - Google Patents

Spectrum water quality detection method Download PDF

Info

Publication number
CN114646606A
CN114646606A CN202210383168.XA CN202210383168A CN114646606A CN 114646606 A CN114646606 A CN 114646606A CN 202210383168 A CN202210383168 A CN 202210383168A CN 114646606 A CN114646606 A CN 114646606A
Authority
CN
China
Prior art keywords
turbidity
water quality
matrix
absorption spectrum
water
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210383168.XA
Other languages
Chinese (zh)
Inventor
许廷发
臧传凯
辜清
罗飞
孙丽敏
杨洋
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Chongqing Innovation Center of Beijing University of Technology
Original Assignee
Chongqing Innovation Center of Beijing University of Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Chongqing Innovation Center of Beijing University of Technology filed Critical Chongqing Innovation Center of Beijing University of Technology
Priority to CN202210383168.XA priority Critical patent/CN114646606A/en
Publication of CN114646606A publication Critical patent/CN114646606A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/17—Systems in which incident light is modified in accordance with the properties of the material investigated
    • G01N21/25—Colour; Spectral properties, i.e. comparison of effect of material on the light at two or more different wavelengths or wavelength bands
    • G01N21/31—Investigating relative effect of material at wavelengths characteristic of specific elements or molecules, e.g. atomic absorption spectrometry
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/17—Systems in which incident light is modified in accordance with the properties of the material investigated
    • G01N21/25—Colour; Spectral properties, i.e. comparison of effect of material on the light at two or more different wavelengths or wavelength bands
    • G01N21/31—Investigating relative effect of material at wavelengths characteristic of specific elements or molecules, e.g. atomic absorption spectrometry
    • G01N21/33—Investigating relative effect of material at wavelengths characteristic of specific elements or molecules, e.g. atomic absorption spectrometry using ultraviolet light
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/10—Complex mathematical operations
    • G06F17/16—Matrix or vector computation, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/10—Complex mathematical operations
    • G06F17/18—Complex mathematical operations for evaluating statistical data, e.g. average values, frequency distributions, probability functions, regression analysis

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Data Mining & Analysis (AREA)
  • Theoretical Computer Science (AREA)
  • Computational Mathematics (AREA)
  • Mathematical Analysis (AREA)
  • Pure & Applied Mathematics (AREA)
  • Mathematical Optimization (AREA)
  • Spectroscopy & Molecular Physics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Analytical Chemistry (AREA)
  • Chemical & Material Sciences (AREA)
  • Software Systems (AREA)
  • Databases & Information Systems (AREA)
  • Pathology (AREA)
  • General Engineering & Computer Science (AREA)
  • Immunology (AREA)
  • General Health & Medical Sciences (AREA)
  • Biochemistry (AREA)
  • Algebra (AREA)
  • Health & Medical Sciences (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computing Systems (AREA)
  • Probability & Statistics with Applications (AREA)
  • Operations Research (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Investigating Or Analysing Materials By Optical Means (AREA)

Abstract

The invention provides a spectral water quality detection method, which comprises the following steps: acquiring an absorption spectrum of the water body to be detected in all ultraviolet-visible light bands; inverting the turbidity of the water body to be detected by using a visible light waveband, and carrying out turbidity correction on the turbidity of the water body to be detected based on a singular value decomposition method to obtain an absorption spectrum after turbidity correction; and converting the absorption spectrum after turbidity correction into various water quality parameters by using an inversion model trained in advance by a partial least square regression method. The invention carries out turbidity correction on the absorption spectrum by a singular value decomposition method, reduces the number of wave bands participating in the turbidity correction, improves the precision of the turbidity correction and further improves the inversion precision of other water quality parameters. And a laboratory standard sample and environmental water sample spectrum database is constructed during regression modeling, so that training samples can be supplemented during subsequent use, the inversion range of the model is expanded, and the environmental adaptability and the inversion precision of the model are improved.

Description

一种光谱水质检测方法A kind of spectral water quality detection method

技术领域technical field

本发明涉及水体水质检测技术领域，具体而言，涉及一种光谱水质检测方法。The invention relates to the technical field of water quality detection, in particular to a spectral water quality detection method.

背景技术Background technique

在水环境治理和保护过程中，检测各类水质参数的浓度能够反映水环境的好坏以及受污染的程度。在《地表水环境质量标准》(GB3838-2002)、《污水综合排放标准》(GB8978-1996)等国家标准中都明确规定了各类水质参数的浓度限值，因此各类水质参数的准确、快速检测非常重要。In the process of water environment treatment and protection, detecting the concentration of various water quality parameters can reflect the quality of the water environment and the degree of pollution. National standards such as "Surface Water Environmental Quality Standard" (GB3838-2002), "Integrated Wastewater Discharge Standard" (GB8978-1996) and other national standards clearly stipulate the concentration limits of various water quality parameters. Quick detection is very important.

传统的水质检测方法为化学法，该方法虽然精度高，但需要花费大量的成本且检测过程中容易产生二次污染。鉴于目前常用的水质检测方法存在的问题，通过光谱法进行水质检测逐渐受到人们的关注。由于许多水质参数在紫外-可见光波段有较强的吸收峰，通过分析水体的吸收光谱，可以对水体所含的水质参数进行定性与定量分析，并且通过光谱法进行水质检测具有快速、免试剂、低成本等优点。The traditional water quality detection method is a chemical method. Although this method has high precision, it requires a lot of cost and is prone to secondary pollution during the detection process. In view of the problems of the commonly used water quality testing methods, the water quality testing by spectrometry has gradually attracted people's attention. Since many water quality parameters have strong absorption peaks in the ultraviolet-visible light band, by analyzing the absorption spectrum of the water body, the water quality parameters contained in the water body can be qualitatively and quantitatively analyzed. advantages of low cost.

然而，目前通过光谱法进行水质检测存在较大的技术难点。这是由于水体中不同组分对光谱的吸收能力和吸收波段存在重叠，实际获得的原位水体吸收光谱往往是由水体中多种组分共同作用下形成的，这种情况使得预测结果与实际结果存在偏差。如水质参数中的COD只在紫外波段有吸收峰，而浊度在紫外-可见光波段都存在不同程度的吸收，因此水体中浊度的存在会影响COD的预测精度。However, there are major technical difficulties in water quality detection by spectroscopy. This is because the absorption capacity and absorption bands of different components in the water body overlap, and the actually obtained in-situ water absorption spectrum is often formed by the joint action of various components in the water body. This situation makes the predicted results consistent with the actual The results are biased. For example, COD in water quality parameters only has absorption peaks in the ultraviolet band, while turbidity has different degrees of absorption in the ultraviolet-visible band. Therefore, the existence of turbidity in the water body will affect the prediction accuracy of COD.

近年来，浊度对其他水质参数反演精度的影响已经有了许多研究，大多数通过浊度补偿的方法扣除浊度对原始吸收光谱的影响，用浊度补偿后的光谱进行各类水质参数的反演。然而该方法很难完全扣除浊度对光谱的影响，特别在紫外波段，因此也影响到后续各类水质参数的反演精度，故而通过原始吸收光谱进行浊度补偿的方法进行水质参数的反演仍然存在较大的偏差。此外，目前应用较多的光谱水质检测方法，多采用紫外-可见光波段的几个特征波段对水质参数进行反演，这样不仅损失了大量的光谱信息，并且浓度的变化也会引起特征波段中心波长的偏移，造成反演精度的下降。因此，我们需要寻找一种新的能够有效去除浊度影响的浊度补偿方法，同时利用全光谱的信息量优势，提高各类水质参数的光谱检测精度。In recent years, there have been many studies on the influence of turbidity on the inversion accuracy of other water quality parameters. Most of them use the turbidity compensation method to deduct the influence of turbidity on the original absorption spectrum, and use the turbidity-compensated spectrum for various water quality parameters. inversion. However, it is difficult for this method to completely deduct the influence of turbidity on the spectrum, especially in the ultraviolet band, so it also affects the inversion accuracy of subsequent water quality parameters. Therefore, the method of turbidity compensation through the original absorption spectrum is used to invert the water quality parameters. There are still large deviations. In addition, many spectral water quality detection methods are currently used, and several characteristic bands in the ultraviolet-visible light band are used to invert water quality parameters, which not only loses a lot of spectral information, but also causes changes in concentration to cause the center wavelength of the characteristic band. , resulting in a decrease in the inversion accuracy. Therefore, we need to find a new turbidity compensation method that can effectively remove the influence of turbidity, and at the same time use the information advantage of the full spectrum to improve the spectral detection accuracy of various water quality parameters.

奇异值分解可以有效提取原始光谱特征，将大部分有效信息集中在几个特征波段中，通过对几个特征波段进行处理，然后逆变换回原始光谱，可以有效实现光谱的浊度矫正。偏最小二乘回归法提供了一种多对多线性回归建模的方法，对于两组变量(自变量与因变量)个数很多，且都存在多重相关性，而样本量较少时，可以提供一个较为精准的回归模型。Singular value decomposition can effectively extract the original spectral features, and concentrate most of the effective information in several characteristic bands. By processing several characteristic bands, and then inversely transforming back to the original spectrum, spectral turbidity correction can be effectively achieved. The partial least squares regression method provides a method for modeling many-to-many linear regression. For the two groups of variables (independent variables and dependent variables), there are a lot of them, and there are multiple correlations, but when the sample size is small, you can Provide a more accurate regression model.

发明内容SUMMARY OF THE INVENTION

本发明旨在提供一种光谱水质检测方法，以提高水质检测效率和精度。The invention aims to provide a spectral water quality detection method to improve the water quality detection efficiency and precision.

本发明提供的一种光谱水质检测方法，包括如下步骤：A spectral water quality detection method provided by the present invention comprises the following steps:

步骤S10，获取待测水体在紫外-可见光全波段的吸收光谱；Step S10, acquiring the absorption spectrum of the water body to be tested in the full wavelength range of ultraviolet-visible light;

步骤S20，利用可见光波段反演待测水体的浊度，并基于奇异值分解法对待测水体的浊度进行浊度校正，获得浊度校正后的吸收光谱；Step S20, inverting the turbidity of the water body to be measured by using the visible light band, and performing turbidity correction on the turbidity of the water body to be measured based on the singular value decomposition method to obtain a turbidity-corrected absorption spectrum;

步骤S30，通过利用偏最小二乘回归方法预先训练好的反演模型将浊度校正后的吸收光谱转化为各类水质参数。In step S30, the turbidity-corrected absorption spectrum is converted into various water quality parameters by using the pre-trained inversion model using the partial least squares regression method.

在一些实施例中，步骤S20包括如下子步骤：In some embodiments, step S20 includes the following sub-steps:

步骤S21，获取不同水质参数浓度水体在紫外-可见光全波段的吸收光谱作为训练数据；Step S21, acquiring absorption spectra of water bodies with different water quality parameter concentrations in the entire ultraviolet-visible light band as training data;

步骤S22，利用训练数据的浊度和在可见光波段的吸光度构建一元线性回归模型；Step S22, using the turbidity of the training data and the absorbance in the visible light band to construct a univariate linear regression model;

步骤S23，基于奇异值分解法对训练数据的吸收光谱进行特征提取，保留特征提取后的前三个波段的奇异值；Step S23, based on the singular value decomposition method, perform feature extraction on the absorption spectrum of the training data, and retain the singular values of the first three bands after the feature extraction;

步骤S24，基于特征提取后的前三个波段的奇异值构建浊度校正模型；Step S24, building a turbidity correction model based on the singular values of the first three bands after feature extraction;

步骤S25，利用一元线性回归模型反演待测水体的浊度，并通过浊度校正模型对待测水体的浊度进行浊度校正，获得浊度校正后的吸收光谱。Step S25 , inverting the turbidity of the water body to be measured by using a univariate linear regression model, and performing turbidity correction on the turbidity of the water body to be measured by using the turbidity correction model to obtain a turbidity-corrected absorption spectrum.

在一些实施例中，步骤S22中利用训练数据的浊度和在可见光波段的吸光度构建一元线性回归模型的方法包括：In some embodiments, the method for constructing a univariate linear regression model using the turbidity of the training data and the absorbance in the visible light band in step S22 includes:

获取训练数据中各水体在可见光波段的吸光度，并利用各水体的浊度与可见光波段的吸光度构建一元线性回归模型：Obtain the absorbance of each water body in the visible light band in the training data, and use the turbidity of each water body and the absorbance in the visible light band to build a univariate linear regression model:

Ttrain＝α*Ltrain+β (1)T train =α*L train +β (1)

式中，Ttrain表示训练数据的浊度，Ltrain表示训练数据在550nm的吸光度，α和β代表一元线性回归模型拟合的参数。where T train represents the turbidity of the training data, L train represents the absorbance of the training data at 550 nm, and α and β represent the parameters fitted by the univariate linear regression model.

在一些实施例中，步骤S23中基于奇异值分解法对训练数据的吸收光谱进行特征提取的方法包括：In some embodiments, the method for performing feature extraction on the absorption spectrum of the training data based on the singular value decomposition method in step S23 includes:

Figure BDA0003593755140000031
Figure BDA0003593755140000031

式中，

Figure BDA0003593755140000032
表示训练数据的吸收光谱矩阵，上标“T”表示矩阵的转置；
Figure BDA0003593755140000033
表示左奇异矩阵；
Figure BDA0003593755140000034
表示奇异值矩阵；
Figure BDA0003593755140000035
表示右奇异矩阵；λ表示特征值矩阵，n表示训练数据的样本数量、m表示训练数据中样本的波段数量；由此保留特征提取后的前三个波段的奇异值
Figure BDA0003593755140000036
作为后续参与浊度校正的数据。In the formula,
Figure BDA0003593755140000032
represents the absorption spectrum matrix of the training data, and the superscript "T" represents the transpose of the matrix;
Figure BDA0003593755140000033
represents a left singular matrix;
Figure BDA0003593755140000034
represents the singular value matrix;
Figure BDA0003593755140000035
represents the right singular matrix; λ represents the eigenvalue matrix, n represents the number of samples in the training data, and m represents the number of bands in the training data; thus, the singular values of the first three bands after feature extraction are retained.
Figure BDA0003593755140000036
As the data for subsequent participation in turbidity correction.

在一些实施例中，步骤S24中基于特征提取后的前三个波段奇异值构建浊度校正模型的方法包括：In some embodiments, the method for constructing a turbidity correction model based on the singular values of the first three bands after feature extraction in step S24 includes:

从奇异值分解后的训练数据中挑选出混合溶液样本D，以及与之对应的0浊度的混合溶液样本E，然后通过公式(3)计算得到浊度在混合溶液样本中各波段的贡献来作为浊度校正模型：The mixed solution sample D and the corresponding mixed solution sample E with 0 turbidity are selected from the training data after singular value decomposition, and then the contribution of turbidity in each band in the mixed solution sample is calculated by formula (3). As a turbidity correction model:

(Dj-Ej)＝γj*T+δj, 1≤j≤3 (3)(D j -E j )=γ j *T+δ j , 1≤j≤3 (3)

式中，Dj表示第j个波段的混合溶液样本D，Ej表示第j个波段的0浊度的混合溶液样本E；T表示混合溶液样本D的浊度；γj和δj表示浊度校正模型拟合的参数。In the formula, D j represents the mixed solution sample D in the j-th band, E j represents the mixed solution sample E with 0 turbidity in the j-th band; T represents the turbidity of the mixed solution sample D; γ j and δ j represent the turbidity The parameters of the degree-corrected model fit.

在一些实施例中，步骤S25中利用一元线性回归模型反演待测水体的浊度，并通过浊度校正模型对待测水体的浊度进行浊度校正，获得浊度校正后的吸收光谱的方法包括：In some embodiments, in step S25, the turbidity of the water body to be measured is inverted by a univariate linear regression model, and the turbidity correction model is used to perform turbidity correction on the turbidity of the water body to be measured to obtain the absorption spectrum after turbidity correction. include:

步骤S251，利用一元线性回归模型反演待测水体的浊度：Step S251, use the univariate linear regression model to invert the turbidity of the water body to be measured:

Ttest＝α*Ltest+β (4)T test = α*L test + β (4)

式中，Ttest表示待测水体的浊度；Ltest表示待测水体在550nm的吸光度；In the formula, T test represents the turbidity of the water body to be tested; L test represents the absorbance of the water body to be tested at 550 nm;

步骤S252，根据待测水体的浊度并利用浊度校正模型计算奇异值差值：Step S252, according to the turbidity of the water body to be measured and using the turbidity correction model to calculate the singular value difference:

Figure BDA0003593755140000041
Figure BDA0003593755140000041

式中，

Figure BDA0003593755140000042
表示待测水体的奇异值差值；In the formula,
Figure BDA0003593755140000042
Represents the singular value difference of the water body to be measured;

步骤S253，利用待测水体的奇异值波段减去奇异值差值完成浊度校正：Step S253, using the singular value band of the water body to be measured minus the singular value difference to complete the turbidity correction:

Figure BDA0003593755140000043
Figure BDA0003593755140000043

式中，

Figure BDA0003593755140000044
表示完成浊度校正的计算结果，
Figure BDA0003593755140000045
表示待测水体的奇异值波段；In the formula,
Figure BDA0003593755140000044
Indicates the calculation result of the completed turbidity correction,
Figure BDA0003593755140000045
Represents the singular value band of the water body to be measured;

步骤S254，对完成浊度校正的计算结果

Figure BDA0003593755140000046
进行奇异值逆变换，得到浊度校正后的吸收光谱：Step S254, the calculation result of the completed turbidity correction
Figure BDA0003593755140000046
Perform inverse singular value transformation to obtain the absorption spectrum after turbidity correction:

Figure BDA0003593755140000047
Figure BDA0003593755140000047

式中，

Figure BDA0003593755140000048
表示浊度校正后的吸收光谱，
Figure BDA0003593755140000049
表示完成浊度校正的计算结果，Σ3×3表示对应的奇异值矩阵，V3×m表示对应的右奇异矩阵。In the formula,
Figure BDA0003593755140000048
represents the absorption spectrum after turbidity correction,
Figure BDA0003593755140000049
Indicates the calculation result of completing the turbidity correction, Σ 3×3 represents the corresponding singular value matrix, and V 3×m represents the corresponding right singular matrix.

在一些实施例中，步骤S30中通过利用偏最小二乘回归方法预先训练好的反演模型将浊度校正后的吸收光谱转化为各类水质参数的方法包括：In some embodiments, the method for converting the turbidity-corrected absorption spectrum into various water quality parameters by using an inversion model pre-trained by the partial least squares regression method in step S30 includes:

步骤S31，记浊度校正后的吸收光谱矩阵为X，对吸收光谱矩阵X和各水质参数矩阵Y进行标准化，得到标准化后的吸收光谱矩阵A和各水质参数矩阵B，以对吸收光谱矩阵X进行标准化为例：Step S31, denote the absorption spectrum matrix after turbidity correction as X, standardize the absorption spectrum matrix X and each water quality parameter matrix Y, and obtain the standardized absorption spectrum matrix A and each water quality parameter matrix B, so as to measure the absorption spectrum matrix X. For example for normalization:

Figure BDA0003593755140000051
Figure BDA0003593755140000051

Figure BDA0003593755140000052
Figure BDA0003593755140000052

Figure BDA0003593755140000053
Figure BDA0003593755140000053

式中，Aij(n×m)表示标准化后的吸收光谱矩阵A中的第i个样本第j个元素元素；Xij表示吸收光谱矩阵X中的第i个样本第j个元素元素；

Figure BDA0003593755140000054
表示吸收光谱矩阵X中第j列的均值；
Figure BDA0003593755140000055
表示吸收光谱矩阵X中第j列的的均方差；同理得到标准化后的各水质参数矩阵B；In the formula, A ij (n×m) represents the j-th element of the ith sample in the normalized absorption spectrum matrix A; X ij represents the j-th element of the ith sample in the absorption spectrum matrix X;
Figure BDA0003593755140000054
represents the mean value of the jth column in the absorption spectrum matrix X;
Figure BDA0003593755140000055
Represents the mean square error of the jth column in the absorption spectrum matrix X; similarly, the standardized water quality parameter matrix B is obtained;

步骤S32，通过计算最大特征值所对应的特征向量ρ，进而计算标准化后的吸收光谱矩阵A的第一对主成分W1及其得分向量

Figure BDA0003593755140000056
Step S32, by calculating the eigenvector ρ corresponding to the maximum eigenvalue, and then calculating the first pair of principal components W1 of the standardized absorption spectrum matrix A and its score vector
Figure BDA0003593755140000056

Figure BDA0003593755140000057
Figure BDA0003593755140000057

式中，ρ1表示第一对主成分W1对应的特征向量；In the formula, ρ 1 represents the eigenvector corresponding to the first pair of principal components W 1 ;

步骤S33，建立标准化后的吸收光谱矩阵A和各水质参数矩阵B对第一对主成分W1的回归，并且求得回归后的残差矩阵A1和B1：Step S33, establish the regression of the standardized absorption spectrum matrix A and each water quality parameter matrix B on the first pair of principal components W 1 , and obtain the regressed residual matrix A 1 and B 1 :

Figure BDA0003593755140000058
Figure BDA0003593755140000058

Figure BDA0003593755140000061
Figure BDA0003593755140000061

Figure BDA0003593755140000062
Figure BDA0003593755140000062

Figure BDA0003593755140000063
Figure BDA0003593755140000063

步骤S34，将残差矩阵A1和B1代替矩阵A和B，重复步骤S31～S33；Step S34, replace the matrices A and B with the residual matrix A 1 and B 1 , and repeat steps S31 to S33;

步骤S35，在重复步骤S31～S33的迭代过程中，根据交叉有效性检验法确定共抽取R个成分(R≤min(n-1,m))对矩阵A和B进行回归：In step S35, in the iterative process of repeating steps S31 to S33, a total of R components (R≤min(n-1,m)) are extracted to perform regression on matrices A and B according to the cross-validity test method:

Figure BDA0003593755140000064
Figure BDA0003593755140000064

再将

Figure BDA0003593755140000065
代入公式(13)，得到矩阵B关于矩阵A的偏最小二乘回归方程：again
Figure BDA0003593755140000065
Substituting into formula (13), the partial least squares regression equation of matrix B with respect to matrix A is obtained:

Figure BDA0003593755140000066
Figure BDA0003593755140000066

式中，I为单位矩阵，r为主成分位置，

Figure BDA0003593755140000067
为第r个成分得分向量，ρr为第r个成分对应的特征向量，
Figure BDA0003593755140000068
代表用矩阵A表示得分向量
Figure BDA0003593755140000069
所需代入的向量；where I is the identity matrix, r is the principal component position,
Figure BDA0003593755140000067
is the score vector of the rth component, ρ r is the feature vector corresponding to the rth component,
Figure BDA0003593755140000068
Represents a vector of scores represented by matrix A
Figure BDA0003593755140000069
The vector to be substituted into;

步骤S36，计算水质参数反演模型的系数W和截距F：Step S36, calculate the coefficient W and intercept F of the water quality parameter inversion model:

Figure BDA00035937551400000610
Figure BDA00035937551400000610

式中，

Figure BDA00035937551400000611
表示水质参数矩阵第k列的均值，
Figure BDA00035937551400000612
表示水质参数矩阵第k列的均方差；In the formula,
Figure BDA00035937551400000611
represents the mean of the kth column of the water quality parameter matrix,
Figure BDA00035937551400000612
Represents the mean square error of the kth column of the water quality parameter matrix;

步骤S37，利用水质参数反演模型反演待测水体水质参数的浓度：Step S37, use the water quality parameter inversion model to invert the concentration of the water quality parameters of the water body to be measured:

Ytest(n,p)＝Xtest(n,m)*W(m,p)+F(p) (16)Y test (n,p)=X test (n,m)*W(m,p)+F(p) (16)

式中，Ytest表示待测水体中水质参数的反演浓度；Xtest为待测水体浊度校正后在紫外-可见光全波段的吸收光谱，p为待测水体中水质参数的种类。In the formula, Y test represents the inversion concentration of water quality parameters in the water body to be measured; X test is the absorption spectrum of the water body to be measured in the full ultraviolet-visible light band after turbidity correction, and p is the type of water quality parameters in the water body to be measured.

在一些实施例中，步骤S21中所述不同水质参数浓度水体包括：实验室配比的标准溶液和地表环境水溶液，其中实验室配比的标准溶液包含单一水质参数标准溶液和多水质参数混合标准溶液，水质参数配比范围参考地表水环境质量标准的各水质参数限值，混合标准溶液包含多种水质参数、多级浓度；标准溶液根据国家标准采用重量-容量法进行配比；地表水选择多种不同类型的水体，覆盖范围涵盖地表Ⅰ—Ⅴ类水以及黑臭水体。In some embodiments, the water bodies with different water quality parameter concentrations in step S21 include: a standard solution formulated in the laboratory and an aqueous solution of the surface environment, wherein the standard solution formulated in the laboratory includes a single water quality parameter standard solution and multiple water quality parameter mixed standards The proportioning range of water quality parameters refers to the limits of various water quality parameters in the surface water environmental quality standard. The mixed standard solution contains a variety of water quality parameters and multi-level concentrations; the standard solution is proportioned by the weight-volume method according to the national standard; surface water selection A variety of different types of water bodies, covering surface I-V water bodies and black and odorous water bodies.

在一些实施例中，所述在紫外-可见光全波段的吸收光谱是指在200-600nm波段的吸收光谱；所述可见光波段是指550nm波段。In some embodiments, the absorption spectrum in the entire ultraviolet-visible light band refers to the absorption spectrum in the 200-600 nm band; the visible light band refers to the 550 nm band.

在一些实施例中，所述吸收光谱的分辨率为1nm。In some embodiments, the absorption spectrum has a resolution of 1 nm.

综上所述，由于采用了上述技术方案，本发明的有益效果是：To sum up, due to the adoption of the above-mentioned technical solutions, the beneficial effects of the present invention are:

本发明通过奇异值分解法对吸收光谱进行了浊度校正，该方法在满足一定数据量的条件下，相对于从原始光谱进行浊度校正的方法，减少了参与浊度校正的波段数量，提高了浊度校正的精度，进一步也提高了其他水质参数的反演精度。并且回归建模时构建了实验室标准样品和环境水样品光谱数据库，可在后续的使用时进行训练样本的补充，扩大模型的反演范围，提高了模型的环境适应性及反演精度。The invention performs turbidity correction on the absorption spectrum through the singular value decomposition method, and the method reduces the number of bands involved in the turbidity correction compared to the method for performing turbidity correction from the original spectrum under the condition that a certain amount of data is satisfied, and improves the The accuracy of turbidity correction is improved, and the inversion accuracy of other water quality parameters is further improved. In addition, a spectral database of laboratory standard samples and environmental water samples is constructed during regression modeling, which can be used to supplement training samples in subsequent use, expand the inversion range of the model, and improve the environmental adaptability and inversion accuracy of the model.

附图说明Description of drawings

为了更清楚地说明本发明实施例的技术方案，下面将对实施例中的附图作简单地介绍，应当理解，以下附图仅示出了本发明的某些实施例，因此不应被看作是对范围的限定，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他相关的附图。In order to explain the technical solutions of the embodiments of the present invention more clearly, the accompanying drawings in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention, and therefore should not be viewed as As a limitation of the scope, for those of ordinary skill in the art, other related drawings can also be obtained according to these drawings without any creative effort.

图1为本发明实施例的光谱水质检测方法的流程图。FIG. 1 is a flowchart of a spectral water quality detection method according to an embodiment of the present invention.

图2为本发明示例的训练数据中部分实验室配比标准溶液以及地表环境水吸收光谱图。FIG. 2 is a partial laboratory proportioning standard solution and an absorption spectrum diagram of surface environmental water in the training data of the example of the present invention.

图3为本发明示例的验证数据输入反演模型得到的各类水质参数的反演结果展示图。FIG. 3 is a display diagram of inversion results of various water quality parameters obtained by inputting verification data into an inversion model according to an example of the present invention.

具体实施方式Detailed ways

为使本发明实施例的目的、技术方案和优点更加清楚，下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本发明一部分实施例，而不是全部的实施例。通常在此处附图中描述和示出的本发明实施例的组件可以以各种不同的配置来布置和设计。In order to make the purposes, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments These are some embodiments of the present invention, but not all embodiments. The components of the embodiments of the invention generally described and illustrated in the drawings herein may be arranged and designed in a variety of different configurations.

因此，以下对在附图中提供的本发明的实施例的详细描述并非旨在限制要求保护的本发明的范围，而是仅仅表示本发明的选定实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。Thus, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the invention as claimed, but is merely representative of selected embodiments of the invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

实施例Example

如图1所示，本实施例提出一种光谱水质检测方法，包括如下步骤：As shown in Figure 1, the present embodiment proposes a spectral water quality detection method, which includes the following steps:

步骤S10，获取待测水体在紫外-可见光全波段的吸收光谱；本实施例中的紫在紫外-可见光全波段的吸收光谱是指在200-600nm波段的吸收光谱，所述吸收光谱的分辨率为1nm；Step S10, acquiring the absorption spectrum of the water body to be tested in the full ultraviolet-visible light band; the absorption spectrum of violet in the ultraviolet-visible light full wavelength band in this embodiment refers to the absorption spectrum in the 200-600 nm wavelength band, and the resolution of the absorption spectrum is is 1nm;

步骤S20，利用可见光波段(550nm波段)反演待测水体的浊度，并基于奇异值分解法对待测水体的浊度进行浊度校正，获得浊度校正后的吸收光谱：Step S20, using the visible light band (550nm band) to invert the turbidity of the water body to be measured, and perform turbidity correction on the turbidity of the water body to be measured based on the singular value decomposition method, to obtain the absorption spectrum after turbidity correction:

步骤S21，获取不同水质参数浓度水体在紫外-可见光全波段的吸收光谱作为训练数据；Step S21, acquiring absorption spectra of water bodies with different water quality parameter concentrations in the entire ultraviolet-visible light band as training data;

所述不同水质参数浓度水体包括：实验室配比的标准溶液和地表环境水溶液，其中实验室配比的标准溶液包含单一水质参数标准溶液和多水质参数混合标准溶液，水质参数配比范围参考地表水环境质量标准的各水质参数限值，混合标准溶液尽量多样，包含多种水质参数、多级浓度；标准溶液根据国家标准采用重量-容量法进行配比；地表水选择多种不同类型的水体，覆盖范围涵盖地表Ⅰ—Ⅴ类水以及黑臭水体。The water bodies with different water quality parameter concentrations include: a standard solution proportioned in the laboratory and an aqueous solution of the surface environment, wherein the standard solution proportioned in the laboratory includes a single water quality parameter standard solution and a multi-water quality parameter mixed standard solution, and the water quality parameter proportioning range refers to the surface. For the water quality parameter limits of the water environment quality standard, the mixed standard solution should be as diverse as possible, including a variety of water quality parameters and multi-level concentrations; the standard solution is proportioned by the weight-volume method according to the national standard; surface water selects a variety of different types of water bodies , the coverage covers surface I-V water and black and odorous water bodies.

步骤S22，利用训练数据的浊度和在可见光波段的吸光度构建一元线性回归模型：Step S22, using the turbidity of the training data and the absorbance in the visible light band to construct a univariate linear regression model:

获取训练数据中各水体在可见光波段的吸光度，并由于浊度与可见光波段的吸光度具有较好的相关性，因此利用各水体的浊度与可见光波段的吸光度构建一元线性回归模型：Obtain the absorbance of each water body in the visible light band in the training data, and since the turbidity has a good correlation with the absorbance of the visible light band, a univariate linear regression model is constructed by using the turbidity of each water body and the absorbance of the visible light band:

Ttrain＝α*Ltrain+β (1)T train =α*L train +β (1)

式中，Ttrain表示训练数据的浊度，Ltrain表示训练数据在550nm的吸光度，α和β代表一元线性回归模型拟合的参数。where T train represents the turbidity of the training data, L train represents the absorbance of the training data at 550 nm, and α and β represent the parameters fitted by the univariate linear regression model.

步骤S23，基于奇异值分解法对训练数据的吸收光谱进行特征提取，保留特征提取后的前三个波段的奇异值：In step S23, feature extraction is performed on the absorption spectrum of the training data based on the singular value decomposition method, and the singular values of the first three bands after feature extraction are retained:

基于奇异值分解法对训练数据的吸收光谱进行特征提取的方法包括：The method for feature extraction of the absorption spectrum of the training data based on the singular value decomposition method includes:

Figure BDA0003593755140000091
Figure BDA0003593755140000091

式中，

Figure BDA0003593755140000092
表示训练数据的吸收光谱矩阵，上标“T”表示矩阵的转置；
Figure BDA0003593755140000093
表示左奇异矩阵；
Figure BDA0003593755140000094
表示奇异值矩阵；
Figure BDA0003593755140000095
表示右奇异矩阵；λ表示特征值矩阵，n表示训练数据的样本数量、m表示训练数据中样本的波段数量；由此保留特征提取后的前三个波段的奇异值
Figure BDA0003593755140000096
作为后续参与浊度校正的数据。In the formula,
Figure BDA0003593755140000092
represents the absorption spectrum matrix of the training data, and the superscript "T" represents the transpose of the matrix;
Figure BDA0003593755140000093
represents a left singular matrix;
Figure BDA0003593755140000094
represents the singular value matrix;
Figure BDA0003593755140000095
represents the right singular matrix; λ represents the eigenvalue matrix, n represents the number of samples in the training data, and m represents the number of bands in the training data; thus, the singular values of the first three bands after feature extraction are retained.
Figure BDA0003593755140000096
As the data for subsequent participation in turbidity correction.

步骤S24，基于特征提取后的前三个波段的奇异值构建浊度校正模型：Step S24, build a turbidity correction model based on the singular values of the first three bands after feature extraction:

从奇异值分解后的训练数据中挑选出混合溶液样本D，以及与之对应的0浊度的混合溶液样本E；由于混合溶液样本D与混合溶液样本E的前三个波段的奇异值差值与浊度具有较强的相关性，因此可以通过公式(3)计算得到浊度在混合溶液样本中各波段的贡献来作为浊度校正模型：The mixed solution sample D and the corresponding mixed solution sample E with 0 turbidity are selected from the training data after singular value decomposition; It has a strong correlation with turbidity, so the contribution of turbidity in each band in the mixed solution sample can be calculated by formula (3) as a turbidity correction model:

(Dj-Ej)＝γj*T+δj, 1≤j≤3 (3)(D j -E j )=γ j *T+δ j , 1≤j≤3 (3)

式中，Dj表示第j个波段的混合溶液样本D，Ej表示第j个波段的0浊度的混合溶液样本E；T表示混合溶液样本D的浊度；γj和δj表示浊度校正模型拟合的参数。In the formula, D j represents the mixed solution sample D in the j-th band, E j represents the mixed solution sample E with 0 turbidity in the j-th band; T represents the turbidity of the mixed solution sample D; γ j and δ j represent the turbidity The parameters of the degree-corrected model fit.

步骤S25，利用一元线性回归模型反演待测水体的浊度，并通过浊度校正模型对待测水体的浊度进行浊度校正，获得浊度校正后的吸收光谱：In step S25, the turbidity of the water body to be measured is inverted using a univariate linear regression model, and the turbidity correction model is used to perform turbidity correction on the turbidity of the water body to be measured to obtain the absorption spectrum after turbidity correction:

步骤S251，利用一元线性回归模型反演待测水体的浊度：Step S251, use the univariate linear regression model to invert the turbidity of the water body to be measured:

Ttest＝α*Ltest+β (4)T test = α*L test + β (4)

式中，Ttest表示待测水体的浊度；Ltest表示待测水体在550nm的吸光度；In the formula, T test represents the turbidity of the water body to be tested; L test represents the absorbance of the water body to be tested at 550 nm;

步骤S252，根据待测水体的浊度并利用浊度校正模型计算奇异值差值：Step S252, according to the turbidity of the water body to be measured and using the turbidity correction model to calculate the singular value difference:

Figure BDA0003593755140000101
Figure BDA0003593755140000101

式中，

Figure BDA0003593755140000102
表示待测水体的奇异值差值；In the formula,
Figure BDA0003593755140000102
Represents the singular value difference of the water body to be measured;

步骤S253，利用待测水体的奇异值波段减去奇异值差值完成浊度校正：Step S253, using the singular value band of the water body to be measured minus the singular value difference to complete the turbidity correction:

Figure BDA0003593755140000103
Figure BDA0003593755140000103

式中，

Figure BDA0003593755140000104
表示完成浊度校正的计算结果，
Figure BDA0003593755140000105
表示待测水体的奇异值波段；In the formula,
Figure BDA0003593755140000104
Indicates the calculation result of the completed turbidity correction,
Figure BDA0003593755140000105
Represents the singular value band of the water body to be measured;

步骤S254，对完成浊度校正的计算结果

Figure BDA0003593755140000106
进行奇异值逆变换，得到浊度校正后的吸收光谱：Step S254, the calculation result of the completed turbidity correction
Figure BDA0003593755140000106
Perform inverse singular value transformation to obtain the absorption spectrum after turbidity correction:

Figure BDA0003593755140000107
Figure BDA0003593755140000107

式中，

Figure BDA0003593755140000111
表示浊度校正后的吸收光谱，
Figure BDA0003593755140000112
表示完成浊度校正的计算结果，Σ3×3表示对应的奇异值矩阵，V3×m表示对应的右奇异矩阵。In the formula,
Figure BDA0003593755140000111
represents the absorption spectrum after turbidity correction,
Figure BDA0003593755140000112
Indicates the calculation result of completing the turbidity correction, Σ 3×3 represents the corresponding singular value matrix, and V 3×m represents the corresponding right singular matrix.

步骤S30，通过利用偏最小二乘回归方法预先训练好的反演模型将浊度校正后的吸收光谱转化为各类水质参数：In step S30, the turbidity-corrected absorption spectrum is converted into various water quality parameters by using the pre-trained inversion model of the partial least squares regression method:

步骤S31，记浊度校正后的吸收光谱矩阵为X，对吸收光谱矩阵X和各水质参数矩阵Y进行标准化，得到标准化后的吸收光谱矩阵A和各水质参数矩阵B，以对吸收光谱矩阵X进行标准化为例：Step S31, denote the absorption spectrum matrix after turbidity correction as X, standardize the absorption spectrum matrix X and each water quality parameter matrix Y, and obtain the standardized absorption spectrum matrix A and each water quality parameter matrix B, so as to measure the absorption spectrum matrix X. For example for normalization:

Figure BDA0003593755140000113
Figure BDA0003593755140000113

Figure BDA0003593755140000114
Figure BDA0003593755140000114

Figure BDA0003593755140000115
Figure BDA0003593755140000115

式中，Aij(n×m)表示标准化后的吸收光谱矩阵A中的第i个样本第j个波段元素；Xij表示吸收光谱矩阵X中的第i个样本第j个波段元素；

Figure BDA0003593755140000116
表示吸收光谱矩阵X中第j列的均值；
Figure BDA0003593755140000117
表示吸收光谱矩阵X中第j列的的均方差；同理得到标准化后的各水质参数矩阵B；In the formula, A ij (n×m) represents the jth band element of the ith sample in the normalized absorption spectrum matrix A; X ij represents the jth band element of the ith sample in the absorption spectrum matrix X;
Figure BDA0003593755140000116
represents the mean value of the jth column in the absorption spectrum matrix X;
Figure BDA0003593755140000117
Represents the mean square error of the jth column in the absorption spectrum matrix X; similarly, the standardized water quality parameter matrix B is obtained;

步骤S32，通过计算最大特征值所对应的特征向量ρ，进而计算标准化后的吸收光谱矩阵A的第一对主成分W1及其得分向量

Figure BDA0003593755140000118
Step S32, by calculating the eigenvector ρ corresponding to the maximum eigenvalue, and then calculating the first pair of principal components W1 of the standardized absorption spectrum matrix A and its score vector
Figure BDA0003593755140000118

Figure BDA0003593755140000119
Figure BDA0003593755140000119

式中，ρ1表示第一对主成分W1对应的特征向量；In the formula, ρ 1 represents the eigenvector corresponding to the first pair of principal components W 1 ;

步骤S33，建立标准化后的吸收光谱矩阵A和各水质参数矩阵B对第一对主成分W1的回归，并且求得回归后的残差矩阵A1和B1：Step S33, establish the regression of the standardized absorption spectrum matrix A and each water quality parameter matrix B on the first pair of principal components W 1 , and obtain the regressed residual matrix A 1 and B 1 :

Figure BDA0003593755140000121
Figure BDA0003593755140000121

步骤S34，将残差矩阵A1和B1代替矩阵A和B，重复步骤S31～S33；Step S34, replace the matrices A and B with the residual matrix A 1 and B 1 , and repeat steps S31 to S33;

步骤S35，在重复步骤S31～S33的迭代过程中，根据交叉有效性检验法确定共抽取R个成分(R≤min(n-1,m))对矩阵A和B进行回归：In step S35, in the iterative process of repeating steps S31 to S33, a total of R components (R≤min(n-1,m)) are extracted to perform regression on matrices A and B according to the cross-validity test method:

Figure BDA0003593755140000122
Figure BDA0003593755140000122

再将

Figure BDA0003593755140000123
代入公式(13)，得到矩阵B关于矩阵A的偏最小二乘回归方程：again
Figure BDA0003593755140000123
Substituting into formula (13), the partial least squares regression equation of matrix B with respect to matrix A is obtained:

Figure BDA0003593755140000124
Figure BDA0003593755140000124

式中，I为单位矩阵，r为主成分位置，

Figure BDA0003593755140000125
为第r个成分得分向量，ρr为第r个成分对应的特征向量，
Figure BDA0003593755140000126
代表用矩阵A表示得分向量
Figure BDA0003593755140000127
所需代入的向量；where I is the identity matrix, r is the principal component position,
Figure BDA0003593755140000125
is the score vector of the rth component, ρ r is the feature vector corresponding to the rth component,
Figure BDA0003593755140000126
Represents a vector of scores represented by matrix A
Figure BDA0003593755140000127
The vector to be substituted into;

步骤S36，计算水质参数反演模型的系数W和截距F：Step S36, calculate the coefficient W and intercept F of the water quality parameter inversion model:

Figure BDA0003593755140000128
Figure BDA0003593755140000128

Figure BDA0003593755140000131
Figure BDA0003593755140000131

式中，

Figure BDA0003593755140000132
表示水质参数矩阵第k列的均值，
Figure BDA0003593755140000133
表示水质参数矩阵第k列的均方差。In the formula,
Figure BDA0003593755140000132
represents the mean of the kth column of the water quality parameter matrix,
Figure BDA0003593755140000133
Represents the mean square error of the kth column of the water quality parameter matrix.

步骤S37，利用水质参数反演模型反演待测水体水质参数的浓度：Step S37, use the water quality parameter inversion model to invert the concentration of the water quality parameters of the water body to be measured:

Ytest(n,p)＝Xtest(n,m)*W(m,p)+F(p) (16)Y test (n,p)=X test (n,m)*W(m,p)+F(p) (16)

式中，Ytest表示待测水体中水质参数的反演浓度；Xtest为待测水体浊度校正后在紫外-可见光全波段的吸收光谱，p为待测水体中水质参数的种类。In the formula, Y test represents the inversion concentration of water quality parameters in the water body to be measured; X test is the absorption spectrum of the water body to be measured in the full ultraviolet-visible light band after turbidity correction, and p is the type of water quality parameters in the water body to be measured.

在一些实施例中，步骤S35中所述交叉有效性检验法如下：In some embodiments, the cross-validity test method described in step S35 is as follows:

(A)、从建模数据中每次舍去第i个数据(i＝1,2,…,n)，对剩下的n-1个数据利用偏最小二乘回归建模，建模过程中选择h(h≤R)个成分进行拟合，然后把舍去的第i个数据代入所拟合的回归方程，得到第i个数据的预测值

Figure BDA0003593755140000134
(A) Discard the i-th data (i=1,2,...,n) from the modeling data each time, and use partial least squares regression to model the remaining n-1 data. The modeling process Select h (h≤R) components for fitting, and then substitute the discarded i-th data into the fitted regression equation to obtain the predicted value of the i-th data
Figure BDA0003593755140000134

(B)、对i＝1,2,…,n重复步骤(A)的验证，根据公式(17)计算选择h个成分时P个参数的预测误差平方和PRESS(h)：(B), repeat the verification of step (A) for i=1,2,...,n, calculate the squared prediction error sum PRESS(h) of P parameters when h components are selected according to formula (17):

Figure BDA0003593755140000135
Figure BDA0003593755140000135

(C)、另外选取所有的数据，选择h个成分进行回归方程的拟合，得到每条数据的预测值

Figure BDA0003593755140000136
同样根据公式(18)计算其误差平方和SS(h)：(C), in addition, select all the data, select h components to fit the regression equation, and obtain the predicted value of each data
Figure BDA0003593755140000136
Also calculate its error sum of squares SS(h) according to formula (18):

Figure BDA0003593755140000137
Figure BDA0003593755140000137

(D)、最后，根据公式(19)定义交叉有效性

Figure BDA0003593755140000141
如果在第h个成分时有
Figure BDA0003593755140000142
(该0.0985可根据需要设定)，则认为模型达到精度要求，可以停止提取成分：(D), finally, define the cross validity according to formula (19)
Figure BDA0003593755140000141
If at the hth component there is
Figure BDA0003593755140000142
(The 0.0985 can be set as needed), then it is considered that the model meets the accuracy requirements, and the extraction of components can be stopped:

Figure BDA0003593755140000143
Figure BDA0003593755140000143

示例：Example:

以浊度、COD、硝酸盐、亚硝酸盐四种水质参数的反演为例，COD标准溶液采用邻苯二甲酸氢钾COD标准溶液，浊度标准溶液采用福尔马肼标准溶液，硝酸盐标准溶液采用硝酸盐氮标准溶液，亚硝酸盐标准溶液采用亚硝酸根标准溶液。Taking the inversion of four water quality parameters, turbidity, COD, nitrate, and nitrite as an example, the COD standard solution uses potassium hydrogen phthalate COD standard solution, the turbidity standard solution uses formazin standard solution, and nitrate standard solution is used. The standard solution adopts nitrate nitrogen standard solution, and the nitrite standard solution adopts nitrite standard solution.

上述的光谱水质检测方法，包括以下步骤：The above-mentioned spectral water quality detection method comprises the following steps:

(1)实验室配比不同浓度的单一水质参数标准溶液和多水质参数混合标准溶液，其中浊度配比浓度范围0～40NTU，COD标准溶液配比范围0～40mg/L，硝酸盐标准溶液配比范围0～10mg/L，亚硝酸盐标准溶液配比范围0～10mg/L。地表环境水参数参考《水质浊度的测定(第一篇分光光度法)》(GB/T 13200-1991)、《水质化学需氧量的测定重铬酸盐法》(HJ828-2017)、《水质无机阴离子的测定离子色谱法》(HJ 84-2016)、《水质亚硝酸盐氮的测定分光光度法》(GB/T 7493-1987)四项国家标准进行测量。所采集的地表环境水浊度范围0～40NTU，COD浓度范围0～70mg/L，硝酸盐浓度范围0～30mg/L，亚硝酸盐浓度范围0～14mg/L。(1) The laboratory mixes different concentrations of a single water quality parameter standard solution and a multi-water quality parameter mixed standard solution, in which the turbidity ratio concentration range is 0~40NTU, the COD standard solution ratio range is 0~40mg/L, and the nitrate standard solution is in the range of 0~40mg/L. The ratio range is 0～10mg/L, and the ratio range of nitrite standard solution is 0～10mg/L. Surface environmental water parameters refer to "Determination of Water Turbidity (Part 1 Spectrophotometric Method)" (GB/T 13200-1991), "Determination of Water Chemical Oxygen Demand by Dichromate Method" (HJ828-2017), "Dichromate Method for Determination of Chemical Oxygen Demand in Water Quality" Measurements were carried out in accordance with four national standards, "Determination of Inorganic Anions in Water Quality by Ion Chromatography" (HJ 84-2016) and "Spectrophotometric Method for the Determination of Nitrite Nitrogen in Water Quality" (GB/T 7493-1987). The collected surface environmental water turbidity ranged from 0 to 40 NTU, COD concentration ranged from 0 to 70 mg/L, nitrate concentration range of 0 to 30 mg/L, and nitrite concentration range of 0 to 14 mg/L.

(2)采集所有溶液在200～600nm范围的吸收光谱，取全部数据的95％作为训练数据，剩下5％作为验证数据，并且验证数据中包含实验室配比样本和地表环境水样本。图2为训练数据中部分实验室配比标准溶液以及地表环境水吸收光谱图。(2) Collect the absorption spectra of all solutions in the range of 200-600 nm, take 95% of all data as training data, and the remaining 5% as verification data, and the verification data includes laboratory proportioning samples and surface environmental water samples. Figure 2 shows some laboratory proportioning standard solutions and the absorption spectrum of surface environmental water in the training data.

(3)利用训练数据在可见光波段的吸光度构建一元线性回归模型；基于奇异值分解法对训练数据的吸收光谱进行特征提取，保留特征提取后的前三个波段的奇异值；基于特征提取后的前三个波段的奇异值构建浊度校正模型；利用一元线性回归模型反演待测水体的浊度，并通过浊度校正模型对待测水体的浊度进行浊度校正，获得浊度校正后的吸收光谱。(3) Use the absorbance of the training data in the visible light band to construct a univariate linear regression model; perform feature extraction on the absorption spectrum of the training data based on the singular value decomposition method, and retain the singular values of the first three bands after feature extraction; The singular values of the first three bands build a turbidity correction model; use the univariate linear regression model to invert the turbidity of the water body to be measured, and perform turbidity correction on the turbidity of the water body to be measured through the turbidity correction model, and obtain the turbidity correction model. absorption spectrum.

(4)对浊度校正后的吸收光谱以及各水质参数浓度进行标准化处理。(4) Standardize the absorption spectrum after turbidity correction and the concentration of each water quality parameter.

(5)计算标准化后的吸收光谱矩阵最大特征值所对应的特征向量ρ，进而计算吸收光谱的第一对主成分U1及其得分向量

Figure BDA0003593755140000151
(5) Calculate the eigenvector ρ corresponding to the maximum eigenvalue of the standardized absorption spectrum matrix, and then calculate the first pair of principal components U1 of the absorption spectrum and its score vector
Figure BDA0003593755140000151

(6)建立标准化处理后的吸收光谱和水质参数对U1的回归，并且求得回归后的残差矩阵。(6) Establish the regression of the standardized absorption spectrum and water quality parameters to U 1 , and obtain the residual matrix after the regression.

(7)将残差矩阵代替原先的吸收光谱和水质参数，重复步骤(4)和步骤(6)。(7) Replace the original absorption spectrum and water quality parameters with the residual matrix, and repeat steps (4) and (6).

(8)在重复步骤(4)和步骤(6)迭代的过程中，通过交叉有效性检验方法，计算选取R个成分(R≤min(n-1,m))时的模型反演误差平方和

Figure BDA0003593755140000152
如果
Figure BDA0003593755140000153
Figure BDA0003593755140000154
则认为模型达到精度要求，可以停止提取成分。(8) In the iterative process of repeating step (4) and step (6), the model inversion error square is calculated when R components (R≤min(n-1,m)) are selected through the cross-validity test method. and
Figure BDA0003593755140000152
if
Figure BDA0003593755140000153
Figure BDA0003593755140000154
It is considered that the model meets the accuracy requirements, and the extraction of components can be stopped.

(9)根据选取的成分个数以及成分与原始数据之间的关系，最终得到浊度、COD、硝酸盐、亚硝酸盐四种水质参数的反演模型：(9) According to the selected number of components and the relationship between the components and the original data, the inversion model of the four water quality parameters of turbidity, COD, nitrate and nitrite is finally obtained:

yk＝w1kx1+…+wmkxm+fk (k＝1,2,…,p) (20)y k = w 1k x 1 +…+w mk x m +f k (k=1,2,…,p) (20)

(10)将步骤(9)得到的反演模型参数应用于验证数据。图3展示的验证数据输入反演模型得到的各类水质参数的反演结果。从反演结果来看，除一些低浓度参数反演会出现负值以外，反演模型对于实验室配比的标准溶液以及地表环境水都具有较好的反演效果。(10) Apply the inversion model parameters obtained in step (9) to the validation data. Figure 3 shows the inversion results of various water quality parameters obtained by inputting the validation data into the inversion model. From the inversion results, except for some low-concentration parameters that may have negative values, the inversion model has a good inversion effect for standard solutions formulated in the laboratory and surface environmental water.

以上所述仅为本发明的优选实施例而已，并不用于限制本发明，对于本领域的技术人员来说，本发明可以有各种更改和变化。凡在本发明的精神和原则之内，所作的任何修改、等同替换、改进等，均应包含在本发明的保护范围之内。The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims (10)

1. A spectral water quality detection method is characterized by comprising the following steps:
step S10, acquiring the absorption spectrum of the water body to be measured in the ultraviolet-visible light full wave band;
step S20, inverting the turbidity of the water body to be detected by using a visible light waveband, and carrying out turbidity correction on the turbidity of the water body to be detected based on a singular value decomposition method to obtain an absorption spectrum after turbidity correction;
and step S30, converting the absorption spectrum after turbidity correction into various water quality parameters by using an inversion model trained in advance by a partial least squares regression method.
2. A spectroscopic water quality detection method as set forth in claim 1 wherein the step S20 includes the substeps of:
step S21, acquiring absorption spectra of water bodies with different water quality parameter concentrations in ultraviolet-visible light full wave bands as training data;
step S22, constructing a unary linear regression model by using the turbidity of the training data and the absorbance in the visible light band;
step S23, extracting the characteristics of the absorption spectrum of the training data based on a singular value decomposition method, and reserving the singular values of the first three bands after the characteristics are extracted;
step S24, constructing a turbidity correction model based on singular values of the first three wave bands after feature extraction;
and step S25, inverting the turbidity of the water body to be detected by using the unitary linear regression model, and carrying out turbidity correction on the turbidity of the water body to be detected by using the turbidity correction model to obtain an absorption spectrum after turbidity correction.
3. The method for detecting water quality according to claim 2, wherein the step S22 of constructing the unary linear regression model using the turbidity of the training data and the absorbance in the visible light band comprises:
and (3) acquiring the absorbance of each water body in the training data in a visible light wave band, and constructing a unitary linear regression model by using the turbidity of each water body and the absorbance of the visible light wave band:
Ttrain＝α*Ltrain+β (1)
in the formula, TtrainTurbidity, L, representing training datatrainRepresenting the absorbance of the training data at 550nm, α and β represent parameters of a one-dimensional linear regression model fit.
4. The method for detecting spectral water quality according to claim 3, wherein the method for extracting the characteristic of the absorption spectrum of the training data based on the singular value decomposition method in step S23 includes:
Figure FDA0003593755130000021
in the formula (I), the compound is shown in the specification,
Figure FDA0003593755130000022
representing an absorption of training dataA spectrum matrix, wherein the superscript T represents the transposition of the matrix;
Figure FDA0003593755130000023
representing a left singular matrix;
Figure FDA0003593755130000024
representing a matrix of singular values;
Figure FDA0003593755130000025
representing a right singular matrix; lambda represents an eigenvalue matrix, n represents the number of samples of the training data, and m represents the number of wave bands of the samples in the training data; thereby preserving singular values of the first three bands after feature extraction
Figure FDA0003593755130000026
As data subsequently participating in turbidity correction.
5. The spectral water quality detection method according to claim 4, wherein the method for constructing the turbidity correction model based on the first three band singular values after feature extraction in step S24 comprises:
selecting a mixed solution sample D and a mixed solution sample E with 0 turbidity corresponding to the mixed solution sample D from training data after singular value decomposition, and calculating the contribution of the turbidity in each waveband in the mixed solution sample through a formula (3) to be used as a turbidity correction model:
(Dj-Ej)＝γj*T+δj,1≤j≤3 (3)
in the formula, DjMixed solution samples D, E representing the jth wave bandjA mixed solution sample E showing 0 turbidity of the jth band; t represents the turbidity of the mixed solution sample D; gamma rayjAnd deltajParameters representing the turbidity correction model fit.
6. The spectral water quality detection method according to claim 5, wherein in step S25, the method for obtaining the absorption spectrum after turbidity correction by inverting the turbidity of the water body to be detected by using the unitary linear regression model and performing turbidity correction on the turbidity of the water body to be detected by using the turbidity correction model comprises:
step S251, inverting the turbidity of the water body to be measured by using a unitary linear regression model:
Ttest＝α*Ltest+β (4)
in the formula, TtestRepresenting the turbidity of the water body to be detected; l istestRepresenting the absorbance of the water body to be detected at 550 nm;
step S252, calculating a singular value difference value by using a turbidity correction model according to the turbidity of the water body to be detected:
Figure FDA0003593755130000031
in the formula (I), the compound is shown in the specification,
Figure FDA0003593755130000032
representing the singular value difference of the water body to be detected;
step S253, subtracting the singular value difference value from the singular value wave band of the water body to be detected to finish turbidity correction:
Figure FDA0003593755130000033
in the formula (I), the compound is shown in the specification,
Figure FDA0003593755130000034
indicating the result of the calculation to complete the turbidity correction,
Figure FDA0003593755130000035
representing a singular value wave band of the water body to be detected;
step S254, calculating the result of the turbidity correction
Figure FDA0003593755130000036
To carry outAnd (3) performing inverse transformation on the singular value to obtain an absorption spectrum after turbidity correction:
Figure FDA0003593755130000037
in the formula (I), the compound is shown in the specification,
Figure FDA0003593755130000038
represents the absorption spectrum after the turbidity correction,
Figure FDA0003593755130000039
indicating the result of the calculation for completing the turbidity correction, sigma3×3Representing corresponding singular value matrices, V3×mRepresenting the corresponding right singular matrix.
7. The spectral water quality detection method according to claim 6, wherein the method for converting the turbidity-corrected absorption spectrum into various water quality parameters by using the inversion model pre-trained by the partial least squares regression method in step S30 comprises:
step S31, recording the absorption spectrum matrix after turbidity correction as X, standardizing the absorption spectrum matrix X and each water quality parameter matrix Y to obtain a standardized absorption spectrum matrix a and each water quality parameter matrix B, taking standardizing the absorption spectrum matrix X as an example:
Figure FDA00035937551300000310
Figure FDA00035937551300000311
Figure FDA0003593755130000041
in the formula, Aij(n × m) represents the ith sample jth band element in the normalized absorption spectra matrix a; xijRepresents the jth wave band element of the ith sample in the absorption spectrum matrix X;
Figure FDA0003593755130000042
represents the mean value of the jth column in the absorption spectrum matrix X;
Figure FDA0003593755130000043
represents the mean square error of the jth column in the absorption spectrum matrix X; obtaining a standardized water quality parameter matrix B in the same way;
step S32 is performed to calculate the eigenvector ρ corresponding to the maximum eigenvalue, and further calculate the first pair of principal components W of the normalized absorption spectrum matrix a1And its score vector
Figure FDA0003593755130000044
Figure FDA0003593755130000045
In the formula, ρ1Represents a first pair of principal components W1A corresponding feature vector;
step S33, establishing a first pair of principal components W of the absorption spectrum matrix A and each water quality parameter matrix B after standardization1And obtaining a residual matrix A after regression1And B1：
Figure FDA0003593755130000046
Step S34, residual error matrix A1And B1Repeating steps S31-S33 instead of matrices A and B;
step S35, in the iterative process of repeating the steps S31-S33, determining to extract R components (R is less than or equal to min (n-1, m)) in total according to a cross validity check method, and regressing the matrixes A and B:
Figure FDA0003593755130000047
then will be
Figure FDA0003593755130000048
Substituting into formula (13) to obtain a partial least squares regression equation for matrix B with respect to matrix a:
Figure FDA0003593755130000051
wherein I is an identity matrix, r is a principal component position,
Figure FDA0003593755130000052
for the r component score vector, prIs the feature vector corresponding to the r-th component,
Figure FDA0003593755130000053
representing score vector by matrix A
Figure FDA0003593755130000054
The vector to be substituted;
step S36, calculating the coefficient W and intercept F of the water quality parameter inversion model:
Figure FDA0003593755130000055
in the formula (I), the compound is shown in the specification,
Figure FDA0003593755130000056
represents the mean value of the kth column of the water quality parameter matrix,
Figure FDA0003593755130000057
k column of matrix representing water quality parametersThe mean square error of (d);
step S37, inverting the concentration of the water quality parameter of the water body to be measured by using the water quality parameter inversion model:
Ytest(n,p)＝Xtest(n,m)*W(m,p)+F(p) (16)
in the formula, YtestExpressing the inversion concentration of the water quality parameter in the water body to be detected; xtestThe water quality parameter is the absorption spectrum of the water body to be detected in the ultraviolet-visible light full wave band after the turbidity correction, and p is the type of the water quality parameter in the water body to be detected.
8. The spectral water quality detection method according to claim 2, wherein the water bodies with different water quality parameter concentrations in step S21 comprise: the water quality parameter matching range refers to each water quality parameter limit value of the surface water environment quality standard, and the mixed standard solution comprises multiple water quality parameters and multilevel concentration; the standard solution is prepared according to the national standard by adopting a weight-volume method; the surface water is selected from various water bodies of different types, and the coverage range covers I-V type water on the surface and black and odorous water bodies.
9. The method for detecting the water quality by the spectrum of claim 1, wherein the absorption spectrum in the whole band of the ultraviolet-visible light is the absorption spectrum in the 200-600nm band; the visible light wave band refers to a 550nm wave band.
10. The spectroscopic water quality measuring method according to claim 1, wherein the resolution of the absorption spectrum is 1 nm.
CN202210383168.XA 2022-04-13 2022-04-13 Spectrum water quality detection method Pending CN114646606A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210383168.XA CN114646606A (en) 2022-04-13 2022-04-13 Spectrum water quality detection method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210383168.XA CN114646606A (en) 2022-04-13 2022-04-13 Spectrum water quality detection method

Publications (1)

Publication Number Publication Date
CN114646606A true CN114646606A (en) 2022-06-21

Family

ID=81997516

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210383168.XA Pending CN114646606A (en) 2022-04-13 2022-04-13 Spectrum water quality detection method

Country Status (1)

Country Link
CN (1) CN114646606A (en)

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115165770A (en) * 2022-07-05 2022-10-11 华南理工大学 Water body COD (chemical oxygen demand) and turbidity simultaneous detection method based on broad spectrum and BPNN (BPNN)
CN116202975A (en) * 2022-12-15 2023-06-02 中国科学院西安光学精密机械研究所 Water parameter prediction method, storage medium and terminal equipment
CN117740748A (en) * 2023-12-20 2024-03-22 湖南省计量检测研究院 BOD online rapid detection method and device
CN117805046A (en) * 2024-02-28 2024-04-02 三亚海慧海洋科技有限公司 Method and device for detecting chemical oxygen demand based on turbidity compensation
CN118362525A (en) * 2024-03-28 2024-07-19 莱森光学（深圳）有限公司 Water quality monitoring method and device based on ultraviolet-visible spectroscopy
CN118483179A (en) * 2024-07-12 2024-08-13 奥谱天成（厦门）光电有限公司 Water quality testing methods, devices, media and equipment
CN119044095A (en) * 2024-10-31 2024-11-29 泉州装备制造研究所 Turbidity compensation method, device and terminal
CN119354898A (en) * 2024-12-24 2025-01-24 成都理工大学 A water quality monitoring method and system based on multi-source data fusion
CN119438109A (en) * 2025-01-13 2025-02-14 浙江大学 A water quality multi-parameter detection method and system based on ultraviolet-visible light spectroscopy

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101865828A (en) * 2010-05-31 2010-10-20 湖南大学 Method for maintaining the predictive power of spectral correction models for complex systems
CN102435556A (en) * 2011-09-20 2012-05-02 湖南大学 Accurate spectrum quantitative analysis method used for complex heterogeneous mixture system
CN111157476A (en) * 2020-02-21 2020-05-15 中国科学院合肥物质科学研究院 Quantitative inversion method for water quality multi-parameter ultraviolet-visible absorption spectrum

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101865828A (en) * 2010-05-31 2010-10-20 湖南大学 Method for maintaining the predictive power of spectral correction models for complex systems
CN102435556A (en) * 2011-09-20 2012-05-02 湖南大学 Accurate spectrum quantitative analysis method used for complex heterogeneous mixture system
CN111157476A (en) * 2020-02-21 2020-05-15 中国科学院合肥物质科学研究院 Quantitative inversion method for water quality multi-parameter ultraviolet-visible absorption spectrum

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
薛海峰: "基于紫外-可见光谱的水质COD检测方法研究", 《中国优秀硕士学位论文电子期刊》, no. 4, 15 April 2020 (2020-04-15), pages 43 - 44 *
陈红艳 等: "去除水分影响提高土壤有机质含量高光谱估测精度", 《农业工程学报》, vol. 30, no. 8, 30 April 2014 (2014-04-30), pages 91 - 92 *

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115165770A (en) * 2022-07-05 2022-10-11 华南理工大学 Water body COD (chemical oxygen demand) and turbidity simultaneous detection method based on broad spectrum and BPNN (BPNN)
CN115165770B (en) * 2022-07-05 2024-03-19 华南理工大学 Simultaneous detection method of water COD and turbidity based on wide spectrum and BPNN
CN116202975A (en) * 2022-12-15 2023-06-02 中国科学院西安光学精密机械研究所 Water parameter prediction method, storage medium and terminal equipment
CN117740748A (en) * 2023-12-20 2024-03-22 湖南省计量检测研究院 BOD online rapid detection method and device
CN117805046A (en) * 2024-02-28 2024-04-02 三亚海慧海洋科技有限公司 Method and device for detecting chemical oxygen demand based on turbidity compensation
CN117805046B (en) * 2024-02-28 2024-06-04 三亚海慧海洋科技有限公司 Method and device for detecting chemical oxygen demand based on turbidity compensation
CN118362525A (en) * 2024-03-28 2024-07-19 莱森光学（深圳）有限公司 Water quality monitoring method and device based on ultraviolet-visible spectroscopy
CN118483179A (en) * 2024-07-12 2024-08-13 奥谱天成（厦门）光电有限公司 Water quality testing methods, devices, media and equipment
CN119044095A (en) * 2024-10-31 2024-11-29 泉州装备制造研究所 Turbidity compensation method, device and terminal
CN119354898A (en) * 2024-12-24 2025-01-24 成都理工大学 A water quality monitoring method and system based on multi-source data fusion
CN119438109A (en) * 2025-01-13 2025-02-14 浙江大学 A water quality multi-parameter detection method and system based on ultraviolet-visible light spectroscopy

Similar Documents

Publication Publication Date Title
CN114646606A (en) Spectrum water quality detection method
US11561169B2 (en) Method for constructing water quality index prediction models and method for monitoring water quality indexes
Meloun et al. Critical comparison of methods predicting the number of components in spectroscopic data
CN104034684B (en) A kind of water quality multi-index detection method based on uv-visible absorption spectra
CN105352895B (en) High-spectrum remote sensing data vegetation information extraction method
CN104949936A (en) Sample component determination method based on optimizing partial least squares regression model
CN116202975B (en) A water body parameter prediction method, storage medium and terminal device
CN105486655A (en) Rapid detection method for organic matters in soil based on infrared spectroscopic intelligent identification model
CN105158200B (en) A kind of modeling method for improving the Qualitative Analysis of Near Infrared Spectroscopy degree of accuracy
CN104062259B (en) A kind of use the method for total saponin content near infrared spectrum quick test complex prescription glue mucilage
CN111563436B (en) Infrared spectrum measuring instrument calibration migration method based on CT-CDD
CN107219188A (en) A kind of method based on the near-infrared spectrum analysis textile cotton content for improving DBN
CN103258330B (en) A Method for Estimating Endmember Abundance in Hyperspectral Images
CN110987846B (en) A Nitrate Concentration Prediction Method Based on iPLS-PA Algorithm
CN110567937A (en) A Competitive Adaptive Reweighted Key Data Extraction Method for Raman Spectroscopy Analysis of Insulating Oil
CN104792752A (en) Method for determining content of pigments in mixed pigment solutions by adopting three-dimensional fluorescence spectroscopy combined with PARAFAC (parallel factor analysis) algorithm
CN105954252A (en) Qualitative detection method for illegal ingredient Sudan red in raw materials of feeds
CN115236045A (en) Method and device for determining chemical oxygen demand in water by using fluorescence spectrometry and storage medium
CN108627486A (en) A method of measuring the active principle and chemical composition content of Chinese medicine
CN105823751B (en) Infrared spectrum Multivariate Correction regression modeling method based on λ-SPXY algorithms
CN117110211A (en) A method for non-destructive detection of crocin I in gardenia based on hyperspectral spectrum information fusion
CN101231270B (en) Method for determining index composition content of Qingkailing injection intermediate body and finished product
CN117607077A (en) Edge calculation-based method for simultaneously monitoring chemical oxygen demand and nitrate nitrogen
CN106198433A (en) Infrared spectrum method for qualitative analysis based on LM GA algorithm
CN114756823A (en) Method for improving pepper spectrum model prediction capability

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination