CN118133435A - Complex spacecraft on-orbit anomaly detection method based on SVR and clustering - Google Patents

Abstract

The present invention discloses a complex spacecraft on-orbit anomaly detection method based on SVR and clustering, including using Granger causality modeling to explore the causal relationship in telemetry data, and then using the SVR model to train the data processed by the principal component analysis method according to the obtained relationship chart to obtain the error between the predicted value and the original data, clustering the errors using a clustering method, and calculating the center point and standard deviation of each cluster. Based on the clustering results, the threshold of the outlier is adaptively selected using statistics such as skewness, kurtosis, and quantiles to further optimize the effect of anomaly detection. The present invention avoids the subjectivity and uncertainty of manually setting the threshold, has better effects when applied across fields, and is suitable for a variety of data types and application scenarios.

Description

Complex spacecraft on-orbit anomaly detection method based on SVR and clustering

Technical Field

The present invention relates to the technical field of reliability engineering, and in particular to an on-orbit anomaly detection method for complex spacecraft based on SVR and clustering.

Background technique

With the increase in the number of spacecraft in orbit and the diversification of their types, spacecraft, as a complex system engineering, integrates many high-performance software and hardware products. Due to the influence of various factors, such as insufficient verification, insufficient production and processing technology, and cumulative damage effects, spacecraft are likely to encounter various abnormal events during their operation, such as sensor failure, battery failure, and communication failure.

Traditional anomaly detection methods have some limitations, such as strong subjectivity and inability to adapt to different data distributions. Therefore, it is necessary to study more efficient and accurate data anomaly detection methods. This method can handle complex abnormal events and overcome the limitations of traditional methods to improve the efficiency and accuracy of anomaly detection.

Data-driven anomaly detection methods have become the focus of researchers because of their advantages such as adaptability, no need for prior knowledge and full utilization of data information.

Therefore, how to provide a data-driven complex spacecraft on-orbit anomaly detection method based on a support vector regression model to achieve adaptive detection of outliers through processing and analysis of telemetry data is a technical problem that needs to be urgently solved by technical personnel in this field.

Summary of the invention

In view of the above research status and existing problems, the present invention provides a complex spacecraft on-orbit anomaly detection method based on SVR and clustering, and predicts the general manifestation of telemetry data based on Granger causality and support vector regression (SVR) model, such as stable form, periodic oscillation, normal peak, step change, gradual change, etc. In combination with clustering method, adaptive detection of outliers is realized.

The present invention provides a complex spacecraft on-orbit anomaly detection method based on SVR and clustering, which comprises the following steps:

S1: Use Granger analysis to analyze the causal relationship between the original spacecraft telemetry data and establish the correlation between different dimensions of the original spacecraft telemetry data;

S2: Use PCA method to reduce the dimension of the original spacecraft telemetry data with causal relationship;

S3: using the SVR algorithm to establish a prediction model, and applying the dimension-reduced, causally related original spacecraft telemetry data to train the prediction model to obtain a pre-trained prediction model;

S4: Use pre-trained prediction models to make predictions on real-time spacecraft telemetry data;

S5: Use a clustering algorithm to divide the predicted data points obtained in S4 into different clusters, and identify outliers based on the distance between clusters.

Preferably, before S1, the step further includes:

The data set is pre-processed to convert the original spacecraft telemetry data into a form suitable for Granger causality analysis. The pre-processing includes: centering, unit root test, stabilization and solving the maximum delay.

Preferably, the S1 comprises:

Two stationary spacecraft telemetry raw time series , Granger causality test estimates the regression:

in, is the length of the time series, /> is the variable value at the current time point, /> is the value of the variable at a past time point, 、/> 、/> 、/> is the regression coefficient, /> 、/> For/> 、/> White noise contained in it;

If a past value of a spacecraft telemetry raw time series has a Granger causal relationship with a current value, the past value is added to the prediction model of the current value for training the prediction model.

Preferably, S3 includes: SVR maps the input data to a high-dimensional space, finds a hyperplane in the space to fit the data, and predicts the value of the output variable by learning a linear or nonlinear regression function.

Preferably, S3 includes:

Solve the following minimization problem:

in, are the original time series data of two stationary spacecraft telemetry, For/> dimensional weight vector,/> is the bias term, /> Indicates the first/> The slack variable for samples, /> and/> is a hyperparameter, /> is the kernel function.

Preferably, a cross-validation method is used to select the optimal hyperparameters, and the model is retrained using the optimal hyperparameters.

Compared with the prior art, it has the following beneficial effects:

The core problem studied in this paper is the anomaly detection problem based on CMG telemetry data. This type of problem has the characteristics of high dimensionality, correlation of unknown variables and noise interference. To solve this problem, this paper proposes a complete anomaly detection process based on Granger analysis, PCA, SVR and clustering methods, providing a comprehensive solution to the anomaly detection problem.

The present invention has adaptability and generalization ability: the cross-validation method is used to select the optimal hyperparameters, so that the anomaly detection model has good adaptability and generalization ability. The model can adapt to different data sets and scenarios and has good prediction ability.

The present invention avoids the subjectivity and uncertainty of manually setting thresholds, has better effects when applied across fields, and is applicable to a variety of data types and application scenarios.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on the provided drawings without creative work.

FIG1 is a flow chart of a complex spacecraft on-orbit anomaly detection method based on SVR and clustering provided by an embodiment of the present invention;

FIG2 is a schematic diagram of Granger causality analysis results of a segment of telemetry data provided by an embodiment of the present invention;

FIG3 is a schematic diagram of a principal component result after PCA processing provided by an embodiment of the present invention;

FIG4 is a schematic diagram of prediction results of four-dimensional data provided by an embodiment of the present invention;

FIG5 is a schematic diagram of a result of anomaly detection of telemetry data provided by an embodiment of the present invention;

FIG. 6 is a schematic diagram of a clustering algorithm provided by an embodiment of the present invention.

Detailed ways

The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

First, the application limitations of the SVR algorithm are explained:

SVR is a powerful regression method that can be used to predict the manifestation of abnormal data. However, its performance is affected by the choice of kernel function and regularization parameters. Choosing appropriate parameters requires certain experience and adjustment. The computational complexity is high when processing large-scale data sets, and the memory and computing resources are large. In addition, SVR is sensitive to noise and outliers, which may cause the shift of classification boundaries and misclassification.

In order to better play the advantages of SVR and overcome its weaknesses, it is necessary to pre-process the data before training, and after obtaining the training results, use a more accurate and efficient method than the threshold method to classify and detect anomalies of the results, so as to form a complete anomaly detection process.

The present invention provides a complex spacecraft on-orbit anomaly detection method based on SVR and clustering, as shown in Figure 1. The present invention combines Granger causality analysis and PCA to pre-process the data set, and uses a clustering method after SVR to cluster the errors to achieve the goal of anomaly detection. It includes the following steps:

S1: Use Granger analysis to analyze the causal relationship between the original spacecraft telemetry data and establish the correlation between different dimensions of the original spacecraft telemetry data.

In one embodiment, before executing S1, the process also includes: performing pre-processing on the data set, including centering, unit root testing, stationarization, determining maximum lag, converting the original data into a form suitable for Granger causality analysis, and performing Granger analysis to establish the relationship between different dimensions of the telemetry data.

In one embodiment, the specific execution process of S1 is as follows:

Suppose there are two stationary spacecraft telemetry raw time series , Granger causality test estimates the regression:

in, is the length of the time series, /> is the variable value at the current time point, /> is the value of the variable at a past time point, 、/> 、/> 、/> is the regression coefficient, /> 、/> For/> 、/> If the past value of a variable can provide the ability to predict the current value, then it can be considered that the former has Granger causality to the latter. At this time, adding past values to the prediction model can significantly improve the prediction accuracy.

It should be noted that in S1, the null hypothesis of Granger causality means that the past value of a variable does not provide additional information for predicting the current value of another variable, that is, the past value of the variable is not a causal factor of the other variable.

For example, if you need to test the variable For variables /> Granger causality, the null hypothesis is:

: Variables/> Past values of the predictor variables/> The current value of provides no additional information, as shown in the following formula:

: Variables/> Past values of the predictor variables/> The current value of provides no additional information, as shown in the following formula:

For example, if we reject the null hypothesis , which means that the variable /> Past values of the predictor variables/> The current value of provides meaningful additional information, namely the existence of Granger causality. If the null hypothesis cannot be rejected/> , then the variable Past values of the predictor variables/> The current value of provides no additional information, i.e., there is no Granger causality.

For example, a Granger causality analysis is performed on a data set of a single-frame CMG system of a spacecraft from October 1, 2011 to March 30, 2015. The data set contains information as shown in Table 1, and the results are shown in Figure 1.

Table 1 Dataset information table

It is worth noting that in the obtained causal relationship indicator table, there are ,/> Position indicator/> ,and ,/> Position indicator/> There is no quantitative correspondence. This is because Granger causality is inferred based on time series data, which analyzes the order and delay of the causal relationship between variables in time. Therefore, the strength and direction of the causal relationship may produce different results in different locations.

This situation is normal in Granger causality analysis, because the existence and strength of causality may be affected by multiple factors, including the characteristics of the data, sampling frequency, time delay, etc. Therefore, such differences in the causality indicator table are reasonable and can provide more comprehensive information about the causal relationship between different variables.

From Figure 2, we can see that the causal relationship between the data in each dimension in the data set “2000_CMG_lite.csv” is shown in Table 2.

Table 2 Causal relationship table between data in each dimension

It is observed that, for example, the CMGx frame angular velocity instruction is the cause and the data sampling time is the result. This situation is caused by the following reasons:

(1) Data sampling rate problem: Granger causality analysis requires a high data sampling rate. If the sampling rate of the angular velocity command is high, while the sampling rate of the time series is low, the angular velocity command may have a higher explanatory power in the time series, and thus be mistaken for a causal factor.

(2) Model selection problem: Granger causality analysis is a method based on autoregressive models, which assumes that the causal relationship of time series can predict future values through past values.

(3) Data characteristic problem: The angular velocity command and the time series have a certain correlation or common trend, which makes the angular velocity command be mistakenly regarded as a causal factor in the Granger causality analysis.

These are due to the characteristics of the data itself or domain knowledge, and can be considered normal for the purpose of pursuing the generalization ability of the detection method.

S2: Use the PCA method to reduce the dimensionality of the raw spacecraft telemetry data with causal relationships.

In the specific implementation, the principal component analysis method is used to map high-dimensional data to low-dimensional space. Each newly obtained component is a linear combination of the original variables, which are independent of each other and retain most of the information of the original variables. Its essence is to seek alternative objects for related variables through the correlation of the original variables and ensure that the information loss of the transformation is minimal.

In one embodiment, assuming that samples, each with /> features, these samples are dimensional space is mapped to/> Dimensional space/> , so that the mapped data retains the information of the original data as much as possible. That is, find a new set of coordinate axes so that the projection variance of the data on this set of new coordinate axes is as large as possible, so as to achieve the purpose of retaining the original data information.

Process all sample data of each feature, subtract the mean value of the feature, ensure that the mean value of each feature is 0, and get a matrix :

Normalize the matrix X and calculate the variance of each feature :

use

Standardize the data to get the standardized matrix :

Calculate the correlation coefficient :

Get the correlation coefficient matrix:

in For/> Matrix No./> The sample sequence of column and the /> The correlation between the sample sequences of the columns is , the correlation between the two sample sequences can be determined by the size of the correlation coefficient:

in, It is called positive linear correlation.

Covariance matrix is a real symmetric matrix, its eigenvalue is non-negative, let its eigenvalue/> , their corresponding orthogonalized unit eigenvectors are:

original The index variable composite vector represented by each column of is denoted as/> , then The first/> The principal components are .

No. The proportion of the variance explained by the first principal component to the total variance, that is, the The contribution rate of the principal components , and before/> The cumulative proportion of the variance explained by the principal components to the total variance, that is, the cumulative contribution rate/> It is derived from the following formula:

in, It is the first/> The eigenvalues of the principal components, is the total number of principal components. Before selection/> The eigenvectors corresponding to the largest eigenvalues form a / > The matrix of , called the principal component matrix. Sample data/> The projection onto the principal component matrix is/> , and get the data after dimensionality reduction. The number of principal components is determined by the cumulative contribution rate, which is generally required to reach more than 85% to ensure that the new variable can include most of the information of the original variable.

Exemplarily, the components with causal relationships obtained through the previous step of analysis are retained for PCA processing, and the principal components obtained are shown in FIG3 .

The processed data is obtained by projecting the original data onto the principal components through linear transformation. Each principal component represents a feature or change direction in the original data. Therefore, the data after PCA can reflect part of the information of the original data, but cannot completely restore the original data.

It is observed that the data features are concentrated in the first principal component. This is because the first principal component can usually capture the most significant features and changes in the data and can well explain the variance of the original data. The remaining principal components contribute less to the changes in the data, so their numerical variation range will be relatively small.

By performing an inverse transformation on the data after PCA, the original data can be approximately restored. However, due to the information loss and dimensionality reduction operation in the PCA process, the data obtained by the inverse transformation is only an approximation of the original data and cannot completely and accurately restore the original data. In general, it is impossible to completely restore the original data through the data after PCA. If you need to use the original data, you can back up and save the original data before performing PCA so that you can use the original data for analysis and processing when needed.

S3: Use the SVR algorithm to establish a prediction model, and use the spacecraft telemetry raw data with causal relationship after dimension reduction to train the prediction model to obtain a pre-trained prediction model. Take a part of the data set as the training set for this stage, use SVR to fit the complex nonlinear relationship between data points, and use it to predict and determine abnormal points. Take the first 6000 groups of data in the data set as the training set for this stage.

In one embodiment, the model can be expressed as:

in, is the independent variable, /> is the kernel function, /> is the regression coefficient, /> is the bias term.

In one embodiment, the following minimization problem is solved:

in, Indicates the first/> The slack variable for samples, /> is the regularization parameter, /> is the exponential of the loss function. The formula for solving the minimization problem is:

in, and/> is the Lagrange multiplier,/> and/> Respectively represent the The input and output of samples. The choice of kernel function, regularization parameter /> Exponential with loss function/> All need to be tuned to improve the performance of the model.

In one embodiment, cross-validation is used to select the optimal hyperparameters. By dividing the data set into a training set and a validation set, cross-validation can evaluate the generalization ability of the model and select the optimal hyperparameter configuration to achieve the best anomaly detection effect. The model is retrained using the optimal hyperparameters.

During the specific implementation, for each hyperparameter combination, the training set is divided into subsets, each time selecting one of them/> The subset is used as the training set, and the remaining subset is used as the validation set to evaluate the performance of the combination. The average of the results of the verification is used as the performance evaluation indicator of the combination. This process is called /> Fold cross validation.

Execute on each hyperparameter combination Fold cross validation is performed to obtain an average value of a performance indicator, and the hyperparameter combination with the best performance indicator is selected as the optimal hyperparameter combination of the model. This process is called grid search. Finally, the model is trained using the training set and the optimal hyperparameter combination, and the performance of the model is evaluated using the test set.

Exemplarily, SVR is used to fit complex nonlinear relationships between data points and is used to predict and determine outliers. SVR can fit complex nonlinear relationships between data points and predict outliers based on these relationships. By establishing a regression model for outliers, new data points can be predicted. In order to obtain the best SVR model performance, the first 6000 sets of data are taken for training, and the cross-validation method is used to select the optimal hyperparameters. By dividing the data set into a training set and a validation set, cross-validation can evaluate the generalization ability of the model and select the optimal hyperparameter configuration to achieve the best anomaly detection effect. Retrain the SVR model to predict the data set.

The model was retrained using the optimal hyperparameters, and the first 300 sets of data in the data set were initially predicted and the error values were calculated to preliminarily evaluate the model performance. They are frame angle, rotor speed, rotor motor current, and frame motor current data, as shown in Figure 4.

It is observed that in the first 300 data samples, the difference between the prediction results using SVR and the true value is very small, and the maximum error is controlled within 0.1.

S4: Use pre-trained prediction models to predict real-time spacecraft telemetry data. SVR maps input data to a high-dimensional space and finds a hyperplane in that space to fit the data, thereby transforming a nonlinear problem into a linear problem and predicting the value of the output variable by learning a linear or nonlinear regression function. SVR minimizes the error between the predicted value and the true value to solve the continuous variable prediction problem.

For example, taking the CMG frame motor current data as an example, continue to read subsequent data for prediction, simulating the telemetry data continuously generated in the actual situation. After obtaining a set of continuously generated prediction values, compare them with the actual values and proceed to the subsequent steps.

S5: Use a clustering algorithm to divide the predicted data points obtained in S4 into different clusters, and identify outliers based on the distance between clusters.

In one embodiment, in S5, a threshold determination is performed every certain amount of data to reduce the amount of computation and improve the efficiency of the algorithm, capture mutations or abnormal behaviors in the data, and be able to adapt to the dynamics of the data, improve the accuracy and reliability of detection, facilitate visualization and analysis, and observe clustering patterns and characteristics.

The data points are divided into K clusters, where the distance between the data points of each cluster and the data points of other clusters is minimized, and then the points with the farthest distance from the data points of other clusters are regarded as outliers, as shown in Figure 6.

For example, after using the SVR model to predict the true value, anomaly detection is performed using a clustering-based algorithm. The clustering algorithm divides the data points into different clusters and identifies the points that are far away from the data points of other clusters as outliers. The result is shown in Figure 5.

The circled points in the figure are points judged as abnormal by the model. At this time, the overall data is basically stable, and the threshold value summarized by the clustering algorithm is small.

The present invention is designed to determine the threshold value every 300 sets of data, with the following considerations:

(1) Reduce the amount of calculation: Clustering algorithms usually require iterative calculations on data, and their computational complexity is high. Dividing the data into smaller subsets and performing cluster analysis on each subset can reduce the overall amount of calculation and improve the efficiency of the algorithm.

(2) Detecting change points: Dividing data into smaller subsets for cluster analysis can make it easier to detect change points in the data. By performing cluster analysis at a certain interval of a certain number of data points, mutations or abnormal behaviors in the data can be captured. This helps to detect possible faults or abnormalities in a timely manner.

(3) Adapting to the dynamic nature of data: Data usually changes dynamically in practical applications and may have different working conditions or environmental conditions. Regular cluster analysis can update the threshold according to the changes in data, so that it can adapt to the dynamic nature of data and improve the accuracy and reliability of detection.

(4) Analyze clustering results: Dividing the data into smaller subsets for cluster analysis can help better understand the clustering results. Smaller data sets are easier to visualize and analyze, and the patterns and characteristics of clusters can be observed more clearly. This helps to deeply understand the structure and behavior of the data and extract useful information from it.

The above is a detailed introduction to the complex spacecraft on-orbit anomaly detection method based on SVR and clustering provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for general technical personnel in this field, according to the idea of the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

In this article, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

Claims (6)

1. A complex spacecraft on-orbit anomaly detection method based on SVR and clustering is characterized by comprising the following steps: the method comprises the following steps:

S1: analyzing causal relations among the spacecraft telemetry original data by using a Granger analysis method, and establishing association relations among different dimensionalities of the spacecraft telemetry original data;

s2: performing dimension reduction processing on the spacecraft telemetry original data with causality by using a PCA method;

s3: establishing a prediction model by using an SVR algorithm, and training the prediction model by applying spacecraft telemetry original data with causal relation after dimension reduction to obtain a pre-trained prediction model;

S4: predicting the telemetry data of the real-time spacecraft by using a pre-trained prediction model;

S5: and (3) dividing the predicted data points obtained in the step S4 into different clusters by using a clustering algorithm, and identifying abnormal points according to inter-cluster distances.

2. The method for detecting on-orbit anomalies of a complex spacecraft based on SVR and clustering according to claim 1, wherein the step of S1 further comprises:

pre-processing the data set to convert spacecraft telemetry raw data into a form suitable for Granger causal relationship analysis, the pre-processing comprising: centralizing, unit root checking, smoothing and solving for maximum delay.

3. The method for detecting on-orbit anomalies of a complex spacecraft based on SVR and clustering according to claim 1, wherein the S1 comprises:

Two stationary spacecraft telemetry raw time sequences Granger causal relationship test estimation regression:

；

；

Wherein, For the time series length,/>Is the variable value of the current time point,/>Is the variable value of the past time point,/>、、/>、/>Is a regression coefficient,/>、/>For/>、/>White noise contained in the internal;

If the past value of a spacecraft telemetry original time sequence has a Granger causal relationship with the current value, the past value is added to a predictive model of the current value for training of the predictive model.

4. The method for detecting on-orbit anomalies of a complex spacecraft based on SVR and clustering according to claim 1, wherein the S3 comprises: SVR maps input data into a high-dimensional space and finds a hyperplane in the space to fit the data, and predicts the value of the output variable by learning a linear or nonlinear regression function.

5. The method for detecting on-orbit anomalies of a complex spacecraft based on SVR and clustering according to claim 1, wherein the S3 comprises:

Solving the following minimization problem:

；

Wherein, Telemetry of raw time series data for two stationary spacecraft,/>Is thatWeight vector of dimension,/>Is an offset term,/>Represents the/>Relaxation variable of individual samples,/>And/>Is super-parameter,/>Is a kernel function.

6. The method for detecting the on-orbit anomalies of the complex spacecraft based on SVR and clustering according to claim 5, wherein the optimal super-parameters are selected by using a cross-validation method, and a model is retrained by using the optimal super-parameters.