CN118856241A - Urban pipeline monitoring method and system based on data analysis - Google Patents

Abstract

The invention discloses a city pipeline monitoring method and system based on data analysis, and particularly relates to the technical field of pipeline monitoring: analyzing operation data and marking abnormal data points by acquiring sensor basic information of different monitoring positions; constructing a spatial distribution model based on the physical positions of the sensors, and evaluating the correlation anomaly degree of the adjacent sensors; identifying abnormal fluctuation in the frequency components through spectrum analysis, and evaluating the stability of the signal frequency; comprehensively analyzing the correlation abnormality and the frequency stability, and determining the signal interference level; the sensor pair with high signal interference level is optimized, data before and after optimization are compared, parameters and deployment modes of the sensor are continuously adjusted, stability of sensor signals and monitoring accuracy of a system are improved through accurate analysis and optimization, misjudgment caused by signal confusion and interference is effectively avoided, and overall reliability and efficiency of the system are improved.

Description

Urban pipeline monitoring method and system based on data analysis

Technical Field

The present invention relates to the technical field of pipeline monitoring, and in particular to a method and system for urban pipeline monitoring based on data analysis.

Background Art

Urban pipeline monitoring based on data analysis refers to the use of advanced sensing technology and data processing methods to monitor and evaluate various pipelines in the city (such as water supply, drainage, gas pipelines, etc.) in real time. By deploying sensors in the pipelines to collect data such as pressure, flow, temperature, vibration, etc., combined with big data analysis and machine learning technology, it can automatically identify abnormal situations, predict potential risks, and optimize pipeline operation and maintenance. Through intelligent data processing methods, pipeline leakage, blockage, aging and other problems can be discovered in a timely manner to avoid sudden accidents and improve the safety and operation efficiency of urban pipeline systems.

The prior art has the following deficiencies:

When multiple sensors are running simultaneously, collecting different types of data (such as flow, pressure, temperature, etc.), but are close in space or operate at similar frequencies, their signals may be confused by the acquisition system. For example, an ultrasonic sensor used to monitor pipeline flow may operate in a similar frequency range as a nearby temperature sensor, causing the system to misread the data and mistake fluctuations in temperature changes for changes in flow. This signal confusion can cause data inconsistencies, making it impossible for the system to accurately judge the actual situation. At the same time, the data analysis system relies on the collaboration between sensors when processing data from different sensors. If the input data of some sensors is inconsistent due to interference, it may cause the algorithm to make incorrect judgments.

Summary of the invention

The purpose of the present invention is to provide a method and system for urban pipeline monitoring based on data analysis to solve the shortcomings of the background technology.

In order to achieve the above object, the present invention provides the following technical solution: a method for monitoring urban pipelines based on data analysis, comprising the following steps:

S1: Obtain basic information of sensors at different monitoring locations in the pipeline system, and conduct preliminary analysis on the actual operation data collected by each sensor. According to the change trend of the actual operation data, identify abnormal data points and mark them;

S2: For abnormal data points, based on the physical location of the sensors, the distance between sensors is calculated, and a spatial distribution model of sensors is established. The correlation analysis of sensor data located at adjacent spatial locations is performed to evaluate the abnormal degree of correlation between different sensor data;

S3: Perform spectrum analysis on the sensor data, convert the time series data into frequency domain data through Fourier transform, identify abnormal fluctuations in the frequency components, and evaluate the stability of the sensor signal frequency;

S4: Analyze the abnormal degree of correlation between different sensor data and the stability of sensor signal frequency, and determine the signal interference level between each pair of sensors based on the analysis results;

S5: Optimize sensor pairs with high signal interference levels, compare the data before and after optimization, evaluate whether the signal interference is reduced, continuously adjust the sensor parameters based on the evaluation results, and continuously optimize the deployment and working mode of the sensors.

Preferably, in S1, according to the changing trend of the actual operation data, abnormal data points are identified and marked, specifically: time series analysis is performed on the historical data of each sensor, and a graph of the changing trend of the data over time is drawn; for each sensor data, a moving average is calculated, that is, the average value is calculated within a specified time window, the difference between the sensor data and its moving average is calculated, and data points with large fluctuations are identified; if the differential value of the data point is greater than a set threshold, it is marked as abnormal.

Preferably, in S2, the linear correlation between the data of two adjacent sensors of the abnormal data point is analyzed to generate a correlation anomaly index, and the degree of correlation anomaly between the data of different sensors is evaluated. The method for obtaining the correlation anomaly index is:

Mark two adjacent sensors as S1 and S2, and record the corresponding time series data respectively. , represents the data sequence of sensor S1 from time 1 to n; , represents the data sequence of sensor S2 from time 1 to m, and constructs the cost matrix C, which records the distance between each pair of data points in the sequence Q and C, and is defined as: ;in yes and The distance between them is calculated as: ; The size of matrix C is n×m, and each element represents the distance between the i-th point of Q and the j-th point of C. Find the optimal matching path between time series and construct the cumulative distance matrix D, whose element D(i,j) represents the minimum path distance from Q[1] to Q[i] and C[1] to C[j]. The recursive relationship of dynamic programming is: ; In the formula, ; Starting from D(n,m), follow the dynamic planning path, that is, trace back to D(1,1), and find the minimum path distance between the time series Q and C. The length of the path is called the DTW distance ,Right now: ; Calculate the correlation anomaly index, the expression is: ; Where L is the length of the optimal matching path, and SD is the correlation anomaly index.

Preferably, in S3, the frequency distribution in the spectrum is analyzed, abnormal fluctuations in the frequency components are identified, a main frequency drift index is generated according to the degree of drift of the position of the main frequency component, and the stability of the sensor signal frequency is evaluated. The main frequency drift index is obtained by:

Obtain continuous data X(t) within a time period of M from the sensor, where t is time, X(t) is the signal value, and M is the size of the time window for obtaining signal data from the sensor. Decompose the original signal X(t) into several intrinsic mode functions , each IMF corresponds to a different frequency component, all local maxima and minima are identified from the signal X(t), and the upper envelope is obtained by interpolation through the maximum points; the lower envelope is obtained by interpolation through the minimum points, and the mean of the upper and lower envelopes is subtracted from the signal to obtain the middle part of the signal, until the remaining part no longer meets the IMF condition, which is regarded as the first IMF. The decomposed signal is expressed as: ; In the formula, is the kth eigenmode function, is the residual component, and Hilbert transform is performed on each IMF: , apply Hilbert transform to get instantaneous frequency and instantaneous amplitude, and transform the signal Convert to complex analytic signal , defined as: ;in, is the instantaneous amplitude, is the instantaneous phase, j is an imaginary unit, and the instantaneous phase Differentiate to get the instantaneous frequency , the expression is: ; From all IMFs, select the main frequency component and track the main frequency The change over time, calculate the absolute difference of frequency change, the expression is: ; In the formula, is the mean of the main frequency, that is, the average main frequency in the entire time period, and the expression is: ; Calculate the main frequency drift index, the expression is: ; Where GF is the main frequency drift index.

Preferably, in S4, the correlation anomaly index and the main frequency drift index are converted into a first eigenvector, and the first eigenvector is used as the input of the machine learning model. The machine learning model predicts the signal interference value label between each pair of sensors with each group of first eigenvectors as the prediction target, and minimizes the sum of prediction errors for all signal interference value labels as the training target. The machine learning model is trained until the sum of prediction errors converges and the model training is stopped. The signal interference value between each pair of sensors is determined according to the model output results, wherein the machine learning model is a polynomial regression model.

Preferably, the obtained signal interference value between each pair of sensors is compared with a gradient standard threshold, the gradient standard threshold includes a first standard threshold and a second standard threshold, and the first standard threshold is less than the second standard threshold, and the signal interference value between each pair of sensors is compared with the first standard threshold and the second standard threshold respectively;

If the signal interference value between each pair of sensors is greater than the second standard threshold, it means that the signal interference between the sensor pairs is high, and they are classified as high signal interference level and processed first;

If the signal interference value between each pair of sensors is greater than or equal to the first standard threshold and less than or equal to the second standard threshold, it means that the signal interference level between the sensor pairs is medium, and they are classified as medium signal interference level and are monitored regularly;

If the signal interference value between each pair of sensors is less than the first standard threshold, it means that the signal interference between the sensor pairs is low, and they are classified as low signal interference level, and no adjustment is required, and they operate normally.

Preferably, in S5, the sensor pairs with high signal interference levels are optimized, and the data before and after the optimization are compared to evaluate whether the signal interference is reduced, specifically:

If the signal interference value between each pair of sensors is greater than the second standard threshold, it is classified as a high signal interference level. If the physical parameters of sensors i and j are , and its optimization process is expressed as: ; Where CN is the signal interference value between each pair of sensors, is the sensor parameter at the kth iteration, is the optimization step size, is the gradient of the current signal interference value to the parameter; the physical locations of sensors i and j are and , signal interference and distance Related; the optimized position reduces interference by updating its physical position, the expression is: ; In the formula, is the sensor position at the kth iteration, β is the position optimization step size, is the gradient of signal interference to sensor spacing. After optimization, the new signal interference value is: ; F is the output function of the model. Comparing the difference in signal interference values before and after optimization, the expression is: .

Preferably, if , that is, the signal interference is reduced, then continue to optimize; if the signal interference is not reduced, further adjust the operating frequency of the sensor , the expression is: ; Where γ is the frequency adjustment step; when the signal interference value changes Less than the set threshold Stop optimization when .

The present invention also provides a city pipeline monitoring system based on data analysis, including an abnormal data point identification module, a correlation analysis module, a signal frequency analysis module, a signal interference level classification module and a signal interference optimization module;

Abnormal data point identification module: obtains basic information of sensors at different monitoring locations in the pipeline system, and performs preliminary analysis on the actual operation data collected by each sensor. According to the change trend of the actual operation data, abnormal data points are identified and marked;

Correlation analysis module: For abnormal data points, based on the physical location of the sensors, the distance between sensors is calculated, and a spatial distribution model of sensors is established. Correlation analysis is performed on sensor data located at adjacent spatial locations to evaluate the degree of abnormal correlation between different sensor data.

Signal frequency analysis module: performs spectrum analysis on sensor data, converts time series data into frequency domain data through Fourier transform, identifies abnormal fluctuations in frequency components, and evaluates the stability of sensor signal frequency;

Signal interference level classification module: Analyzes the abnormal degree of correlation between different sensor data and the stability of sensor signal frequency, and determines the signal interference level between each pair of sensors based on the analysis results;

Signal Interference Optimization Module: Optimizes sensor pairs with high signal interference levels, compares the data before and after optimization, evaluates whether the signal interference has been reduced, continuously adjusts the sensor parameters based on the evaluation results, and continuously optimizes the deployment and working mode of the sensors.

In the above technical solution, the technical effects and advantages provided by the present invention are:

1. The present invention solves the signal confusion problem in the pipeline monitoring system caused by the simultaneous operation of multiple sensors, similar operating frequencies or too close physical locations through precise data analysis and optimization technology. Based on the time series data and spatial distribution model of the sensor, combined with spectrum analysis and machine learning models, correlation anomalies and signal frequency drift can be effectively identified. Through the refined division of signal interference levels, it can flexibly respond to sensors with high, medium and low interference levels, especially optimize sensors with high interference levels, adjust their physical parameters, operating frequencies and location configurations, and make continuous adjustments based on the optimization results, thereby improving signal stability and monitoring accuracy.

2. The present invention not only improves the overall performance of the pipeline monitoring system, but also significantly reduces the system maintenance cost by reducing unnecessary interference and misjudgment. Compared with traditional methods, the present invention achieves efficient processing of sensor signal interference through automated data analysis and continuous optimization mechanisms, ensuring that the system can operate stably in the long term, reducing the need for manual intervention, extending the service life of sensor equipment, and improving the utilization of system resources. These optimizations bring higher monitoring accuracy and cost-effectiveness, effectively improving the level of intelligent management of urban pipeline networks.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

Fig. 1 is a flow chart of the method of the present invention;

FIG2 is a system module diagram of the present invention;

Figure 3 is a diagram showing the signal interference optimization results.

DETAILED DESCRIPTION

In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

Embodiment 1, referring to FIG. 1 and FIG. 3 , the urban pipeline monitoring method based on data analysis described in this embodiment includes the following steps:

S1: Obtain basic information of sensors at different monitoring locations in the pipeline system, and conduct preliminary analysis on the actual operation data collected by each sensor. According to the change trend of the actual operation data, identify abnormal data points and mark them;

S2: For abnormal data points, based on the physical location of the sensors, the distance between sensors is calculated, and a spatial distribution model of sensors is established. The correlation analysis of sensor data located at adjacent spatial locations is performed to evaluate the abnormal degree of correlation between different sensor data;

S3: Perform spectrum analysis on the sensor data, convert the time series data into frequency domain data through Fourier transform, identify abnormal fluctuations in the frequency components, and evaluate the stability of the sensor signal frequency;

S4: Analyze the abnormal degree of correlation between different sensor data and the stability of sensor signal frequency, and determine the signal interference level between each pair of sensors based on the analysis results;

S5: Optimize sensor pairs with high signal interference levels, compare the data before and after optimization, evaluate whether the signal interference is reduced, continuously adjust the sensor parameters based on the evaluation results, and continuously optimize the deployment and working mode of the sensors.

Among them, in S1, the basic information of sensors at different monitoring locations in the pipeline system is obtained, and the actual operation data collected by each sensor is preliminarily analyzed. According to the change trend of the actual operation data, abnormal data points are identified and marked, specifically:

Obtain basic information about each sensor in the pipeline monitoring system, including: sensor type (such as pressure, flow, temperature, etc.), sensor physical location (spatial coordinates or pipeline section), sensor operating parameters (sampling frequency, operating mode, etc.); remove obviously erroneous or invalid collected data, such as negative values or non-physical extreme values (such as negative temperature, ultra-high pressure, etc.). Use interpolation or mean to fill in missing data points to ensure data integrity.

Perform time series analysis on the historical data of each sensor and plot the trend of the data over time. For example, plot the fluctuations of pressure, flow, and temperature over time. Check the data for seasonal trends, periodic fluctuations, or sudden fluctuations. Different types of sensors may have different normal fluctuation patterns, such as flow differences between day and night, seasonal temperature changes, etc. For each sensor data, calculate the moving average, that is, calculate the average within a specified time window (such as the past 10 minutes or the past 10 sampling points) to smooth short-term fluctuations and highlight the overall trend of the data. Calculate the difference between the sensor data and its moving average to identify data points with large fluctuations. If the difference value of a data point is greater than the set threshold (such as twice the average difference), it is marked as an anomaly.

S2: For abnormal data points, based on the physical location of the sensors, the distance between sensors is calculated, and a spatial distribution model of the sensors is established. Correlation analysis is performed on sensor data located at adjacent spatial locations to evaluate the degree of abnormal correlation between different sensor data.

Collect the physical location data of all sensors in the pipeline system. This data can be expressed as three-dimensional coordinates (x, y, z) or as specific locations along the pipeline (such as meters from the starting point, segment markers, etc.); ensure that the location data of all sensors is stored in a unified format for subsequent calculations. For example: Sensor A: Location , Sensor B: Position ; Use the three-dimensional Euclidean distance formula to calculate the physical distance d between any two sensors. The expression is: ; Pairwise distances are calculated for all sensors, generating a distance matrix where each element represents the physical distance between two sensors. This helps determine which sensors are spatially adjacent.

Use a 3D graphics tool or a geographic information system (GIS) tool to draw a spatial distribution map of sensors, mark the locations of all sensors on the map, and connect adjacent sensors with lines or arrows. This helps to intuitively identify which sensors are close in space and are likely to affect each other. Based on the distance matrix of the sensors, set a distance threshold (such as 5 meters or 10 meters) to determine which sensors are adjacent sensors. Adjacent sensors are those sensor pairs whose physical distance is within the set threshold. For example, if the threshold is 10 meters, sensors A and B are adjacent, B and C are adjacent, but A and C are not adjacent. After determining the adjacent sensors, construct an adjacency matrix to indicate whether each sensor pair is adjacent. If the sensor pair is adjacent, the corresponding element in the matrix is 1, otherwise it is 0.

For pairs of sensors that are physically adjacent, collect their sensor data in the same time period. Ensure that the data collection frequency and time are synchronized to allow for effective comparison. For example, collect pressure, flow, or temperature data from adjacent sensors and store them as a time series.

After analyzing the linear correlation between the data of two adjacent sensors of the abnormal data point, a correlation anomaly index is generated to evaluate the degree of correlation anomaly between different sensor data. The correlation anomaly index is obtained as follows:

Mark two adjacent sensors as S1 and S2, and record the corresponding time series data respectively. , represents the data sequence of sensor S1 from time 1 to n; , represents the data sequence of sensor S2 from time 1 to m, and constructs the cost matrix C, which records the distance between each pair of data points in the sequence Q and C, and is defined as: ;in yes and The distance between them is calculated as: ; The size of matrix C is n×m, and each element represents the distance between the i-th point of Q and the j-th point of C. Find the optimal matching path between time series and construct the cumulative distance matrix D, whose element D(i,j) represents the minimum path distance from Q[1] to Q[i] and C[1] to C[j]. The recursive relationship of dynamic programming is: ; In the formula, , the min function represents the distance of the current point plus the minimum cumulative distance of the previous step (to the left, up, or to the upper left corner); starting from D(n,m), follow the dynamic planning path, that is, trace back to D(1,1), and find the minimum path distance between the time series Q and C. The length of the path is called the DTW distance ,Right now: ; Calculate the correlation anomaly index, the expression is: ; Where L is the length of the optimal matching path, represents the total number of regularization steps of the two time series, and SD is the correlation anomaly index.

The larger the correlation anomaly index, the higher the degree of correlation anomaly between the data of the two sensors, indicating that their behavior patterns have deviated significantly. In other words, the time series data of the two sensors show large inconsistencies or mismatches at different time points, which may be caused by environmental changes, equipment failures, signal interference, etc. When the correlation anomaly index is close to 1, it indicates that the measurement results of the two sensors have significantly deviated from their normal correlation patterns, which may mean that there are serious problems or abnormalities in the system.

The smaller the correlation anomaly index, the lower the degree of correlation anomaly between the data of the two sensors, indicating that their time series data have a high degree of consistency or match at most time points. A lower anomaly index close to 0 means that the sensor measurements remain stable and in line with expectations, and the correlation between them is high, reflecting the normal correlation pattern between the data. This situation usually indicates that the system is operating well and there is no significant interference or abnormal signals between the sensors.

S3: Perform spectrum analysis on the sensor data, convert the time series data into frequency domain data through Fourier transform, identify abnormal fluctuations in the frequency components, and evaluate the stability of the sensor signal frequency.

Get continuous data from the target sensor within a time period of M and record it as a time series. For example, if the sensor collects flow, pressure or temperature, record the continuously changing values of these parameters and mark the sensor data as X(t), where t is time and X is the sampled value at each time t. These data can be represented as a time series, such as X=[x1,x2,…,xn].

Apply Fourier transform to the time series data X(t) and convert it into frequency domain data. Use the fast Fourier transform algorithm to improve the computational efficiency. The expression is: ; In the formula, Represents data in the frequency domain, reflecting the signal strength at each frequency f. x(t) is the data point of the sensor in the time domain, n is the total length of the time series, f is the frequency component, and the corresponding unit is usually Hz; after Fourier transform, the result X(f) contains components of different frequencies and their corresponding amplitudes. Draw a spectrum graph, with the horizontal axis representing the frequency f and the vertical axis representing the amplitude or intensity at each frequency. .

Analyze the frequency distribution in the spectrum, identify abnormal fluctuations in the frequency components, generate the main frequency drift index according to the degree of drift of the main frequency component, and evaluate the stability of the sensor signal frequency. The main frequency drift index is obtained as follows:

Obtain continuous data X(t) within a time period of M from the sensor, where t is time, X(t) is the signal value, and M is the size of the time window for obtaining signal data from the sensor. Decompose the original signal X(t) into several intrinsic mode functions , each IMF corresponds to a different frequency component, all local maxima and minima are identified from the signal X(t), and the upper envelope is obtained by interpolation through the maximum points; the lower envelope is obtained by interpolation through the minimum points, and the mean of the upper and lower envelopes is subtracted from the signal to obtain the middle part of the signal, until the remaining part no longer meets the IMF conditions (local symmetry, zero mean, etc.). This part is regarded as the first IMF, and the decomposed signal is expressed as: ; In the formula, is the kth eigenmode function, is the residual component, and Hilbert transform is performed on each IMF: , apply Hilbert transform to get instantaneous frequency and instantaneous amplitude, and transform the signal Convert to complex analytic signal , defined as: ;in, is the instantaneous amplitude, is the instantaneous phase, j is an imaginary unit, and the instantaneous phase Differentiate to get the instantaneous frequency , the expression is: ; From all IMFs, select the main frequency component and track the main frequency The change over time, calculate the absolute difference of frequency change, the expression is: ; In the formula, is the mean of the main frequency, that is, the average main frequency in the entire time period, and the expression is: ; Calculate the main frequency drift index, the expression is: ; Where GF is the main frequency drift index.

The larger the main frequency drift index is, the more serious the frequency drift of the sensor signal is, indicating that the main frequency has fluctuated greatly or is unstable over time. This means that the sensor's signal frequency is unstable and may be affected by external interference, equipment aging, or environmental changes, resulting in significant deviations or fluctuations in the signal frequency. The more unstable the frequency is, the worse the reliability of the signal is, which can easily affect the normal operation of the system and the accuracy of the monitoring results.

The smaller the main frequency drift index, the less the frequency drift of the sensor signal is, indicating that the main frequency remains relatively constant over the entire time period. The more stable the signal frequency is, the better the sensor is operating, the better the signal quality is, and the signal is less affected by external interference or problems with the device itself. A stable signal frequency helps improve the reliability and accuracy of the monitoring system.

S4: Analyze the abnormal degree of correlation between different sensor data and the stability of sensor signal frequency, and determine the signal interference level between each pair of sensors based on the analysis results.

The correlation anomaly index and the main frequency drift index are converted into the first eigenvector, and the first eigenvector is used as the input of the machine learning model. The machine learning model predicts the signal interference value label between each pair of sensors with each group of first eigenvectors as the prediction target, and minimizes the sum of prediction errors for all signal interference value labels as the training target. The machine learning model is trained until the sum of prediction errors converges and the model training is stopped. The signal interference value between each pair of sensors is determined according to the model output results, wherein the machine learning model is a polynomial regression model.

The method for obtaining the signal interference value between each pair of sensors is as follows: from the first feature vector training data of the trained machine learning model, the corresponding function expression is obtained: ; Where F is the output function of the model, SD is the correlation anomaly index, GF is the main frequency drift index, and CN is the signal interference value between each pair of sensors.

The obtained signal interference value between each pair of sensors is compared with a gradient standard threshold, where the gradient standard threshold includes a first standard threshold and a second standard threshold, and the first standard threshold is less than the second standard threshold, and the signal interference value between each pair of sensors is compared with the first standard threshold and the second standard threshold respectively;

If the signal interference value between each pair of sensors is greater than the second standard threshold, it means that the signal interference between the sensor pairs is high, and it is classified as a high signal interference level. High interference may have a significant impact on the overall monitoring quality of the system and needs to be handled with priority;

If the signal interference value between each pair of sensors is greater than or equal to the first standard threshold and less than or equal to the second standard threshold, it means that the signal interference between the sensor pairs is medium, and they are classified as medium signal interference levels. These sensor pairs with medium interference levels are regularly monitored to track changes in interference values. If the interference value continues to rise, further intervention measures may be required;

If the signal interference value between each pair of sensors is less than the first standard threshold, it means that the signal interference degree between the sensor pairs is low, and they are classified as low signal interference levels. For sensor pairs with low interference levels, usually no additional processing or adjustment is required, and the system can operate normally.

S5: Optimize sensor pairs with high signal interference levels, compare the data before and after optimization, evaluate whether the signal interference is reduced, continuously adjust the sensor parameters based on the evaluation results, and continuously optimize the deployment and working mode of the sensors.

If the signal interference value between each pair of sensors is greater than the second standard threshold, it is classified as a high signal interference level. If the physical parameters of sensors i and j are (including operating frequency, sampling frequency, sensor spacing, etc.), the optimization process is expressed as: ; Where CN is the signal interference value between each pair of sensors, is the sensor parameter at the kth iteration, is the optimization step size, is the gradient of the current signal interference value to the parameter; the physical locations of sensors i and j are and , signal interference and distance Related; the optimized position reduces interference by updating its physical position, the expression is: ; In the formula, is the sensor position at the kth iteration, β is the position optimization step size, is the gradient of signal interference to sensor spacing. After optimization, the new signal interference value is: ; F is the output function of the model. Comparing the difference in signal interference values before and after optimization, the expression is: .

if , that is, the signal interference is reduced, then continue to optimize; if the signal interference is not reduced, further adjust the operating frequency of the sensor , the expression is: ; Where γ is the frequency adjustment step; when the signal interference value changes Less than the set threshold Stop optimization when .

In this embodiment, during the electromagnetic interference monitoring and health assessment process of the switch cabinet, first, by installing electromagnetic interference detection equipment around the switch cabinet and at the communication nodes, and using the Internet of Things platform for real-time monitoring, the electromagnetic field strength data in different time periods and the real-time status data of the switch cabinet are obtained. Then, the electromagnetic field strength data in different time periods are assigned weight coefficients, and the overall electromagnetic interference index of the switch cabinet is calculated by weighted average. Finally, the overall electromagnetic interference index and the real-time status data of the switch cabinet are comprehensively analyzed using fuzzy logic to generate a comprehensive evaluation report, which displays the health status of the switch cabinet, helps predict the operating status of the equipment, and provides maintenance suggestions.

Embodiment 2, as shown in FIG2 , the urban pipeline monitoring system based on data analysis described in this embodiment includes an abnormal data point identification module, a correlation analysis module, a signal frequency analysis module, a signal interference level classification module and a signal interference optimization module;

Abnormal data point identification module: obtains basic information of sensors at different monitoring locations in the pipeline system, and performs preliminary analysis on the actual operation data collected by each sensor. According to the change trend of the actual operation data, abnormal data points are identified and marked.

Correlation analysis module: For abnormal data points, based on the physical location of the sensors, the distance between sensors is calculated, and a spatial distribution model of sensors is established. Correlation analysis is performed on sensor data located at adjacent spatial locations to evaluate the degree of abnormal correlation between different sensor data.

Signal frequency analysis module: performs spectrum analysis on sensor data, converts time series data into frequency domain data through Fourier transform, identifies abnormal fluctuations in frequency components, and evaluates the stability of sensor signal frequency;

Signal interference level classification module: Analyzes the abnormal degree of correlation between different sensor data and the stability of sensor signal frequency, and determines the signal interference level between each pair of sensors based on the analysis results;

Signal Interference Optimization Module: Optimizes sensor pairs with high signal interference levels, compares the data before and after optimization, evaluates whether the signal interference has been reduced, continuously adjusts the sensor parameters based on the evaluation results, and continuously optimizes the deployment and working mode of the sensors.

The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters in the formula are set by technicians in this field according to actual conditions.

The above embodiments can be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

It should be understood that the term "and/or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and/or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character "/" in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and/or" relationship. Please refer to the context for specific understanding.

In this application, "at least one" means one or more, and "plurality" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can mean: a, b, c, a-b, a-c, b-c, or a-b-c, where a, b, c can be single or multiple.

It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

Claims (9)

1. A method for monitoring urban pipelines based on data analysis, characterized in that it comprises the following steps: S1: Obtain basic information of sensors at different monitoring locations in the pipeline system, and conduct preliminary analysis on the actual operation data collected by each sensor. According to the change trend of the actual operation data, identify abnormal data points and mark them; S2: For abnormal data points, based on the physical location of the sensors, the distance between sensors is calculated, and a spatial distribution model of sensors is established. The correlation analysis of sensor data located at adjacent spatial locations is performed to evaluate the degree of abnormal correlation between different sensor data. S3: Perform spectrum analysis on the sensor data, convert the time series data into frequency domain data through Fourier transform, identify abnormal fluctuations in the frequency components, and evaluate the stability of the sensor signal frequency; S4: Analyze the abnormal degree of correlation between different sensor data and the stability of sensor signal frequency, and determine the signal interference level between each pair of sensors based on the analysis results; S5: Optimize sensor pairs with high signal interference levels, compare the data before and after optimization, evaluate whether the signal interference is reduced, continuously adjust the sensor parameters based on the evaluation results, and continuously optimize the deployment and working mode of the sensors. 2. The urban pipeline monitoring method based on data analysis according to claim 1 is characterized in that: in S1, according to the change trend of the actual operation data, abnormal data points are identified and marked, specifically: time series analysis is performed on the historical data of each sensor, and a data change trend graph is drawn over time. For each sensor data, a moving average is calculated, that is, the average value is calculated within a specified time window, and the difference between the sensor data and its moving average value is calculated to identify data points with large fluctuations. If the difference value of the data point is greater than a set threshold, it is marked as abnormal. 3. The urban pipeline monitoring method based on data analysis according to claim 1 is characterized in that: in S2, the linear correlation between the data of two adjacent sensors of the abnormal data point is analyzed to generate a correlation anomaly index, and the degree of correlation anomaly between the data of different sensors is evaluated. The method for obtaining the correlation anomaly index is: Mark two adjacent sensors as S1 and S2, and record the corresponding time series data respectively. , represents the data sequence of sensor S1 from time 1 to n; , represents the data sequence of sensor S2 from time 1 to m, and constructs the cost matrix C, which records the distance between each pair of data points in the sequence Q and C, and is defined as: ;in yes and The distance between them is calculated as: ; The size of the matrix C is n×m, and each element represents the distance between the i-th point of Q and the j-th point of C. Find the optimal matching path between time series and construct the cumulative distance matrix D, whose element D(i,j) represents the minimum path distance from Q[1] to Q[i] and C[1] to C[j]. The recursive relationship of dynamic programming is: ; In the formula, ; Starting from D(n,m), follow the dynamic planning path, that is, trace back to D(1,1), and find the minimum path distance between the time series Q and C. The length of the path is called the DTW distance ,Right now: ; Calculate the correlation anomaly index, the expression is: ; Where L is the length of the optimal matching path, and SD is the correlation anomaly index. 4. The urban pipeline monitoring method based on data analysis according to claim 3 is characterized in that: in S3, the frequency distribution in the spectrum is analyzed, the abnormal fluctuation in the frequency component is identified, the main frequency drift index is generated according to the degree of drift of the position of the main frequency component, and the stability of the sensor signal frequency is evaluated. The main frequency drift index is obtained by: Obtain continuous data X(t) within a time period of M from the sensor, where t is time, X(t) is the signal value, and M is the size of the time window for obtaining signal data from the sensor. Decompose the original signal X(t) into several intrinsic mode functions , each IMF corresponds to a different frequency component, all local maxima and minima are identified from the signal X(t), and the upper envelope is obtained by interpolation through the maximum points; the lower envelope is obtained by interpolation through the minimum points, and the mean of the upper and lower envelopes is subtracted from the signal to obtain the middle part of the signal, until the remaining part no longer meets the IMF condition, which is regarded as the first IMF. The decomposed signal is expressed as: ; In the formula, is the kth eigenmode function, is the residual component, and Hilbert transform is performed on each IMF: , apply Hilbert transform to get instantaneous frequency and instantaneous amplitude, and transform the signal Convert to complex analytic signal , defined as: ;in, is the instantaneous amplitude, is the instantaneous phase, j is an imaginary unit, and the instantaneous phase Differentiate to get the instantaneous frequency , the expression is: ; From all IMFs, select the main frequency component and track the main frequency The change over time, calculate the absolute difference of frequency change, the expression is: ; In the formula, is the mean of the main frequency, that is, the average main frequency in the entire time period, and the expression is: ; Calculate the main frequency drift index, the expression is: ; Where GF is the main frequency drift index. 5. The urban pipeline monitoring method based on data analysis according to claim 4 is characterized in that: in S4, the correlation anomaly index and the main frequency drift index are converted into a first eigenvector, and the first eigenvector is used as the input of the machine learning model. The machine learning model predicts the signal interference value label between each pair of sensors with each group of first eigenvectors as the prediction target, and minimizes the sum of prediction errors of all signal interference value labels as the training target. The machine learning model is trained until the sum of prediction errors converges, and the model training is stopped. The signal interference value between each pair of sensors is determined according to the model output result, wherein the machine learning model is a polynomial regression model. 6. The urban pipeline monitoring method based on data analysis according to claim 5 is characterized in that: the obtained signal interference value between each pair of sensors is compared with the gradient standard threshold, the gradient standard threshold includes a first standard threshold and a second standard threshold, and the first standard threshold is less than the second standard threshold, and the signal interference value between each pair of sensors is compared with the first standard threshold and the second standard threshold respectively; If the signal interference value between each pair of sensors is greater than the second standard threshold, it means that the signal interference between the sensor pairs is high, and they are classified as high signal interference level and processed first; If the signal interference value between each pair of sensors is greater than or equal to the first standard threshold and less than or equal to the second standard threshold, it means that the signal interference level between the sensor pairs is medium, and they are classified as medium signal interference level and are monitored regularly; If the signal interference value between each pair of sensors is less than the first standard threshold, it means that the signal interference between the sensor pairs is low, and they are classified as low signal interference level, and no adjustment is required, and they operate normally. 7. The urban pipeline monitoring method based on data analysis according to claim 1 is characterized in that: in S5, the sensor pairs with high signal interference levels are optimized, and the data before and after the optimization are compared to evaluate whether the signal interference is reduced, specifically: If the signal interference value between each pair of sensors is greater than the second standard threshold, it is classified as a high signal interference level. If the physical parameters of sensors i and j are , and its optimization process is expressed as: ; Where CN is the signal interference value between each pair of sensors, is the sensor parameter at the kth iteration, is the optimization step size, is the gradient of the current signal interference value to the parameter; the physical locations of sensors i and j are and , signal interference and distance Related; the optimized position reduces interference by updating its physical position, the expression is: ; In the formula, is the sensor position at the kth iteration, β is the position optimization step size, is the gradient of signal interference to sensor spacing. After optimization, the new signal interference value is: ; F is the output function of the model. Comparing the difference in signal interference values before and after optimization, the expression is: . 8. The urban pipeline monitoring method based on data analysis according to claim 7 is characterized in that: if , that is, the signal interference is reduced, then continue to optimize; if the signal interference is not reduced, further adjust the operating frequency of the sensor , the expression is: ; Where γ is the frequency adjustment step; when the signal interference value changes Less than the set threshold Stop optimization when . 9. A city pipeline monitoring system based on data analysis, used to implement the city pipeline monitoring method based on data analysis according to any one of claims 1 to 8, characterized in that it comprises an abnormal data point identification module, a correlation analysis module, a signal frequency analysis module, a signal interference level classification module and a signal interference optimization module; Abnormal data point identification module: obtains basic information of sensors at different monitoring locations in the pipeline system, and performs preliminary analysis on the actual operation data collected by each sensor. According to the change trend of the actual operation data, abnormal data points are identified and marked; Correlation analysis module: For abnormal data points, based on the physical location of the sensors, the distance between sensors is calculated, and a spatial distribution model of sensors is established. Correlation analysis is performed on sensor data located at adjacent spatial locations to evaluate the degree of abnormal correlation between different sensor data. Signal frequency analysis module: performs spectrum analysis on sensor data, converts time series data into frequency domain data through Fourier transform, identifies abnormal fluctuations in frequency components, and evaluates the stability of sensor signal frequency; Signal interference level classification module: Analyzes the abnormal degree of correlation between different sensor data and the stability of sensor signal frequency, and determines the signal interference level between each pair of sensors based on the analysis results; Signal Interference Optimization Module: Optimizes sensor pairs with high signal interference levels, compares the data before and after optimization, evaluates whether the signal interference has been reduced, continuously adjusts the sensor parameters based on the evaluation results, and continuously optimizes the deployment and working mode of the sensors.