CN114297186B - Power consumption data preprocessing method and system based on deviation coefficient - Google Patents

Abstract

The present invention discloses a method and system for preprocessing electricity consumption data based on a deviation coefficient, which corrects the electricity consumption data according to the electricity consumption deviation coefficient τ, and then performs data cleaning on the corrected electricity consumption data. The present invention corrects the electricity consumption data according to the deviation of the electricity consumption data, reduces the interference of abnormal factors on the electricity consumption data, eliminates the influence of false abnormal data, and provides relatively continuous and reliable reference data for electricity consumption data monitoring.

Description

A method and system for preprocessing electricity consumption data based on deviation coefficient

Technical Field

The present invention relates to the technical field of data processing, and more particularly to a method and system for preprocessing electricity consumption data based on a deviation coefficient.

Background technique

With the use of smart meters and the development of industry intelligence, it has become an industry trend to monitor user electricity consumption through the Internet of Things. To remotely monitor user electricity consumption through the Internet of Things, it is necessary to identify and judge abnormal user electricity consumption, which requires analyzing and processing the user's historical electricity consumption data, and using the processed data as the basis for predicting the operation of the power grid. In the actual operation of the power grid, there are some factors that cause the electricity consumption data to deviate from the actual value, such as the increase in line loss caused by more line joints and the loss caused by large voltage fluctuations. These factors will make some good data appear as abnormal data, which leads to an increase in abnormal data and reduces the reference data required for data analysis, which will affect the accuracy of data analysis and further affect the prediction of the operation of the power grid.

Summary of the invention

To this end, the technical problem to be solved by the present invention is to provide a method and system for preprocessing electricity consumption data based on a deviation coefficient, correct the electricity consumption data according to the deviation of the electricity consumption data, reduce the interference of abnormal factors on the electricity consumption data, eliminate false abnormal data, and provide relatively continuous and reliable reference data for electricity consumption data monitoring.

In order to solve the above technical problems, the present invention provides the following technical solutions:

A method for preprocessing electricity consumption data based on a deviation coefficient comprises the following steps:

a) According to the power consumption deviation coefficient τ, the power consumption data is corrected using the following formula:

τ＝ _Qinitial /ΔQ

Where: α is the correction coefficient;

Q _initial is the power consumption collected at the power consumption statistical time point T _n , n ≥ 1;

_Qcorrected is the corrected electricity consumption;

ΔQ is the average power consumption per unit time during the power consumption data statistical period T;

b) Clean the corrected data.

In the above-mentioned method for preprocessing electricity consumption data based on deviation coefficient, in step a), the correction coefficient α is determined by the following method:

The statistical period T of electricity consumption data is divided into time lengths Δt, the overall standard deviation of each time period is calculated, and the average power consumption Δq in the time period with the smallest overall standard deviation is selected as the corrected reference power consumption. Then, the correction coefficient α is determined by the following formula:

α＝(Δq/ΔQ) ^(2Δt/T) .

In the above-mentioned electricity consumption data preprocessing method based on deviation coefficient, Δt/T is greater than or equal to 0.05.

In the above-mentioned electricity consumption data preprocessing method based on deviation coefficient, Δt/T is less than or equal to 0.5.

In the above-mentioned electricity consumption data preprocessing method based on deviation coefficient, in step a), before correcting the electricity consumption data, the volatility of the electricity consumption data is evaluated using the fluctuation ratio ζ, wherein ζ＝|(Qn ₊₁ - _Qn )/ _Qn |, and n is greater than or equal to 1; when ζ is greater than or equal to 0.25 and the fluctuation ratio ζ of the electricity consumption data collected by greater than or equal to 3 consecutive collection points starting from the (n+1)th collection point is less than or equal to 0.1, then when correcting the electricity consumption data, the electricity consumption data from the (n+1)th to the (n+m)th collection points are corrected using independent correction coefficients, wherein m is greater than or equal to 3.

In the above-mentioned electricity consumption data preprocessing method based on the deviation coefficient, when ζ is greater than or equal to 0.25, when there are less than 3 consecutive collection points since the (n+1)th collection point whose electricity consumption data fluctuation ratio ζ is less than or equal to 0.1, the data collected from the (n+1)th collection point is still corrected using the original correction coefficient.

In the above-mentioned electricity consumption data preprocessing method based on the deviation coefficient, when ζ is greater than or equal to 0.25, the fluctuation ratio ζ of the electricity consumption data collected by greater than or equal to [0.8m] collection points from the (n+1)th to the (n+m)th collection points and the electricity consumption data collected by the (n+1)th collection point is less than or equal to 0.1, then when correcting the electricity consumption data, the electricity consumption data from the (n+1)th to the (n+m)th collection points are corrected using an independent correction coefficient, wherein m is greater than or equal to 5.

A system for preprocessing electricity consumption data using the above-mentioned electricity consumption data preprocessing method based on deviation coefficient comprises:

A data collection module, used to collect and group electricity consumption data;

Data deviation analysis module, used to perform deviation analysis on electricity consumption data;

Data fluctuation analysis module, used to perform fluctuation analysis on the power consumption data of two adjacent collection points;

A data correction module, used to correct the electricity consumption data;

A data cleaning module, used to clean the corrected electricity consumption data;

The data acquisition module is respectively connected to the data deviation analysis module and the data fluctuation analysis module in communication, the data deviation analysis module and the data fluctuation analysis module are respectively connected to the data correction module in communication, and the data correction module is connected to the data cleaning module in communication.

The above system also includes a data sorting module, the data deviation analysis module and the data fluctuation analysis module are respectively connected to the data sorting module in communication, and the data sorting module is connected to the data correction module in communication.

The above system also includes a data storage module, the data sorting module is communicatively connected to the data storage module, and the data storage module is communicatively connected to the data correction module.

The technical solution of the present invention achieves the following beneficial technical effects:

1. The present invention corrects the electricity consumption data according to the deviation of the electricity consumption data, reduces the possibility of false abnormal data being cleaned, and provides sample analysis data for monitoring the electricity consumption data of the power grid using historical electricity consumption data.

2. Use different correction methods for electricity consumption data according to the fluctuation of electricity consumption data to avoid under- or over-correction of electricity consumption data during periods of large fluctuations.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a working principle diagram of the power consumption data preprocessing system based on the deviation coefficient in the present invention;

Figure 2 Line chart of electricity consumption data before correction;

Figure 3: Corrected line chart of electricity consumption data.

Detailed ways

As shown in Figure 1, the system for preprocessing electricity data using the electricity data preprocessing method based on the deviation coefficient in the present invention includes a data acquisition module, a data deviation analysis module, a data fluctuation analysis module, a data correction module, a data cleaning module, a data sorting module and a data storage module. The data acquisition module is communicated with the data deviation analysis module and the data fluctuation analysis module respectively, the data deviation analysis module and the data fluctuation analysis module are communicated with the data sorting module respectively, the data sorting module is communicated with the data storage module, the data storage module is communicated with the data correction module, and the data correction module is communicated with the data cleaning module.

Among them, the data acquisition module is used to collect and group electricity consumption data; the data deviation analysis module is used to perform deviation analysis on electricity consumption data; the data fluctuation analysis module is used to perform fluctuation analysis on electricity consumption data of two adjacent collection points; the data correction module is used to correct electricity consumption data; and the data cleaning module is used to clean the corrected electricity consumption data.

In the actual electricity consumption process, there are many factors that will cause the electricity consumption data collected by the smart meter to deviate from the actual electricity consumption data. Before correcting the electricity consumption data, it is necessary to perform a deviation analysis on the electricity consumption data to avoid adjusting the correct data to wrong data or adjusting the wrong data to more outrageous data. First, the electricity consumption data must be analyzed for deviation and a deviation threshold must be set. The deviation threshold set in this embodiment is (0.85, 1.3], and then the electricity consumption data must be corrected according to the degree of deviation of the electricity consumption data. Finally, the corrected electricity consumption data must be cleaned. That is, the electricity consumption data preprocessing system is used to preprocess the historical electricity consumption data. The specific steps are as follows:

a) According to the power consumption deviation coefficient τ, the power consumption data is corrected using the following formula:

τ＝ _Qinitial /ΔQ

Where: α is the correction coefficient;

Q _initial is the power consumption collected at the power consumption statistical time point T _n , n ≥ 1;

_Qcorrected is the corrected electricity consumption;

ΔQ is the average power consumption per unit time during the power consumption data statistical period T;

b) Clean the corrected data.

In step a), the correction coefficient α is determined by the following method:

The statistical period T of electricity consumption data is divided into time lengths Δt, the overall standard deviation of each time period is calculated, and the average power consumption Δq in the time period with the smallest overall standard deviation is selected as the corrected reference power consumption. Then, the correction coefficient α is determined by the following formula:

α＝(Δq/ΔQ) ^(2Δt/T) .

Among them, the correction coefficient α is proposed based on the regular fluctuation of electricity consumption data, that is, the electricity consumption data are similar within a certain period of time, the fluctuation range is small, and it has relatively obvious regularity. For example, the time points and duration of use of equipment with high power consumption have certain regularity. Therefore, the average power consumption Δq in the time period with the smallest overall standard deviation is selected as the corrected reference power consumption, and it is correlated with the average power consumption per unit time ΔQ in the electricity consumption data statistical period T, as well as the time length Δt and the electricity consumption data statistical period T to derive the calculation formula of the correction coefficient α.

In order to reduce the difficulty of electricity consumption data preprocessing and improve the efficiency of electricity consumption data preprocessing, in this embodiment, Δt/T is greater than or equal to 0.05 and less than or equal to 0.5, that is, when Δt/T is less than 0.05, the size of Δt is adjusted.

In view of the fact that there are certain differences in the power consumption data in different power consumption periods, using the same correction coefficient to correct the power consumption data with large differences will result in over-correction, that is, the corrected data will be distorted. Therefore, in step a), before correcting the power consumption data, the volatility of the power consumption data is evaluated using the fluctuation ratio ζ, where ζ = |(Q _n+1 -Q _n )/Q _n |, n is greater than or equal to 1; when ζ is greater than or equal to 0.25 and there are more than or equal to 3 consecutive collection points since the (n+1)th collection point whose power consumption data fluctuation ratio is less than or equal to ζ, then when correcting the power consumption data, the power consumption data from the (n+1)th to the (n+m)th collection points are corrected using an independent correction coefficient, wherein m is greater than or equal to 3, and when ζ is greater than or equal to 0.25, when there are less than 3 consecutive collection points since the (n+1)th collection point whose power consumption data fluctuation ratio is less than or equal to ζ, the data collected from the (n+1)th collection point is still corrected using the original correction coefficient. In view of the fact that certain factors may cause more frequent fluctuations in electricity consumption data, when ζ is greater than or equal to 0.25, and the fluctuation ratio ζ of the electricity consumption data collected by greater than or equal to [0.8m] collection points from the (n+1)th to the (n+m)th collection points and the electricity consumption data collected by the (n+1)th collection point is less than or equal to 0.1, when correcting the electricity consumption data, the electricity consumption data from the (n+1)th to the (n+m)th collection points are corrected using an independent correction coefficient, where m is greater than or equal to 5.

The electricity consumption data preprocessing method in the present invention is used to process the electricity consumption data of user A from October 3, 2020 to November 11, 2020, and the processing results are shown in Table 1.

Table 1 Historical electricity consumption data of user A and its preprocessing results

date energy used Deviation coefficient τ Corrected electricity consumption data October 3, 2020 2.14 1.196 2.14 October 4, 2020 1.98 1.106 1.98 October 5, 2020 1.72 0.961 1.72 October 6, 2020 1.48 0.823 1.49 October 7, 2020 2.14 1.196 2.14 October 8, 2020 1.58 0.883 1.58 October 9, 2020 2.00 1.117 2.00 October 10, 2020 1.51 0.844 1.52 October 11, 2020 2.10 1.173 2.10 October 12, 2020 1.52 0.849 1.53 October 13, 2020 2.10 1.173 2.10 October 14, 2020 1.60 0.894 1.60 October 15, 2020 1.75 0.978 1.75 October 16, 2020 2.09 1.168 2.09 October 17, 2020 1.85 1.034 1.85 October 18, 2020 1.80 1.006 1.80 October 19, 2020 1.73 0.966 1.73 October 20, 2020 1.47 0.821 1.48 October 21, 2020 1.99 1.112 1.99 October 22, 2020 1.51 0.844 1.52 October 23, 2020 1.60 0.894 1.60 October 24, 2020 1.43 0.799 1.44 October 25, 2020 1.62 0.905 1.62 October 26, 2020 2.18 1.218 2.18 October 27, 2020 1.64 0.916 1.64 October 28, 2020 2.00 1.117 2.00 October 29, 2020 2.02 1.128 2.02 October 30, 2020 1.73 0.966 1.73 October 31, 2020 1.52 0.849 1.53 November 1, 2020 1.76 0.983 1.76 November 2, 2020 1.50 0.838 1.51 November 3, 2020 2.25 1.257 2.25 November 4, 2020 1.42 0.793 1.43 November 5, 2020 1.45 0.810 1.46 November 6, 2020 1.55 0.866 1.55 November 7, 2020 1.41 0.788 1.42 November 8, 2020 2.25 1.257 2.25 November 9, 2020 1.61 0.899 1.61 November 10, 2020 2.13 1.190 2.13 November 11, 2020 2.06 1.151 2.06 November 12, 2020 1.72 0.961 1.72 November 13, 2020 1.33 0.743 1.34 November 14, 2020 1.46 0.816 1.47

Among them, T = 43, Δt = 5, and Δq is the average electricity consumption from October 15, 2020 to October 19, 2020. The electricity consumption data before and after correction are plotted as line graphs, as shown in Figures 2 and 3. The line graph of the electricity consumption data after correction does not change significantly from the line graph of the electricity consumption data before correction, but the electricity consumption data with a large deviation tends to be close to the electricity consumption change trend composed of the basic change data and change trend data of the electricity consumption data after correction. Among them, the basic change data of the electricity consumption data refers to the electricity consumption data that only changes with the length of use time but not with the change of seasons, such as lighting and other daily electricity consumption, and the change trend data of the electricity consumption data refers to the electricity consumption data that changes with the change of seasons, such as air conditioning cooling or heating electricity.

Obviously, the above embodiments are merely examples for the purpose of clear explanation, and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the claims of this patent application.

Claims (6)

1. The power consumption data preprocessing method based on the deviation coefficient is characterized by comprising the following steps of:

a) And correcting the electricity consumption data according to the electricity consumption deviation coefficient tau by using the following formula:

τ＝Q _{Initially, the method comprises} /ΔQ

Wherein: alpha is a correction coefficient;

Q _{Initially, the method comprises} is the electricity consumption acquired at the electricity consumption statistics time point T _n, and n is more than or equal to 1;

Q _correction is the corrected electricity consumption;

Delta Q is average electricity consumption per unit time in the electricity consumption data statistics period T;

wherein, the correction coefficient alpha is determined by the following method:

Dividing the electricity consumption data statistics period T by using the time length delta T, calculating the total standard deviation of each time period, selecting the average value delta q of the electricity consumption in the time period with the minimum total standard deviation as the correction reference electricity consumption, and then determining the correction coefficient alpha by the following formula:

α＝(Δq/ΔQ)^(2Δt/T)；

before correcting the power consumption data, the fluctuation of the power consumption data is evaluated by using a fluctuation ratio ζ, wherein ζ= | (Q _n+1-Q_n)/Q_n |, n is larger than or equal to 1), when ζ is larger than or equal to 0.25 and the fluctuation ratio ζ of the power consumption data collected from the (n+1) th collection point is larger than or equal to 0.1 continuously, the power consumption data from the (n+1) th collection point to the (n+m) th collection point is corrected by using an independent correction coefficient when ζ is larger than or equal to 0.25, and when ζ is larger than or equal to 0.25, the data collected from the (n+1) th collection point is still corrected by using the original correction coefficient when ζ is larger than or equal to 0.25 and the data collected from the (n+1) th collection point to the (n+1) th collection point is still corrected by using the independent correction coefficient when ζ is larger than or equal to 0.25, and when ζ is larger than or equal to 0.1) th collection point, the data collected from the (n+1) th collection point to the (n+1) th collection point is continuously less than 3 collection point is still corrected by using the original correction coefficient when ζ is larger than or equal to 0.1;

b) And cleaning the corrected data.

2. The method for preprocessing electricity consumption data based on a deviation coefficient according to claim 1, wherein Δt/T is greater than or equal to 0.05.

3. The power consumption data preprocessing method based on the deviation coefficient according to claim 2, wherein Δt/T is less than or equal to 0.5.

4. A system for preprocessing electricity data using the electricity data preprocessing method based on the deviation coefficient according to any one of claims 1 to 3, characterized by comprising:

the data acquisition module is used for acquiring and grouping power consumption data;

The data deviation analysis module is used for performing deviation analysis on the electricity consumption data;

The data fluctuation analysis module is used for carrying out fluctuation analysis on the electricity utilization data of two adjacent acquisition points;

the data correction module is used for correcting the electricity consumption data;

The data cleaning module is used for cleaning the corrected electricity consumption data;

The data acquisition module is respectively in communication connection with the data deviation analysis module and the data fluctuation analysis module, the data deviation analysis module and the data fluctuation analysis module are respectively in communication connection with the data correction module, and the data correction module is in communication connection with the data cleaning module.

5. The system of claim 4, further comprising a data sort module, wherein the data skew analysis module and the data surge analysis module are each in communication with the data sort module, and wherein the data sort module is in communication with the data correction module.

6. The system of claim 5, further comprising a data storage module, the data collation module being in communication with the data storage module, the data storage module being in communication with the data correction module.