Attributing Minimum Night Flow to Individual Pipes in Real-World Water Distribution Networks Using Machine Learning ^†

Minimum night flow (MNF) is an important metric commonly used to estimate and understand leakage [1] within district meter areas (DMAs) and is the most common leakage assessment methodology used in the UK [2]. Leakage is a pervasive problem with economic and environmental consequences [3], making it important for water companies, regulators, and governments. One of the main drawbacks to MNF is the resolution of data; even with the complete coverage of smart meters, it is difficult to attribute the remaining water balance to particular pipes. An alternative way to understand leakage is to study known cases of historic leaks and bursts [3], i.e., pipe failures. This relies on the records of utility companies regarding engineering work conducted on the infrastructure. One of the downsides of using historic pipe failures to understand leakage is that this will not include unreported or background leakage (unlike MNF). MNF approximates leakage under the assumption that legitimate water usage is lowest at night, and therefore that most of the flow during this period is leakage.

Previous studies have used machine learning methods to estimate the MNF of entire DMAs based on various factors such as total customers, total pipe length, etc. [4,5]. This paper differs from these previous works by predicting the contribution of individual pipes to MNF. Furthermore, this approach uses data from 800 real-world DMAs that are readily available to water companies, making it applicable to a wide range of real-world scenarios. Finally, by attributing MNF to particular pipes in an explainable manner, this methodology provides more information to decision makers and practitioners, which could contribute to improved leakage assessments, leak localization practices, and sustainable water supply management.

This paper presents a linear regression model that predicts the amount of MNF a pipe is responsible for, henceforth referred to as pipe-MNF. The data used in this study cover the average MNF for these DMAs in September 2023, pipe failure, and pipe asset data. The following asset data were used for each pipe: diameter, age, material (grouped into metal, plastic, and other), number of domestic connections, number of commercial connections, number of hospital connections, and number of agricultural connections. Separate diameter and age features were created for each material (i.e., metal age, plastic age, metal diameter, etc.) to ensure that all features were numeric. This also allowed for the linear regression algorithm to have different coefficients for these various aspects; for example, the model could assign more importance to the age of metal pipes than that of plastic pipes. Pipe failures were also recorded and were loosely defined as any repair or replacement action undertaken by the water utility company that had some leakage component. These actions were then associated with the closest pipe.

The dataset was split by DMAs, with 70% of DMAs in the training set and the remainder in the test set. The same DMA split was used to filter the pipe asset data into training and testing sets. The pipe asset data and MNF data were used to train a linear regression model that predicted the pipe MNF. However, because MNF is observed over an entire DMA, there was no direct way to evaluate the accuracy of the model’s pipe MNF predictions. Therefore, two different validation methods were used: (1) taking the sum of the pipe MNF predictions over a whole DMA and comparing it to the observed MNF, and (2) using the pipe MNF prediction as a prediction of the likelihood that a pipe failure had occurred. The loose definition of pipe failure was used because the model attempted to predict the pipe’s flow contribution to MNF, not burst or leak likelihood. The application of these two validation methods to the linear regression model showed promising results.

Once trained, the resulting linear regression model made pipe MNF predictions such as those shown in Figure 1. This figure shows that the model highlighted a small number of high-risk pipes that contributed significantly more to the MNF of the DMA than most of the other pipes in this DMA. These predictions could then be used to direct or inform further action within the DMA such as leak localization. If a larger area of a DMA was highlighted, it suggested that leak management strategies such as pressure-reducing valves would be useful. Table 1 shows the regression metrics for this model using validation method (1). Figure 2a shows the predictions versus observed values for validation method (1).

Figure 2b shows the ROC curve for the linear regression model’s pipe MNF predictions using validation method (2). The pipe MNF predictions were not modified for this task but were interpreted as a measure of how likely a pipe was to have failed, i.e., a higher pipe MNF meant a higher likelihood of pipe failure. This was based on the assumption that MNF approximates, or is proportional to, leakage, and that pipe failure is a direct measure of where leakage has been found. Figure 2b indicates that the pipe MNF predictions have predictive power for pipe failure and show good results in comparison with those of other studies [6], which further validates the pipe MNF predictions that the linear regression model made. Although validation method (2) and Figure 2b only give an indication of accuracy with regard to reported leaks and bursts, in combination with the regression results from Table 1 and Figure 2a, they suggest that the pipe MNF predictions are reasonably accurate. Table 2 shows the coefficients of the linear regression model, which correspond to the L/h increase in MNF for each feature. This table clearly shows the large and expected impact of different types of consumers on MNF. In addition, according to this model, plastic pipes, per mm in diameter, have a lower impact on MNF than metal pipes do, but have a higher impact with increasing years of age.

This paper presents a model that predicts the contribution of individual pipes to MNF and that achieved good results using two validation methods, showing that the predictions are accurate. By using a linear regression model, the impact of each feature on the final prediction was determined and could be explored and explained. Predictions from this model could be used to direct or inform leak management strategies for water utilities, and by understanding leakage through pipe-level MNF estimates, water utility companies will be able to make better and more informed decisions on how and where to tackle leakage.