CN112632677B - Bridge full-life maintenance strategy optimization method based on half Markov decision process - Google Patents

Abstract

The invention discloses a method for optimizing a bridge full-life maintenance strategy based on a half Markov decision process, which comprises the following steps: s1, determining the annual failure probability of the bridge and the corresponding reliability index, and defining the bridge state according to the reliability index; s2, only considering the reliability index degradation caused by corrosion, and setting the degradation process to accord with the gamma process; s3, calculating the failure probability of the bridge in the decision-making interval; s4, performing health detection on the bridge every year, judging the degradation condition of the bridge protection layer, determining the state of the bridge, and determining the adopted decision according to the state of the bridge, wherein the decision problem adopts a half Markov decision process model; and S5, solving the half Markov decision process model to obtain the optimal full-life maintenance strategy of the bridge. The method is used for uniformly optimizing a preventive maintenance strategy and a necessary maintenance strategy based on the reliability index of the bridge, and meanwhile, the randomness in the performance degradation process of the bridge and the time-varying reliability of the bridge in a decision-making interval are considered.

Description

Optimization method of bridge life-cycle maintenance strategy based on semi-Markov decision process

technical field

The invention relates to the technical field of bridge engineering, in particular to a method for optimizing a bridge life-cycle maintenance strategy based on a semi-Markov decision process.

Background technique

Bridges will inevitably be affected by various environmental factors during their service, such as carbonization, chloride ion erosion, and vehicle fatigue loads, resulting in a gradual decrease in strength, thereby affecting safety. If not maintained properly, old bridges are prone to collapse under extreme vehicle loads, causing serious social impacts. On the other hand, over-maintenance will cause unnecessary waste of resources. Therefore, how to optimize the life-cycle maintenance strategy of bridges and achieve a trade-off between maintenance costs and collapse risk is an important issue that bridge management departments are concerned about. The maintenance measures of bridges can be divided into two categories: preventive maintenance and essential maintenance. The former is to repair the bridge protective layer to delay the further development of corrosion, and the latter is to restore the bridge to a good state, and the cost is also higher. Currently, commonly used optimization methods treat preventive maintenance strategies and necessary maintenance strategies differently, optimizing both based on time and performance, respectively, rather than optimizing within a unified framework.

SUMMARY OF THE INVENTION

In order to overcome the deficiencies of the above technologies, the present invention provides an optimization method for bridge life-cycle maintenance strategy based on a semi-Markov decision process.

The technical scheme adopted by the present invention to overcome its technical problems is:

An optimization method for bridge life-cycle maintenance strategy based on semi-Markov decision process, including steps:

S1. Based on the annual extreme value distribution of the degraded bridge strength and vehicle load, calculate the annual failure probability of the bridge and the corresponding reliability index, and define the bridge state according to the reliability index;

S2. Only the reliability index degradation caused by corrosion is considered, and the degradation process is set to conform to the gamma process;

S3. The time-varying reliability method is used to calculate the failure probability of the bridge in the decision interval; wherein, the annual failure probability described in step S1 refers to the probability that the bridge will fail within one year, and the failure probability of the bridge in the decision interval refers to the bridge The probability of failure between two adjacent decision moments;

S4. Define the state space of the bridge and the action space of the bridge maintenance, respectively, conduct a health inspection of the bridge every year, judge the degradation of the bridge protective layer and determine the bridge state according to the inspection results, and determine the decision to be taken according to the bridge state. Semi-Markov decision process model;

S5 , using the Q-learning algorithm to solve the semi-Markov decision process model described in step S4 to obtain the optimal full-life maintenance strategy for the bridge.

Further, in the step S1, the method of obtaining the degraded bridge strength is as follows:

Perform health inspection on bridges to obtain data on the degree of bridge corrosion, and then calculate the degraded bridge strength through finite element analysis or section strength formula; the finite element analysis and section bearing capacity formula are both two types of bridge strength calculations Commonly used methods, the former has a large workload but high precision, and the latter is simple and easy to implement but has relatively low precision.

Further, in the step S1, the annual extreme value distribution of the vehicle load is obtained statistically according to the measured data or obtained with reference to the bridge design specification.

Further, in the step S1, calculating the reliability index and defining the bridge state specifically includes the following:

可靠度指标为β，所述可靠度指标β的计算为：In turn, the annual failure probability of the bridge is set as

The reliability index is β, and the calculation of the reliability index β is:

In the above formula, Φ( ) is the standard normal distribution function, and the reliability index degradation coefficient D can be obtained from β:

In the above formula, β ₀ is the initial reliability index of the bridge;

The reliability index degradation coefficient is uniformly dispersed into N equal parts from 0 to 1, so as to obtain N+1 discrete values, corresponding to N+1 bridge states, which are respectively s ₀ , s ₁ , ..., s _N , among which, s ₀ and s _N correspond to the intact state and the state where the reliability index is 0 respectively, and s ₁ to s _N-1 are between the intact state and the state where the reliability index is 0, and the corresponding reliability indexes decrease in turn.

Further, in the step S2, only the reliability index degradation caused by corrosion is considered, and the degradation process is set to conform to the gamma process, that is, the degradation coefficient D(t) satisfies the following conditions:

1) D(0)=0 is established according to probability 1;

2) D(t) is an independent incremental process;

3) For any t ₂ >t ₁ ≥0, D(t ₂ )-D(t ₁ ) obeys a gamma distribution with shape parameter v(t ₂ )-v(t ₁ ) and scale parameter u;

The probability density function of a gamma distribution with shape parameter v and scale parameter u is as follows:

为伽马函数，x为伽马分布概率密度函数的自变量，y为伽马函数中的积分变量；In the above formula,

is the gamma function, x is the independent variable of the probability density function of the gamma distribution, and y is the integral variable in the gamma function;

It is further assumed that the shape parameter increases linearly with time, that is,

v(t)=ζt

In the above formula, ζ is the environmental constant.

Further, in the step S3, it is assumed that the time interval between two adjacent decision-making moments is Δt years, and the reliability indexes of the bridges at the two successive decision-making moments are respectively β ₀ (1-D ₁ ) and β ₀ ( 1-D ₂ ), for the convenience of calculation, suppose that the reliability index in the decision-making interval decreases year by year, then the reliability index of the i-th year is:

为：Therefore, the failure probability of the bridge in Δt years

for:

Further, in the step S4, the state space of the bridge is set to be S={s ₀ ,...,s _N }, and the action space of bridge maintenance is A={a ₀ , a ₁ , a ₂ }, where a ₀ , a ₁ , and a ₂ represent no maintenance, preventive maintenance, and necessary maintenance, respectively; a health inspection is carried out on the bridge every year. When it is found that the protective layer of the bridge has completely failed or the degradation degree of the bridge has increased by one level, it is decided whether to take preventive measures. Maintenance measures or necessary maintenance measures.

Further, in the step S4, the semi-Markov decision process model is as follows:

In the above formula, V _t ^* (s) is the optimal value function, representing the minimum total expected discounted cost after time t; π _t (s) is the maintenance measure determined by the policy π _t and the state s; Π is the full life of the bridge maintain policy space;

s' is the next possible state of the bridge, and its value depends on the current state and the current maintenance measures, specifically: 1) if no maintenance measures are currently taken, s' is one level higher than s but does not exceed the highest level, 2) if If preventive maintenance measures are currently taken, then s'=s, 3) If necessary maintenance measures are currently taken, then s'=s ₀ ;

Δt is the time interval between two adjacent decision moments, and its probability distribution p(Δt|π _t (s)) is determined by the current maintenance measures, specifically: 1) If no maintenance measures are currently taken, then Δt is the bridge The probability distribution of the residence time in the current state is determined by the degradation process of the reliability index. 2) If preventive maintenance measures or necessary maintenance measures are currently taken, Δt is the start time of rust and obeys a normal distribution;

T is the design life of the bridge; λ is the discount rate; c ₀ is the bridge maintenance cost, which is determined by the bridge state and maintenance measures; c ₁ is the expected discounted indirect loss caused by bridge collapse within the time interval Δt, which can be calculated as follows:

In the above formula, C _f is the indirect economic loss caused by bridge failure; i and j are both summation variables representing time.

The beneficial effects of the present invention are:

This method optimizes the preventive maintenance strategy and the necessary maintenance strategy in a unified manner based on the reliability index of the bridge. At the same time, the randomness of the bridge performance degradation process and the time-varying reliability of the bridge in the decision interval are considered, which is convenient for the bridge management department according to the bridge. The optimal maintenance measures are taken according to the detection results of the bridge, so that the sum of the whole-life maintenance cost of the bridge and the expected discounted indirect loss is minimized.

Description of drawings

FIG. 1 is a sample of the degradation process of the bridge reliability index according to the embodiment of the present invention.

FIG. 2 is a flowchart of a Q-learning algorithm according to an embodiment of the present invention.

FIG. 3 is a Q value convergence process according to an embodiment of the present invention.

FIG. 4 shows Q values corresponding to different states and different maintenance measures according to the embodiment of the present invention.

FIG. 5 shows Q values corresponding to different service times and different maintenance measures according to the embodiment of the present invention.

FIG. 6 is a schematic diagram of an optimal full-life maintenance strategy according to an embodiment of the present invention.

Detailed ways

In order to facilitate those skilled in the art to better understand the present invention, the present invention will be described in further detail below with reference to the accompanying drawings and specific embodiments. The following are only exemplary and do not limit the protection scope of the present invention.

In this embodiment, a virtual reinforced concrete simply supported girder bridge is used as an example to describe in detail the method for optimizing the bridge lifetime maintenance strategy based on the semi-Markov decision process according to the present invention.

The parameters are set as follows: the design service life is T = 100 years; the discount rate is λ = 0.05; the initial reliability index of the bridge is β ₀ = 4.5; the number of bridge states is taken as 11, and the corresponding reliability index degradation coefficients D are 0, 10%, 20%, ..., 100%; the initial construction cost of the bridge is taken as C _con = 1×10 ⁷ yuan; it is assumed that the indirect economic loss caused by the collapse of the bridge is three times the initial construction cost of the bridge, that is, C _f = 3 × 10 ⁷ yuan; the cost of preventive maintenance measures is taken as 5% of the initial construction cost of the bridge, that is, C _PM = 5 × 10 ⁵ yuan; the cost of necessary maintenance measures depends on the current state of the bridge, and is calculated as follows:

C _EM =min(1,0.05+0.02D ² )·C _con

The mean and standard deviation of the onset time of corrosion were 15 years and 1 year, respectively; the environmental constants and scale parameters of the corrosion gamma process were ζ=1.0, u=0.01, respectively. As an illustration, Figure 1 shows 100 samples of the degradation process of reliability indicators within 100 years after the bridge was in service. It can be seen that due to the effect of the protective layer, the reliability indicators of the bridge did not degrade in the initial stage of service; with the increase of service time , the reliability index gradually degrades. If no maintenance measures are taken during service, the bridge reliability index may even drop to 0.

The whole-life maintenance strategy is optimized with the Q-learning algorithm of 1×10 ⁵ iterative sequences. The algorithm flow is shown in Figure 2. Both the learning rate α and the greedy parameter ε adopt an exponential decay model, and the decay constant is taken as 1×10 ^-4 . In order to check the convergence of the calculation results, Fig. 3 shows the iterative process of the Q value corresponding to the bridge state-maintenance measure pair (s ₃ , no maintenance) after the bridge has been in service for 50 years. It can be seen that the Q value has converged after about 4×10 ⁴ sequence iterations. Figure 4 shows the Q-values corresponding to the no-maintenance, preventive maintenance and necessary maintenance measures when the bridge is in different states after 50 years of service. By comparing the Q values, the optimal maintenance measures for each state can be determined. Specifically, if the bridge is in state s ₀ , s ₁ or s ₂ , no maintenance action is required; if the bridge is in state s ₃ or s ₄ , preventive maintenance measures should be taken to repair its protective layer; for the remaining states , the necessary maintenance measures should be taken to restore the bridge to a good state. Figure 5 shows the Q value corresponding to no maintenance, preventive maintenance and necessary maintenance measures when the bridge is in a certain state (taking s ₅ as an example) after different periods of service. Since the state s ₅ (refer to Figure 1) may not appear until the bridge has been in service for 50 years, the abscissa in Figure 5 starts from 50 years. Similarly, comparing the Q value can determine the optimal maintenance measures under different service times. Specifically, if the remaining life of the bridge is 30, 40 or 50 years, necessary maintenance measures should be taken to restore it to a good state; if the bridge has only 10 or 20 years of life remaining, only preventive measures should be taken. Maintenance measures repair its protective layer. Finally, Figure 6 presents the complete optimal life-cycle maintenance strategy, where the three colors from light to dark represent no maintenance, preventive maintenance and essential maintenance, respectively. Given any combination of bridge state and service time, the corresponding optimal maintenance measures can be determined.

The above only describes the basic principles and preferred embodiments of the present invention, and those skilled in the art can make many changes and improvements based on the above description, and these changes and improvements should belong to the protection scope of the present invention.

Claims (7)

1. a bridge full-life maintenance strategy optimization method based on a semi-Markov decision process, is characterized in that, comprises the steps: S1. Based on the annual extreme value distribution of the degraded bridge strength and vehicle load, calculate the annual failure probability of the bridge and the corresponding reliability index, and define the bridge state according to the reliability index; S2. Only the reliability index degradation caused by corrosion is considered, and the degradation process is set to conform to the gamma process; S3. Use the time-varying reliability method to calculate the failure probability of the bridge within the decision interval; S4. Define the state space of the bridge and the action space of the bridge maintenance, respectively, conduct a health inspection of the bridge every year, judge the degradation of the bridge protective layer and determine the bridge state according to the inspection results, and determine the decision to be taken according to the bridge state. Semi-Markov decision process model; The semi-Markov decision process model is as follows:

In the above formula,

is the optimal value function, representing the minimum total expected discounted cost after time t; π _t (s) is the maintenance measure determined by the policy π _t and the state s; Π is the bridge life-cycle maintenance strategy space; s' is the next possible state of the bridge, and its value depends on the current state and the current maintenance measures, specifically: 1) if no maintenance measures are currently taken, s' is one level higher than s but does not exceed the highest level, 2) if If preventive maintenance measures are currently taken, then s'=s, 3) If necessary maintenance measures are currently taken, then s'=s ₀ , and s ₀ is the intact state of the bridge; Δt is the time interval between two adjacent decision moments, and its probability distribution p(Δt|π _t (s)) is determined by the current maintenance measures, specifically: 1) If no maintenance measures are currently taken, then Δt is the bridge The probability distribution of the residence time in the current state is determined by the degradation process of the reliability index. 2) If preventive maintenance measures or necessary maintenance measures are currently taken, Δt is the start time of rust and obeys a normal distribution; T is the design life of the bridge; λ is the discount rate; c ₀ is the bridge maintenance cost, which is determined by the bridge state and maintenance measures; c ₁ is the expected discounted indirect loss caused by bridge collapse within the time interval Δt, which can be calculated as follows:

In the above formula, C _f is the indirect economic loss caused by bridge failure; i and j are both summation variables representing time; β _i and β _j represent the reliability index of the i-th and j-th years, respectively; S5 , using the Q-learning algorithm to solve the semi-Markov decision process model described in step S4 to obtain the optimal full-life maintenance strategy for the bridge. 2. The method according to claim 1, wherein, in the step S1, the method of obtaining the degraded bridge strength is as follows: Perform health inspection on bridges to obtain data on the degree of bridge corrosion, and then calculate the degraded bridge strength through finite element analysis or section bearing capacity formula. 3 . The method according to claim 1 , wherein, in the step S1 , the annual extreme value distribution of the vehicle load is obtained statistically according to the measured data or obtained with reference to the bridge design specification. 4 . 4. The method according to claim 1, wherein, in the step S1, calculating the reliability index and defining the bridge state, specifically including the following: In turn, the annual failure probability of the bridge is set as

The reliability index is β, and the calculation of the reliability index β is:

In the above formula, Φ( ) is the standard normal distribution function, and the reliability index degradation coefficient D can be obtained from β:

In the above formula, β ₀ is the initial reliability index of the bridge; The reliability index degradation coefficient is uniformly dispersed into N equal parts from 0 to 1, so as to obtain N+1 discrete values, corresponding to N+1 bridge states, which are respectively s ₀ , s ₁ , ..., s _N , among which, s ₀ and s _N correspond to the intact state and the state where the reliability index is 0 respectively, and s ₁ to s _N-1 are between the intact state and the state where the reliability index is 0, and the corresponding reliability indexes decrease in turn. 5. The method according to claim 4, characterized in that, in the step S2, only the reliability index degradation caused by corrosion is considered, and the degradation process is set to conform to the gamma process, that is, the degradation coefficient D(t) satisfies the following condition: 1) D(0)=0 is established according to probability 1; 2) D(t) is an independent incremental process; 3) For any t ₂ >t ₁ ≥0, D(t ₂ )-D(t ₁ ) obeys a gamma distribution with shape parameter v(t ₂ )-v(t ₁ ) and scale parameter u; The probability density function of a gamma distribution with shape parameter v and scale parameter u is as follows:

In the above formula,

is the gamma function, x is the independent variable of the probability density function of the gamma distribution, and y is the integral variable in the gamma function; It is further assumed that the shape parameter increases linearly with time, that is, v(t)=ζt In the above formula, ζ is the environmental constant. 6. The method according to claim 5, wherein, in the step S3, the time interval between two adjacent decision-making moments is set to be Δt years, and the reliability indexes of the bridges at the two successive decision-making moments are respectively β ₀ (1-D ₁ ) and β ₀ (1-D ₂ ), assuming that the reliability index in the decision-making interval decreases year by year, the reliability index of the i-th year is:

Therefore, the failure probability of the bridge in Δt years

for:

7. The method according to claim 6, wherein in the step S4, the state space of the bridge is set to be S={s ₀ ,...,s _N }, and the action space of the bridge maintenance is A={a ₀ , a ₁ , a ₂ }, where a ₀ , a ₁ , and a ₂ represent no maintenance, preventive maintenance, and necessary maintenance, respectively; a health inspection is carried out on the bridge every year. If the level is increased by one level, it is determined whether preventive maintenance measures or necessary maintenance measures are required.

Images