CN107330294A - The application process of many hidden layer extreme learning machines of online sequential with forgetting factor - Google Patents

Abstract

The present invention relates to a kind of application process of many hidden layer extreme learning machines of the online sequential with forgetting factor, comprise the following steps：1) an extreme learning machine model with many hidden layers is sought, the output expression formula of many hidden layer extreme learning machine models is obtained；2) real-time update is carried out to above-mentioned many hidden layer extreme learning machine models, the expression formula of model after output updates.The present invention handles the data variation of batch process using the method for many hidden layer extreme learning machines of the online sequential with forgetting factor, method can adjust model according to the change in data structure, depth optimization can also be carried out to model parameter, reach more preferable effect, it is ensured that final hiding output hides output closer to expected.

Description

Application method of online sequential multi-hidden layer extreme learning machine with forgetting factor

technical field

The invention relates to a learning machine, in particular to an application method of an online sequential multi-hidden layer extreme learning machine with forgetting factors.

Background technique

Due to the strong nonlinearity and coupling between the variables of the intermittent process, and many real-time data change with time, it has strong timeliness, while the traditional single-hidden-layer extreme learning machine or multi-hidden-layer extreme learning machine cannot It is based on the established model to predict the output in advance, and the model cannot be adjusted according to the change in the data structure. The model is too rigid; the extreme learning machine with a single hidden layer is not thorough enough to optimize the model structural parameters, and cannot effectively reduce noise. interference, there is no guarantee that the final hidden output is closer to the expected hidden output. Therefore, it is necessary to use an integrated online sequential single hidden layer extreme learning machine (EOS-ELM), or an integrated online sequential single hidden layer extreme learning machine with a forgetting mechanism (FOS-ELM), which can adjust the model according to changes in the data structure Changes to adapt to different time stages to achieve better results; because real-time changing data has a series of unavoidable noise signal interference.

Currently, no online sequential learning machine that can meet the above requirements has been reported.

Contents of the invention

Aiming at the disadvantages of single hidden layer extreme learning machine or multi-hidden layer extreme learning machine that cannot adjust model changes according to changes in data structure in the prior art, the problem to be solved by the present invention is to provide a It is an application method of an online time-series multi-hidden layer extreme learning machine with forgetting factor to adjust the model by changing the model parameters in depth.

In order to solve the problems of the technologies described above, the technical solution adopted in the present invention is:

An application method of an online time-series multi-hidden layer extreme learning machine with a forgetting factor of the present invention comprises the following steps:

1) seek an extreme learning machine model with multiple hidden layers, and obtain the output expression of the extreme learning machine model with multiple hidden layers;

2) The above-mentioned multi-hidden layer extreme learning machine model is updated in real time, and the expression of the updated model is output.

In step 1), seek an extreme learning machine model with multiple hidden layers, and obtain the output expression of the extreme learning machine model with multiple hidden layers, specifically:

11) Given a sample and a network structure of multiple hidden layers, the activation function of the hidden layer is g, and the network output is g(a,b,X), where a is the distance between the input layer and the first hidden layer Weight, b is the bias of the first hidden layer, X is the input matrix;

12) Assuming that the data changes in batches, and each batch lasts for S units of time, the data of the k-th unit time is expressed as N _j is the number of j batches of data; χ _k is valid within the range of [k k+s], j=0,1,...k., t _i is a flag variable, at (k+1)-th The data of unit time is expressed as k is any large positive integer, _xi is the input sample, t _i is the sample mark, and th is the batch.

13) Suppose k≥s-1, the number of training data is much larger than the number of hidden layer nodes, Z _k+1 is the predicted result of the (k+1)-th unit time, set l=k-s+ 1, k-s+2,...k; the output of the first hidden layer of the network at the l-th moment of the data is:

(a _i , b _i ) is the weight and threshold between the input layer and the first hidden layer, i=1,...L. Random initialization; G is the hidden layer activation function, T is [k- s+1, k] inner batch data sample mark amount, T ₁ is the mark amount of the lth batch data sample; l is a positive integer in [k-s+1, k];

The output weight β of the final hidden layer is obtained as:

and

14) Suppose the weight and bias of the second hidden layer are W ₁ , B ₁ , then the output of the second hidden layer is:

15) Assuming W _HE = [B ₁ W ₁ ], then the weight and bias of the second hidden layer are calculated by And suppose H _E =[1 H] ^T , 1 is a one-dimensional row vector whose elements are all 1, g ^-1 (x) is the inverse function of g(x) of the activation function, W _HE and HE are assumed variables;

16) Update the output of the second hidden layer as H ₂ =g(W _HE H _E ); update the output weight β of the final hidden layer as

17) Suppose the weight and bias of the third hidden layer are W ₂ , B ₂ , then the output of the third hidden layer is

18) Assuming W _HE1 = [B ₂ W ₂ ], then the weight and bias of the third hidden layer are calculated by And H _E1 =[1 H ₂ ] ^T , 1 is a one-dimensional row vector whose elements are all 1; g ^-1 (x) is the inverse function of g(x) of the activation function; W _HE1 and HE1 are hypothetical variables;

The output of the updated third hidden layer is:

H ₄ ＝g(W _HE1 H _E1 ) (2)

19) Update the output weight β of the final hidden layer as:

Then the final output is f=β _new1 H ₄ .

For a network structure with multiple hidden layers, loop iteration formulas (1), (2), (3), three hidden layers iterate once, four hidden layers iterate twice, and N hidden layers iterate N-2 times , so that β _new = β _new1 , H ₂ = H ₄ at the end of each iteration.

In step 2), the above multi-hidden layer extreme learning machine model is updated in real time, and the expression of the updated model is output, specifically:

21) Assuming that Z _k+2 is the predicted result of the (k+2)-th unit time, when the result of the (k+1)-th time is known, keep the same (a _i ,b _i ),i=1,...L, the output weight is expressed as:

and Then P _k+1 is expressed by P _k as:

and

but

22) Suppose the weight and bias of the second hidden layer are W ₁ , B ₁ , then the output of the second hidden layer is

23) Assuming W _HE = [B ₁ W ₁ ], then the weight and bias of the second hidden layer are calculated by And H _E ＝[1 H] ^T , 1 is a one-dimensional row vector whose elements are all 1; g ^-1 (x) is the inverse function of g(x) of the activation function,

24) Update the output of the second hidden layer as H ₂ =g(W _HE H _E ); update the output weight β of the final hidden layer as

25) Suppose the weight and bias of the third hidden layer are W ₂ , B ₂ , then the output of the third hidden layer is

26) Assuming W _HE1 = [B ₂ W ₂ ], then the weight and bias of the third hidden layer are calculated by And H _E1 =[1 H ₂ ] ^T , 1 is a one-dimensional row vector whose elements are all 1, and g ^-1 (x) is the inverse function of g(x) of the activation function;

27) Update the output of the third hidden layer as: H ₄ =g(W _HE1 H _E1 ) (5)

28) Update the output weight β of the final hidden layer as:

Then the final output is f=β _new1 H ₄ .

The present invention also includes the following steps:

For a network structure with multiple hidden layers, formulas (4), (5), and (6) are iterated cyclically, three hidden layers are iterated once, four hidden layers are iterated twice, and N hidden layers are iterated N-2 times , each iteration ends, so that β _new = β _new1 , H ₂ = H ₄ .

The present invention has the following beneficial effects and advantages:

1. The present invention adopts the method of online sequential multi-hidden layer extreme learning machine with forgetting factor to process the data change of the intermittent process, the method can adjust the model according to the change in the data structure, and can also carry out deep optimization to the model parameters, Achieving better results ensures that the final hidden output is closer to the expected hidden output.

Description of drawings

Figure 1 shows the root mean square error value of the training set when each batch of data has only 2 valid times under the FOS-ELM model and the FOS-MELM model;

Figure 2 shows the root mean square error value of the test set under the FOS-ELM model and the FOS-MELM model when each batch of data has only 2 valid times;

Figure 3 shows the root mean square error value of the training set when each batch of data has only 3 valid times under the FOS-ELM model and the FOS-MELM model;

Figure 4 shows the root mean square error value of the test set under the FOS-ELM model and the FOS-MELM model when each batch of data has only 3 valid times.

detailed description

The present invention will be further elaborated below in conjunction with the accompanying drawings of the description.

In order to make the training output closer to the actual output, considering that the optimization of the model structure parameters by the single hidden layer extreme learning machine is not thorough enough to effectively reduce the noise interference, so in order to ensure that the final hidden output is closer to the expected hidden output, Combined with the previous advantageous results of improving the extreme learning machine, the present invention uses a method of online sequential multi-hidden layer extreme learning machine with forgetting factor to deal with the data changes of the intermittent process. To adjust the model, you can also deeply optimize the model parameters to achieve better results.

The application method of the online sequential multi-hidden layer extreme learning machine with forgetting factor of the present invention comprises the following steps:

1) seek an extreme learning machine model with multiple hidden layers, and obtain the output expression of the extreme learning machine model with multiple hidden layers;

2) The above-mentioned multi-hidden layer extreme learning machine model is updated in real time, and the expression of the updated model is output.

This embodiment uses an ELM network structure with three hidden layers as an example to analyze the algorithm steps of an online sequential multi-hidden layer extreme value learning machine (FOS-MELM) with a forgetting mechanism. In step 1), seek an extreme learning machine model with multiple hidden layers, and obtain the output expression of the extreme learning machine model with multiple hidden layers, specifically:

11) First, given the sample and the network structure of three hidden layers (each hidden layer contains L nodes), the activation function of the hidden layer is selected as g, so the network output is g(a,b,X), where a is the weight between the input layer and the first hidden layer, b is the bias of the first hidden layer, and X is the input matrix;

12) Assume that the data changes batch by batch, and each batch lasts for S unit time. So the data of the k-th unit time can be expressed as And N _j is the number of j batches of data, χ _k is valid within the range of [kk+s] j=0,1,...k., t _i is a flag variable. So the data of the (k+1)-th unit time can be expressed as

13) Suppose k≥s-1, the number of training data is much larger than the number of hidden layer nodes, Z _k+1 is the predicted result of the (k+1)-th unit time, set l=k-s+ 1,k-s+2,...k. The output of the first hidden layer of the network at the l-th moment of the data is:

(a _i , b _i ) are the weights and thresholds between the input layer and the first hidden layer, i=1,...L. are randomly initialized; the output weight β of the final hidden layer can be obtained for:

and

24) Suppose the weight and bias of the second hidden layer are W ₁ , B ₁ , then the output of the second hidden layer is:

15) Assuming W _HE = [B ₁ W ₁ ], then the weight and bias of the second hidden layer are calculated by And H _E =[1 H] ^T , 1 is a one-dimensional row vector whose elements are all 1. g ^-1 (x) is the inverse function of g(x) of the activation function,

16) Update the output of the second hidden layer as H ₂ =g(W _HE H _E ); update the output weight β of the final hidden layer as

17) Now suppose the weight and bias of the third hidden layer are W ₂ , B ₂ , then the output of the third hidden layer is

18) Assuming W _HE1 = [B ₂ W ₂ ], then the weight and bias of the third hidden layer are calculated by And H _E1 =[1 H ₂ ] ^T , 1 is a one-dimensional row vector whose elements are all 1; g ^-1 (x) is the inverse function of g(x) of the activation function; the output of updating the third hidden layer is H ₄ =g(W _HE1 H _E1 )(2)

19) Update the output weight β of the final hidden layer as

Then the final output is f=β _new1 H ₄ .

For a network structure with multiple hidden layers, loop iteration formulas (1), (2), (3), three hidden layers iterate once, four hidden layers iterate twice, and N hidden layers iterate N-2 times , so that β _new = β _new1 , H ₂ = H ₄ at the end of each iteration.

In step 2), the above multi-hidden layer extreme learning machine model is updated in real time, and the expression of the updated model is output, specifically:

21) Assuming that Z _k+2 is the predicted result of the (k+2)-th unit time, when the result of the (k+1)-th time is known, keep the same (a _i ,b _i ),i=1,...L; the output weight can be expressed as:

and Then P _k+1 can be expressed by P _k as

and

but

22) Now suppose the weight and bias of the second hidden layer are W ₁ , B ₁ , then the output of the second hidden layer is

23) Now suppose W _HE =[B ₁ W ₁ ], then the weight and bias of the second hidden layer are calculated by And H _E ＝[1 H] ^T , 1 is a one-dimensional row vector whose elements are all 1, g ^-1 (x) is the inverse function of g(x) of the activation function,

24) Update the output of the second hidden layer as H ₂ =g(W _HE H _E ); update the output weight β of the final hidden layer as

25) Now suppose the weight and bias of the third hidden layer are W ₂ , B ₂ , then the output of the third hidden layer is

26) Now suppose W _HE1 = [B ₂ W ₂ ], then the weight and bias of the third hidden layer are calculated by And H _E1 =[1 H ₂ ] ^T , 1 is a one-dimensional row vector whose elements are all 1. g ^-1 (x) is the inverse function of g(x) of the activation function;

27) Update the output of the third hidden layer as H ₄ =g(W _HE1 H _E1 ) (5)

28) Update the output weight β of the final hidden layer as

Then the final output is f=β _new1 H ₄ .

For a network structure with multiple hidden layers, formulas (4), (5), and (6) are iterated cyclically, three hidden layers are iterated once, four hidden layers are iterated twice, and N hidden layers are iterated N-2 times , each iteration ends, so that β _new = β _new1 , H ₂ = H ₄ .

This embodiment uses the polymerization reactor of styrene as the research object. The production purpose of this reactor is to adjust the temperature of the reactor so that the conversion rate of the reaction, the number-average chain length and the weight-average chain length of the reaction product are at the end of the reaction. close to optimum. The mechanistic model of the reaction is as follows:

Table 3.2 Nominal parameter values

In the formula, T represents the absolute temperature of the reactor, Tc is the temperature in Celsius, Aw and B are the correlation coefficients between the weight-average chain length and temperature, Am and Em are the frequency factor and activation energy of the monomer polymerization reaction, and r1-r4 is the density - temperature correction value, Mm and χ are monomer molecular weight and polymer interaction parameters, respectively. The quality index of the product includes the conversion rate y ₁ , the dimensionless number average chain length y ₂ , and the dimensionless weight average chain length y ₃ , that is, the final quality index is: y=[y ₁ (t _f )y ₂ (t _f )y ₃ (t _f )], t _f represents the end point. Assuming that the total reaction time is set to 400 minutes, the control variable is the reactor temperature T, which is divided into 20 intervals according to the time, and the temperature in each section is kept constant, that is, the process variable is X=[T ₁ T ₂ T ₃ ... T ₂₀ ], the parameters in the equation are given in Table 3.2. Generate 60 batches of data X (60×20), and implement it on the reactor to obtain a quality variable matrix Y (60×3). The first 20 data are taken as training samples to establish the initial model; the middle 20 samples are used as update samples; the last 20 data are used as test samples to test the prediction effect of the model. The activation function is selected as the sigmoid function before modeling.

As shown in Figures 1 to 4, Figures 1 and 2 respectively show the root mean square error values of the training set and the test set under two different models (FOS-ELM and FOS-MELM) and different hidden layer nodes; Figures 3 and 4 respectively show that when each batch of data has only 3 valid times, the root mean square error values of the training set and the test set are in two different models Lower (FOS-ELM and FOS-MELM) and different hidden layer nodes.

The above illustration shows that due to the timeliness of the parameter changes of the polymerization reactor, in order to effectively predict the temperature value, we use this method to update the system model online in time series, reducing or even avoiding the impact of drastic data changes on the model and prediction output.

Claims (5)

1. an application method of online sequential multi-hidden layer extreme learning machine with forgetting factor, it is characterized in that comprising the following steps: 1) seek an extreme learning machine model with multiple hidden layers, and obtain the output expression of the extreme learning machine model with multiple hidden layers; 2) The above-mentioned multi-hidden layer extreme learning machine model is updated in real time, and the expression of the updated model is output. 2. the application method of the online sequential multi-hidden layer extreme learning machine of band forgetting factor according to claim 1, it is characterized in that: In step 1), seek an extreme learning machine model with multiple hidden layers, and obtain the output expression of the extreme learning machine model with multiple hidden layers, specifically: 11) Given a sample and a network structure of multiple hidden layers, the activation function of the hidden layer is g, and the network output is g(a,b,X), where a is the distance between the input layer and the first hidden layer Weight, b is the bias of the first hidden layer, X is the input matrix; 12) Assuming that the data changes in batches, and each batch lasts for S units of time, the data of the k-th unit time is expressed as N _j is the number of j batches of data; χ _k is valid within the range of [k k+s], j=0,1,...k., t _i is a flag variable, at (k+1)-th The data of unit time is expressed as k is any large positive integer, _xi is the input sample, t _i is the sample mark, and th is the batch. 13) Suppose k≥s-1, the number of training data is much larger than the number of hidden layer nodes, Z _k+1 is the predicted result of the (k+1)-th unit time, set l=k-s+ 1, k-s+2,...k; the output of the first hidden layer of the network at the l-th moment of the data is: (a _i , b _i ) is the weight and threshold between the input layer and the first hidden layer, i=1,...L. Random initialization; (G is the activation function of the hidden layer, T is [k -s+1, k] inner batch data sample mark amount, T _l is the mark amount of the lth batch data sample; l is a positive integer in [k-s+1, k]; The output weight β of the final hidden layer is obtained as: <mrow><mi>&amp;beta;</mi><mrow><mo>(</mo><mi>k</mi><mo>)</mo></mrow><mo>=</mo><msup><mfenced open = "[" close = "]"><mtable><mtr><mtd><msub><mi>H</mi><mrow><mi>k</mi><mo>-</mo><mi>s</mi><mo>+</mo><mn>1</mn></mrow></msub></mtd></mtr><mtr><mtd><mo>.</mo></mtd></mtr><mtr><mtd><mo>.</mo></mtd></mtr><mtr><mtd><mo>.</mo></mtd></mtr><mtr><mtd><msub><mi>H</mi><mi>k</mi></msub></mtd></mtr></mtable></mfenced><mo>+</mo></msup><mfenced open = "[" close = "]"><mtable><mtr><mtd><msub><mi>T</mi><mrow><mi>k</mi><mo>-</mo><mi>s</mi><mo>+</mo><mn>1</mn></mrow></msub></mtd></mtr><mtr><mtd><mo>.</mo></mtd></mtr><mtr><mtd><mo>.</mo></mtd></mtr><mtr><mtd><mo>.</mo></mtd></mtr><mtr><mtd><msub><mi>T</mi><mi>k</mi></msub></mtd></mtr></mtable></mfenced><mo>=</mo><msub><mi>P</mi><mi>k</mi></msub><msup><mfenced open = "[" close = "]"><mtable><mtr><mtd><msub><mi>H</mi><mrow><mi>k</mi><mo>-</mo><mi>s</mi><mo>+</mo><mn>1</mn></mrow></msub></mtd></mtr><mtr><mtd><mo>.</mo></mtd></mtr><mtr><mtd><mo>.</mo></mtd></mtr><mtr><mtd><mo>.</mo></mtd></mtr><mtr><mtd><msub><mi>H</mi><mi>k</mi></msub></mtd></mtr></mtable></mfenced><mi>T</mi></msup><mfenced open = "[" close = "]"><mtable><mtr><mtd><msub><mi>T</mi><mrow><mi>k</mi><mo>-</mo><mi>s</mi><mo>+</mo><mn>1</mn></mrow></msub></mtd></mtr><mtr><mtd><mo>.</mo></mtd></mtr><mtr><mtd><mo>.</mo></mtd></mtr><mtr><mtd><mo>.</mo></mtd></mtr><mtr><mtd><msub><mi>T</mi><mi>k</mi></msub></mtd></mtr></mtable></mfenced></mrow> and 14) Suppose the weight and bias of the second hidden layer are W ₁ , B ₁ , then the output of the second hidden layer is: <mrow><msub><mi>H</mi><mn>1</mn></msub><mo>=</mo><mfenced open = "[" close = "]"><mtable><mtr><mtd><msub><mi>T</mi><mrow><mi>k</mi><mo>-</mo><mi>s</mi><mo>+</mo><mn>1</mn></mrow></msub></mtd></mtr><mtr><mtd><mo>.</mo></mtd></mtr><mtr><mtd><mo>.</mo></mtd></mtr><mtr><mtd><mo>.</mo></mtd></mtr><mtr><mtd><msub><mi>T</mi><mi>k</mi></msub></mtd></mtr></mtable></mfenced><mi>&amp;beta;</mi><msup><mrow><mo>(</mo><mi>k</mi><mo>)</mo></mrow><mo>+</mo></msup><mo>;</mo></mrow> 15) Assuming W _HE = [B ₁ W ₁ ], then the weight and bias of the second hidden layer are calculated by And suppose H _E =[1 H] ^T , 1 is a one-dimensional row vector whose elements are all 1, g ^-1 (x) is the inverse function of g(x) of the activation function, W _HE and HE are assumed variables; 16) Update the output of the second hidden layer as H ₂ =g(W _HE H _E ); update the output weight β of the final hidden layer as 17) Suppose the weight and bias of the third hidden layer are W ₂ , B ₂ , then the output of the third hidden layer is <mrow><msub><mi>H</mi><mn>3</mn></msub><mo>=</mo><mfenced open = "[" close = "]"><mtable><mtr><mtd><msub><mi>T</mi><mrow><mi>k</mi><mo>-</mo><mi>s</mi><mo>+</mo><mn>1</mn></mrow></msub></mtd></mtr><mtr><mtd><mo>.</mo></mtd></mtr><mtr><mtd><mo>.</mo></mtd></mtr><mtr><mtd><mo>.</mo></mtd></mtr><mtr><mtd><msub><mi>T</mi><mi>k</mi></msub></mtd></mtr></mtable></mfenced><msubsup><mi>&amp;beta;</mi><mrow><mi>n</mi><mi>e</mi><mi>w</mi></mrow><mo>+</mo></msubsup><mo>-</mo><mo>-</mo><mo>-</mo><mrow><mo>(</mo><mn>1</mn><mo>)</mo></mrow></mrow> 18) Assuming W _HE1 = [B ₂ W ₂ ], then the weight and bias of the third hidden layer are calculated by And H _E1 =[1 H ₂ ] ^T , 1 is a one-dimensional row vector whose elements are all 1; g ^-1 (x) is the inverse function of g(x) of the activation function; W _HE1 and HE1 are hypothetical variables; The output of the updated third hidden layer is: H ₄ ＝g(W _HE1 H _E1 ) (2) 19) Update the output weight β of the final hidden layer as: <mrow><msub><mi>&amp;beta;</mi><mrow><mi>n</mi><mi>e</mi><mi>w</mi><mn>1</mn></mrow></msub><mo>=</mo><msubsup><mi>H</mi><mn>4</mn><mo>+</mo></msubsup>< mfenced open = "[" close = "]"><mtable><mtr><mtd><msub><mi>T</mi><mrow><mi>k</mi><mo>-</mo><mi>s</mi><mo>+</mo><mn>1</mn></mrow></msub></mtd></mtr><mtr><mtd><mo>.</mo></mtd></mtr><mtr><mtd><mo>.</mo></mtd></mtr><mtr><mtd><mo>.</mo></mtd></mtr><mtr><mtd><msub><mi>T</mi><mi>k</mi></msub></mtd></mtr></mtable></mfenced><mo>-</mo><mo>-</mo><mo>-</mo><mrow><mo>(</mo><mn>3</mn><mo>)</mo></mrow></mrow> Then the final output is f=β _new1 H ₄ . 3. the application method of the online sequential multi-hidden layer extreme learning machine of band forgetting factor according to claim 2, it is characterized in that: For a network structure with multiple hidden layers, loop iteration formulas (1), (2), (3), three hidden layers iterate once, four hidden layers iterate twice, and N hidden layers iterate N-2 times , so that β _new = β _new1 , H ₂ = H ₄ at the end of each iteration. 4. the application method of the online sequential multi-hidden layer extreme learning machine of band forgetting factor according to claim 1, it is characterized in that: In step 2), the above multi-hidden layer extreme learning machine model is updated in real time, and the expression of the updated model is output, specifically: 21) Assuming that Z _k+2 is the predicted result of the (k+2)-th unit time, when the result of the (k+1)-th time is known, keep the same (a _i ,b _i ),i=1,...L, the output weight is expressed as: <mrow><mfenced open = "[" close = "]"><mtable><mtr><mtd><msub><mi>H</mi><mrow><mi>k</mi><mo>-</mo><mi>s</mi><mo>+</mo><mn>2</mn></mrow></msub></mtd></mtr><mtr><mtd><mo>.</mo></mtd></mtr><mtr><mtd><mo>.</mo></mtd></mtr><mtr><mtd><mo>.</mo></mtd></mtr><mtr><mtd><msub><mi>H</mi><mrow><mi>k</mi><mo>+</mo><mn>1</mn></mrow></msub></mtd></mtr></mtable></mfenced><mi>&amp;beta;</mi><mo>=</mo><mfenced open = "[" close = "]"><mtable><mtr><mtd><msub><mi>T</mi><mrow><mi>k</mi><mo>-</mo><mi>s</mi><mo>+</mo><mn>2</mn></mrow></msub></mtd></mtr><mtr><mtd><mo>.</mo></mtd></mtr><mtr><mtd><mo>.</mo></mtd></mtr><mtr><mtd><mo>.</mo></mtd></mtr><mtr><mtd><msub><mi>T</mi><mrow><mi>k</mi><mo>+</mo><mn>1</mn></mrow></msub></mtd></mtr></mtable></mfenced><mi>a</mi><mi>n</mi><mi>d</mi><mi>&amp;beta;</mi><mrow><mo>(</mo><mi>k</mi><mo>+</mo><mn>1</mn><mo>)</mo></mrow><mo>=</mo><msup><mfenced open = "[" close = "]"><mtable><mtr><mtd><msub><mi>H</mi><mrow><mi>k</mi><mo>-</mo><mi>s</mi><mo>+</mo><mn>2</mn></mrow></msub></mtd></mtr><mtr><mtd><mo>.</mo></mtd></mtr><mtr><mtd><mo>.</mo></mtd></mtr><mtr><mtd><mo>.</mo></mtd></mtr><mtr><mtd><msub><mi>H</mi><mrow><mi>k</mi><mo>+</mo><mn>1</mn></mrow></msub></mtd></mtr></mtable></mfenced><mo>+</mo></msup><mfenced open = "[" close = "]"><mtable><mtr><mtd><msub><mi>T</mi><mrow><mi>k</mi><mo>-</mo><mi>s</mi><mo>+</mo><mn>2</mn></mrow></msub></mtd></mtr><mtr><mtd><mo>.</mo></mtd></mtr><mtr><mtd><mo>.</mo></mtd></mtr><mtr><mtd><mo>.</mo></mtd></mtr><mtr><mtd><msub><mi>T</mi><mrow><mi>k</mi><mo>+</mo><mn>1</mn></mrow></msub></mtd></mtr></mtable></mfenced><mo>=</mo><msub><mi>P</mi><mrow><mi>k</mi><mo>+</mo><mn>1</mn></mrow></msub><msup><mfenced open = "[" close = "]"><mtable><mtr><mtd><msub><mi>H</mi><mrow><mi>k</mi><mo>-</mo><mi>s</mi><mo>+</mo><mn>2</mn></mrow></msub></mtd></mtr><mtr><mtd><mo>.</mo></mtd></mtr><mtr><mtd><mo>.</mo></mtd></mtr><mtr><mtd><mo>.</mo></mtd></mtr><mtr><mtd><msub><mi>H</mi><mrow><mi>k</mi><mo>+</mo><mn>1</mn></mrow></msub></mtd></mtr></mtable></mfenced><mi>T</mi></msup><mfenced open = "[" close = "]"><mtable><mtr><mtd><msub><mi>T</mi><mrow><mi>k</mi><mo>-</mo><mi>s</mi><mo>+</mo><mn>2</mn></mrow></msub></mtd></mtr><mtr><mtd><mo>.</mo></mtd></mtr><mtr><mtd><mo>.</mo></mtd></mtr><mtr><mtd><mo>.</mo></mtd></mtr><mtr><mtd><msub><mi>T</mi><mrow><mi>k</mi><mo>+</mo><mn>1</mn></mrow></msub></mtd></mtr></mtable></mfenced><mo>-</mi>mo><mo>-</mo><mo>-</mo><mrow><mo>(</mo><mn>35</mn><mo>)</mo></mrow></mrow> and Then P _k+1 is expressed by P _k as: <mfenced open = "" close = ""><mtable><mtr><mtd><mrow><msub><mi>P</mi><mrow><mi>k</mi><mo>+</mo><mn>1</mn></mrow></msub><mo>=</mo><msup><mrow><mo>(</mo><munderover><mo>&amp;Sigma;</mo><mrow><mi>l</mi><mo>=</mo><mi>k</mi><mo>-</mo><mi>s</mi><mo>+</mo><mn>2</mn></mrow><mrow><mi>k</mi><mo>+</mo><mn>1</mn></mrow></munderover><msubsup><mi>H</mi><mi>l</mi><mi>T</mi></msubsup><msub><mi>H</mi><mi>l</mi></msub><mo>)</mo></mrow><mrow><mo>-</mo><mn>1</mn></mrow></msup><mo>=</mo><msup><mrow><mo>(</mo><mrow><munderover><mo>&amp;Sigma;</mo><mrow><mi>l</mi><mo>=</mo><mi>k</mi><mo>-</mo><mi>s</mi><mo>+</mo><mn>1</mn></mrow><mrow><mi>k</mi><mo>+</mo><mn>1</mn></mrow></munderover><msubsup><mi>H</mi><mi>l</mi><mi>T</mi></msubsup><msub><mi>H</mi><mi>l</mi></msub><mo>+</mo><msup><mfenced open = "[" close = "]"><mtable><mtr><mtd><mrow><mo>-</mo><msub><mi>H</mi><mrow><mi>k</mi><mo>-</mo><mi>s</mi><mo>+</mo><mn>1</mi>mn></mrow></msub></mrow></mtd></mtr><mtr><mtd><msub><mi>H</mi><mrow><mi>k</mi><mo>+</mo><mn>1</mn></mrow></msub></mtd></mtr></mtable></mfenced><mi>T</mi></msup><mfenced open = "[" close = "]"><mtable><mtr><mtd><msub><mi>H</mi><mrow><mi>k</mi><mo>-</mo><mi>s</mi><mo>+</mo><mn>1</mn></mrow></msub></mtd></mtr><mtr><mtd><msub><mi>H</mi><mrow><mi>k</mi><mo>+</mo><mn>1</mn></mrow></msub></mtd></mtr></mtable></mfenced></mrow><mo>)</mo></mrow><mrow><mo>-</mo><mn>1</mn></mrow></msup></mrow></mtd></mtr><mtr><mtd><mrow><mo>=</mo><msub><mi>P</mi><mi>k</mi></msub><mo>-</mo><msub><mi>P</mi><mi>k</mi></msub><msup><mfenced open = "[" close = "]"><mtable><mtr><mtd><mrow><mo>-</mo><msub><mi>H</mi><mrow><mi>k</mi><mo>-</mo><mi>s</mi><mo>+</mo><mn>1</mn></mrow></msub></mrow></mtd></mtr><mtr><mtd><msub><mi>H</mi><mrow><mi>k</mi><mo>+</mo><mn>1</mn></mrow></msub></mtd></mtr></mtable></mfenced><mi>T</mi></msup><msup><mrow><mo>(</mo><mrow><mi>I</mi><mo>+</mo><mfenced open = "[" close = "]"><mtable><mtr><mtd><mrow><mo>-</mo><msub><mi>H</mi><mrow><mi>k</mi><mo>-</mo><mi>s</mi><mo>+</mo><mn>1</mn></mrow></msub></mrow></mtd></mtr><mtr><mtd><msub><mi>H</mi><mrow><mi>k</mi><mo>+</mo><mn>1</mn></mrow></msub></mtd></mtr></mtable></mfenced><msub><mi>P</mi><mi>k</mi></msub><msup><mfenced open = "[" close = "]"><mtable><mtr><mtd><mrow><mo>-</mo><msub><mi>H</mi><mrow><mi>k</mi><mo>-</mo><mi>s</mi><mo>+</mo><mn>1</mn></mrow></msub></mrow></mtd></mtr><mtr><mtd><msub><mi>H</mi><mrow><mi>k</mi><mo>+</mo><mn>1</mn></mrow></msub></mtd></mtr></mtable></mfenced><mi>T</mi></msup></mrow><mo>)</mo></mrow><mrow><mo>-</mo><mn>1</mn></mrow></msup><mo>&amp;times;</mo><mfenced open = "[" close = "]"><mtable><mtr><mtd><msub><mi>H</mi><mrow><mi>k</mi><mo>-</mo><mi>s</mi><mo>+</mo><mn>1</mn></mrow></msub></mtd></mtr><mtr><mtd><msub><mi>H</mi><mrow><mi>k</mi><mo>+</mo><mn>1</mn></mrow></msub></mtd></mtr></mtable></mfenced><msub><mi>P</mi><mi>k</mi></msub></mrow></mtd></mtr></mtable></mfenced> and but 22) Suppose the weight and bias of the second hidden layer are W ₁ , B ₁ , then the output of the second hidden layer is <mrow><msub><mi>H</mi><mn>1</mn></msub><mo>=</mo><mfenced open = "[" close = "]"><mtable><mtr><mtd><msub><mi>T</mi><mrow><mi>k</mi><mo>-</mo><mi>s</mi><mo>+</mo><mn>2</mn></mrow></msub></mtd></mtr><mtr><mtd><mo>.</mo></mtd></mtr><mtr><mtd><mo>.</mo></mtd></mtr><mtr><mtd><mo>.</mo></mtd></mtr><mtr><mtd><msub><mi>T</mi><mrow><mi>k</mi><mo>+</mo><mn>1</mn></mrow></msub></mtd></mtr></mtable></mfenced><mi>&amp;beta;</mi><msup><mrow><mo>(</mo><mi>k</mi><mo>+</mo><mn>1</mn><mo>)</mo></mrow><mo>+</mo></msup><mo>;</mo></mrow> 23) Assuming W _HE = [B ₁ W ₁ ], then the weight and bias of the second hidden layer are calculated by And H _E ＝[1H] ^T , 1 is a one-dimensional row vector whose elements are all 1; g ^-1 (x) is the inverse function of g(x) of the activation function, 24) Update the output of the second hidden layer as H ₂ =g(W _HE H _E ); update the output weight β of the final hidden layer as 25) Suppose the weight and bias of the third hidden layer are W ₂ , B ₂ , then the output of the third hidden layer is <mrow><msub><mi>H</mi><mn>3</mn></msub><mo>=</mo><mfenced open = "[" close = "]"><mtable><mtr><mtd><msub><mi>T</mi><mrow><mi>k</mi><mo>-</mo><mi>s</mi><mo>+</mo><mn>2</mn></mrow></msub></mtd></mtr><mtr><mtd><mo>.</mo></mtd></mtr><mtr><mtd><mo>.</mo></mtd></mtr><mtr><mtd><mo>.</mo></mtd></mtr><mtr><mtd><msub><mi>T</mi><mrow><mi>k</mi><mo>+</mo><mn>1</mn></mrow></msub></mtd></mtr></mtable></mfenced><msubsup><mi>&amp;beta;</mi><mrow><mi>n</mi><mi>e</mi><mi>w</mi></mrow><mo>+</mo></msubsup><mo>-</mo><mo>-</mo><mo>-</mo><mrow><mo>(</mo><mn>4</mn><mo>)</mo></mrow></mrow> 26) Assuming W _HE1 = [B ₂ W ₂ ], then the weight and bias of the third hidden layer are calculated by And H _E1 =[1 H ₂ ] ^T , 1 is a one-dimensional row vector whose elements are all 1, and g ^-1 (x) is the inverse function of g(x) of the activation function; 27) Update the output of the third hidden layer as: H ₄ =g(W _HE1 H _E1 ) (5) 28) Update the output weight β of the final hidden layer as: Then the final output is f=β _new1 H ₄ . 5. the application method of the online sequential multi-hidden layer extreme learning machine of band forgetting factor according to claim 4, it is characterized in that also comprising the following steps: For a network structure with multiple hidden layers, formulas (4), (5), and (6) are iterated cyclically, three hidden layers are iterated once, four hidden layers are iterated twice, and N hidden layers are iterated N-2 times , each iteration ends, so that β _new = β _new1 , H ₂ = H ₄ .