Design of a corporate financial crisis prediction model based on improved ABC-RNN+Bi-LSTM algorithm in the context of sustainable development

RNN fusion Bi-LSTM network

In RNN networks, the recurrent structure keeps the state value of the hidden neuron at the current moment and inputs it to the neuron as part of the following recurrent input. The input signal of an RNN takes a temporal input. Each layer shares network weights and biases at each input step, significantly reducing the number of parameters to be learned in the network. The structure of an RNN neural network is shown in Fig. 2. The input layer is denoted as $\left\{X_0,\dots ,X_{i-1},X_i,X_{i+1}\right\}$, the output layer is denoted as $\left\{O_0,\dots ,O_{i-1},O_i,O_{i+1}\right\}$, and the hidden layer is denoted as $\left\{S_0,\dots ,S_{i-1},S_i,S_{i+1}\right\}.U,V,W$ are the weight matrixs. At the time i, S_i−1 the current X_i are used as input, and the result O_i is calculated as output and passed to time i + 1 so that each input layer can get the output weight of the previous layer.

Bi-LSTM is an LSTM variant, which not only has the long-range sequence learning capability of the LSTM model but also further improves LSTM to learn the association relationship between sequence data forward and backwards, which makes the model more advantageous for classification problems. We use financial text data to train the forward LSTM and backward LSTM of BiLSTM, whose structure is shown in Fig. 3. The output vector calculation formula for the time step is shown in (1). (1)${\displaystyle H_t=W_{hf}^{\prime }{h}^{\prime }+W_{hb}^{″}{h}^{″}+b_h.}$

RNN can perform effective feature extraction, but it cannot solve the gradient drop problem, and the prediction accuracy worsens as the series data increases. Bi-LSTM can preserve the long-term dependency between data by merging memory units that can update the previous hidden state. Therefore, combining RNN and Bi-LSTM can solve the long-term dependence problem. Compared with using only the Bi-LSTM network, the RNN and Bi-LSTM network can receive the feature vector matrix of the RNN model to extract the time-series data for financial crisis prediction and capture deeper input data features.

Feature attention mechanism

The attention mechanism has been applied to various aspects, and its proper use can help the network make better predictions of features. The feature attention mechanism used in this article takes the RNN network output data matrix X₁, X₂, …, X_M as input at the previous moment of the feature time series. It obtains e₁, e₂, …, e_M by calculating the weights of each feature at the current moment, then performs normalization and enhances or weakens the expression of relevant input information according to the attention weights. Then it multiplies the weights obtained at the current moment with the corresponding features and outputs them as b₁X₁, b₂X₂, …, b_MX_M. The structure is shown in Fig. 4.

ABC algorithm optimization

ABC algorithm is a search algorithm that simulates group behaviour. It has the characteristics of simple operation, few control parameters, high robustness, and robust searchability for optimal global solutions (Sreenu & Naidu, 2022). It can realize the performance optimization of the network model. In this article, the ABC algorithm is introduced into the company financial crisis prediction model to search the optimal weights and biases of the RNN+Bi-LSTM network. The specific steps of the ABC algorithm include the following parts.

1. Create an RNN network. Denote the weights and biases of the RNN network by the row vector X, $X=\left[w,b\right]$, where w is the row vector representation of the network weights with dimension N × M + M × M + M × L, and b is the row vector representation of the network biases with dimension M + L.N, M, Lare the number of nerve elements, and the dimension of X is denoted as D.

2. Initialize the training parameters of the ABC algorithm: the number of food sources S_N, the bee colony size N_C, the number of employed bees N_e, the number of following bees N_o, and the maximum number of cycles MCN. Where S_N = N_e = N_o, N_c = N_e + N_o. Each food source corresponds to an initialized solution X.

3. The suitability values for each food source were calculated as follows. (2)${\displaystyle fi{t}_i=\frac{1}{1+f_i}.}$

4. Enter the loop phase until the MCN is less than the number of iterations.

5. The employed bees perform a domain search for the alternate food source Vi from the current food source Xi according to $V_i^j=X_i^j+rand\left(-1,1\right)\left(X_i^j-X_k^j\right)$, where $k\in \left\{1,2,\dots ,S_N\right\}$ and k ≠ i, $j\in \left\{1,2,\dots ,D\right\}$. Compare the fitness values of the current food source X_i with the new food source V_i andselect the food source with a higher fitness value according to the greed principle.

6. The odds ratio for food sources is calculated as: (3)${\displaystyle P_i=\frac{fi{t}_i}{\sum _{i-1}^{S_N}fi{t}_i}.}$

The following bees perform a domain search for new food sources based on the probability formula for the current food source and select the food source with higher fitness based on the greed principle.

7. If the MCN is lower than the number of iterations, perform step 9; otherwise, perform step 5.

8. Initialize the parameters of the RNN+Bi-LSTM network using the solution of the optimal food source, and then start training the RNN+Bi-LSTM network.

9. Evaluate the performance of the improved ABC-RNN+Bi-LSTM network.

Analysis of the prediction effect of the ABC-RNN+Bi-LSTM model

The model performs crisis prediction on the training set to obtain the confusion matrix, as shown in Table 3.

From the prediction results of the training samples, we can see that there are two misclassifications in 31 positive samples and three misclassifications in 32 negative samples, i.e., five misclassifications in 63 samples. It can be deduced that the model’s accuracy is 95.06% and recall is 96.55%, i.e., 29 out of 31 ST companies are detected commonly. The classification accuracy curves of the ABC-RNN+Bi-LSTM+Bi-LSTM model with feature attention mechanism at 150 iterations on the training set are shown in Fig. 5.

It can be seen that the model in this article has been stable after 18 iterations. This is mainly due to the attention mechanism and the ABC algorithm introduced in this article can help the model extract more high-quality features quickly. So the proposed ABC-RNN+Bi-LSTM company financial crisis prediction model with feature attention mechanism has good robustness and generalization ability.

The model performs crisis prediction on the test set to obtain the confusion matrix, as shown in Table 4.

As can be seen from the prediction results of the test samples, there are seven misclassified companies in 63 samples, and the model’s accuracy is 88.94%. Among them, recall is 88.23%, which means that 30 out of 34 ST companies are detected as normal. Recall measures the model’s ability to identify positive example samples. If a listed company is ST and the model’s judgment is non-ST, it cannot detect that the company is likely to be in crisis. In this sense, a model with a low recall value cannot be called a good model. The classification accuracy curve of the model in this article at 150 iterations on the test set is shown in Fig. 6.

The model in this article still performs well on the test set and compares with its performance on the training set. It can achieve fast convergence and stability overall, although it fluctuates slightly during the iterations.

Comparison of prediction errors of different models

To verify the improvement of the feature attention mechanism and Bi-LSTM on the prediction ability of RNN, CNN+RNN, GAN+RNN, RNN, and the model in this article are compared with the error experiment. In the error analysis, since the error calculation is trying to know the deviation size of the real point and the predicted point, Mean Square Error(MSE) and Root Mean Square Error (RMSE) can be an excellent measure of the error offset between the real situation and the predicted result. The smaller value indicates the better performance of the model. The error comparison of the four models is shown in Table 5.

It can be seen that the MSE and RMSE of the model in this article are the lowest. This is mainly because the model in this article can deeply capture the input financial data information. In addition, the feature attention mechanism modifies the weight of feature information and further improves the model’s prediction performance. Therefore, compared with other models, the model proposed in this article has apparent advantages in financial crisis prediction.

To verify the enhancement of the feature attention mechanism and RNN on the prediction ability of Bi-LSTM, the prediction accuracy of RNN+Bi-LSTM and Attention+ Bi-LSTM, AR-Bi-LSTM, and the model in this article were compared. The overall MSE and RMSE for the four models are shown in Table 6.

It can be seen that: the MSE and RMSE of this article’s model are below the other three models, reflecting the better prediction enhancement effect of RNN and feature attention mechanism for Bi-LSTM, which illustrates the stability and good prediction ability of this article’s model. In summary, the feature attention mechanism and RNN better the LSTM’s prediction enhancement for the financial crisis than using it alone.