CN109543406B - Android malicious software detection method based on XGboost machine learning algorithm - Google Patents

Abstract

The invention relates to an Android malware detection method based on an XGBoost machine learning algorithm. First, the Permission, Intent, Component and API call features are extracted by decompiling the apk file, and the feature matrix is quantified to form a feature matrix, and the parallelism of the ant colony algorithm is used. The robustness of the XGBoost classifier is optimized to obtain the optimal target and the optimal parameter combination of XGBoost. Compared with the traditional XGBoost algorithm, the improved XGBoost machine learning algorithm proposed by this invention has higher classification accuracy in Android malware detection, improves the correct rate of malware detection, and reduces the damage caused by detection errors to the Android system. probability of attack.

Description

An Android malware detection method based on XGBoost machine learning algorithm

Technical Field

The present invention relates to the technical field of malware detection on an Android platform, and in particular to an Android malware detection method based on an XGBoost machine learning algorithm.

Background Art

The Android system was officially released by Google on November 5, 2007. As an operating system based on the Linux kernel, its open source and free features have made the Android system quickly become the largest smart mobile device operating system in the market. However, while it is popular among App developers and users, it has also become the preferred target of malicious attackers. The rapid growth of Android malware has posed a serious threat to the security and privacy of users. Malware steals users' private data, causing property losses, and exploits system vulnerabilities to obtain higher permissions and achieve greater harm. With the continuous advancement of the mobile payment industry, the Internet+ concept is popular, mobile payment is developing rapidly, and mobile payment viruses are emerging in an endless stream, seriously endangering the property safety of users. Therefore, a method that can quickly and effectively detect malware is needed.

Currently, there are three main detection methods for Android malware: static detection method, dynamic detection method, and a combination of static detection and dynamic detection.

Among them, the static detection method is to decompile the installation package of the application through reverse engineering without running the Android application, and extract relevant features, such as permission information, API calls, instruction features, etc., to characterize the operations that the program may perform when running, so as to identify whether the application is malware. Static detection mostly uses machine learning algorithms to classify and detect the extracted feature information. However, the classification accuracy of this static detection method is not high, and the accuracy of malware detection is low, which increases the probability of the Android system being attacked due to detection errors.

Summary of the invention

The purpose of the present invention is to overcome the shortcomings of the prior art and provide an Android malware detection method based on the XGBoost machine learning algorithm, which has high classification accuracy, high accuracy of malware detection, and greatly reduces the probability of the Android system being attacked due to detection errors.

To achieve the above purpose, the technical solution provided by the present invention is:

An Android malware detection method based on XGBoost machine learning algorithm extracts Permission, Intent, Component and API call features by decompiling apk files and quantizing the feature matrix. The ant colony optimization algorithm is used to optimize the parameters of the XGBoost ensemble learning framework to quickly find the global optimal solution. After multiple iterations, the optimal target value is obtained and the optimal parameter combination of XGBoost, namely, shrinkage step size and minimum sample weight threshold min_child_weight in child nodes, is obtained. Finally, the optimized XGBoost algorithm is applied to the Android malware detection model.

Furthermore, the specific steps of the Android malware detection method based on the XGBoost machine learning algorithm are as follows:

S1: Use apktool to decompile the apk file to get AndroidManifest.xml and classes.dex;

S2: Extract Permission, Intent, Component and API call features;

S3: Feature quantization, the output value is a one-hot vector, if the feature exists, it is marked as 1, otherwise it is marked as 0;

S4: All feature vectors are formed into a feature vector set, and the feature selection algorithm is used to reduce the dimension of the feature vector set to select the optimal feature subset;

S5: Use the ant colony optimization algorithm to optimize the parameters of the XGBoost integrated learning framework, quickly find the global optimal solution, obtain the optimal target value after multiple iterations, and obtain the optimal parameter combination of XGBoost, the shrinkage step size, and the minimum sample weight threshold min_child_weight in the child node;

S6: Randomly extract 10% of the optimized feature vectors as the test set, and the remaining 90% as the training set and input them into the optimized XGBoost ensemble learning framework for optimization learning;

S7: Evaluate the classification results from the perspective of true positive rate, false positive rate, and classification accuracy to determine whether the XGBoost algorithm optimized by the ant colony algorithm is used to generate the Android malware detection model that meets the detection requirements.

Furthermore, the specific steps of using the ant colony optimization algorithm to optimize the parameters of the XGBoost integrated learning framework are as follows:

A. Set the shrinkage step size of the XGBoost classifier parameters, the upper and lower limits of the minimum sample weight threshold min_child_weight in the child node, the maximum number of iterations MaxIter, the ant colony size M, and the information evaporation coefficient Rho;

B. Initialize the population, that is, initialize shrinkage and min_child_weight as the position vector of each ant;

C. Perform ant colony search;

D. Perform XGBoost training;

E. Use XGBoost classifier to calculate the objective function value and pheromone value of each ant and find the current optimal ant;

F. Determine whether the termination condition is met: If the number of iterations is greater than MaxIter, output the optimal value of the ant colony and the corresponding shrinkage and min_child_weight values, and execute step G; otherwise, increase the number of iterations by 1 and execute step C;

G. Update pheromones;

H. Use the output shrinkage and min_child_weight in the Android malware detection model.

Furthermore, the ant colony optimization algorithm is specifically as follows:

Ant colony position initialization:

Assume that XGBoost's classification accuracy is used as the objective function value

max{F(s ₁ ,w ₁ ),F(s ₂ ,w ₂ ),...,F(s _m ,w _m )}, denoted as max fitness＝max{F(X)},X＝{x ₁ ,x ₂ ,...,x _m }, where _xi represents ants. The steps to generate the initialized population using chaotic sequence are as follows:

1) Generate a D-dimensional random vector:

2) Logistics mapping, using the above formula as the initial iteration, the Logistics mapping equation is as follows:

In the formula, μ=1,i=1,2,...,N,d=1,2,...,D;

3) Map the chaotic space to the search space of optimization variables:

In the formula, max ^d is the upper limit value, and min ^d is the lower limit value;

Ant movement rules:

为第k迭代第j个蚂蚁的位置向量，定义，目标函数越大，其位置信息素浓度越大，则保存当前目标值最大的蚂蚁为

以及其信息素最大值

After the ant colony is initialized, its objective function is calculated.

is the position vector of the jth ant at the kth iteration. It is defined that the larger the objective function is, the greater the pheromone concentration is at its position. The ant with the largest current objective value is

and its pheromone maximum value

Select Local Search or Global Search:

The probability of ant migration is defined as follows:

In the formula, S is the standard deviation of the fitness function, and the calculation formula is as follows:

In the formula, m is the number of ants, and _Fave is the average fitness value;

越近，蚂蚁的转移概率就越大，其搜索的方法如下：From the above formula, we can see that

The closer the ant is, the greater the probability of its transfer. The search method is as follows:

If P( _xi )≤P0, where P0 is a constant and 0<P0<1, the ant searches in the nearby local position and the movement formula is as follows:

为移动后的位置，

为移动前的位置，a为移动步长，定义如下：In the formula

is the position after moving,

is the position before moving, a is the moving step length, which is defined as follows:

If P( _xi )>P0, the ant searches in the solution space;

Pheromone Update:

According to the value of the individual position function, the pheromone is updated as follows:

Where ρ is the information evaporation coefficient.

Compared with the existing technology, the principles and advantages of this solution are as follows:

Compared with the traditional XGBoost machine learning algorithm, which affects the performance of XGBoost algorithm classification due to parameter selection in Android malware detection, this scheme uses the ant colony algorithm to optimize the parameters of XGBoost and quickly find the optimal parameters, so that the XGBoost algorithm has good classification performance and is applied to the Android malware detection model, so that it has higher classification accuracy in Android malware detection, greatly improving the accuracy of malware detection, thereby reducing the probability of Android system being attacked due to detection errors.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a detection flow chart of an Android malware detection method based on an XGBoost machine learning algorithm of the present invention;

FIG2 is a flow chart of feature extraction in an Android malware detection method based on an XGBoost machine learning algorithm according to the present invention;

FIG3 is a flow chart of applying the ant colony algorithm to optimize XGBoost parameters in an Android malware detection method based on the XGBoost machine learning algorithm of the present invention.

DETAILED DESCRIPTION

The present invention will be further described below in conjunction with specific embodiments:

The Android malware detection method based on the XGBoost machine learning algorithm described in this embodiment is specifically as follows:

XGBoost (eXtreme Gradient Boosting) is an ensemble learning algorithm proposed by Tian Chen in 2015. In the XGBoost ensemble learning framework, the main parameters that directly affect its classification performance are the shrinkage step size (shrinkage) and the minimum sample weight threshold (min_child_weight) in the child node. Too small shrinkage will cause the algorithm to overfit, and too large shrinkage will cause the algorithm to fail to converge. For min_child_weight, too small will cause the algorithm to overfit, and too large mini_child_weight will cause the algorithm to have poor classification performance for linearly inseparable data.

Therefore, this embodiment extracts Permission, Intent, Component and APIcall features by decompiling the apk file to quantify the feature matrix, and then uses the ant colony optimization algorithm to optimize the parameters of the XGBoost integrated learning framework, quickly find the global optimal solution, obtain the optimal target value after multiple iterations, and obtain the optimal parameter combination of XGBoost, the shrinkage step size, and the minimum sample weight threshold min_child_weight in the child node. Finally, the optimized XGBoost algorithm is applied to the Android malware detection model. As shown in Figure 1, the specific steps are as follows:

S1: Use apktool to decompile the apk file to get AndroidManifest.xml and classes.dex;

S2: Extract Permission, Intent, Component and API call features. The specific process is shown in Figure 2.

S3: Feature quantization, the output value is a one-hot vector, if the feature exists, it is marked as 1, otherwise it is marked as 0;

S4: All feature vectors are formed into a feature vector set, and the feature selection algorithm is used to reduce the dimension of the feature vector set to select the optimal feature subset;

S5: Use the ant colony optimization algorithm to optimize the parameters of the XGBoost integrated learning framework, quickly find the global optimal solution, obtain the optimal target value after multiple iterations, and obtain the optimal parameter combination of XGBoost, the shrinkage step size, and the minimum sample weight threshold min_child_weight in the child node;

S6: Randomly extract 10% of the optimized feature vectors as the test set, and the remaining 90% as the training set and input them into the optimized XGBoost ensemble learning framework for optimization learning;

S7: Evaluate the classification results from the perspective of true positive rate, false positive rate, and classification accuracy to determine whether the XGBoost algorithm optimized by the ant colony algorithm is used to generate the Android malware detection model that meets the detection requirements.

In the above, as shown in Figure 3, the specific steps of using the ant colony optimization algorithm to optimize the parameters of the XGBoost integrated learning framework are as follows:

A. Set the shrinkage step size of the XGBoost classifier parameters, the upper and lower limits of the minimum sample weight threshold min_child_weight in the child node, the maximum number of iterations MaxIter, the ant colony size M, and the information evaporation coefficient Rho;

B. Initialize the population, that is, initialize shrinkage and min_child_weight as the position vector of each ant;

C. Perform ant colony search;

D. Perform XGBoost training;

E. Use XGBoost classifier to calculate the objective function value and pheromone value of each ant and find the current optimal ant;

F. Determine whether the termination condition is met: If the number of iterations is greater than MaxIter, output the optimal value of the ant colony and the corresponding shrinkage and min_child_weight values, and execute step G; otherwise, increase the number of iterations by 1 and execute step C;

G. Update pheromones;

H. Use the output shrinkage and min_child_weight in the Android malware detection model.

The specific ant colony optimization algorithm is as follows:

Ant colony position initialization:

Assume that XGBoost's classification accuracy is used as the objective function value

max{F(s ₁ ,w ₁ ),F(s ₂ ,w ₂ ),...,F(s _m ,w _m )}, denoted as max fitness＝max{F(X)},X＝{x ₁ ,x ₂ ,...,x _m }, where _xi represents ants. The steps to generate the initialized population using chaotic sequence are as follows:

1) Generate a D-dimensional random vector:

2) Logistics mapping, using the above formula as the initial iteration, the Logistics mapping equation is as follows:

In the formula, μ=1,i=1,2,...,N,d=1,2,...,D;

3) Map the chaotic space to the search space of optimization variables:

In the formula, max ^d is the upper limit value, and min ^d is the lower limit value;

Ant movement rules:

为第k迭代第j个蚂蚁的位置向量，定义，目标函数越大，其位置信息素浓度越大，则保存当前目标值最大的蚂蚁为

以及其信息素最大值

After the ant colony is initialized, its objective function is calculated.

is the position vector of the jth ant at the kth iteration. It is defined that the larger the objective function is, the greater the pheromone concentration is at its position. The ant with the largest current objective value is

and its pheromone maximum value

Select Local Search or Global Search:

The probability of ant migration is defined as follows:

In the formula, S is the standard deviation of the fitness function, and the calculation formula is as follows:

In the formula, m is the number of ants, and _Fave is the average fitness value;

越近，蚂蚁的转移概率就越大，其搜索的方法如下：From the above formula, we can see that

The closer the ant is, the greater the probability of its transfer. The search method is as follows:

If P( _xi )≤P0, where P0 is a constant and 0<P0<1, the ant searches in the nearby local position and the movement formula is as follows:

为移动后的位置，

为移动前的位置，a为移动步长，定义如下：In the formula

is the position after moving,

is the position before moving, a is the moving step length, which is defined as follows:

If P( _xi )>P0, the ant searches in the solution space;

Pheromone Update:

According to the value of the individual position function, the pheromone is updated as follows:

Where ρ is the information evaporation coefficient.

This embodiment first extracts Permission, Intent, Component and APIcall features by decompiling the apk file, and quantizes the feature matrix, and optimizes the XGBoost classifier parameters by using the parallelism and strong robustness of the ant colony algorithm to obtain the optimal target and the optimal parameter combination of XGBoost. Compared with the traditional XGBoost algorithm, the improved XGBoost machine learning algorithm proposed in this embodiment has higher classification accuracy in Android malware detection, improves the accuracy of malware detection, and reduces the probability of Android system being attacked due to detection errors.

The embodiments described above are only preferred embodiments of the present invention and are not intended to limit the scope of implementation of the present invention. Therefore, all changes made according to the shape and principle of the present invention should be included in the protection scope of the present invention.

Claims (2)

1. An Android malicious software detection method based on an XGboost machine learning algorithm is characterized in that Permission, intent, component and APIcall characteristics are extracted by decompiling an apk file, a characteristic matrix is formed in a quantized mode, an ant colony optimization algorithm is used for carrying out parameter optimization on an XGboost integrated learning framework, a global optimal solution is quickly found, an optimal target value is obtained after multiple iterations, an optimal parameter combination contraction step length shrinkage of the XGboost and a minimum sample weight threshold value min _ child _ weight in a child node are obtained, and finally the optimized XGboost algorithm is applied to an Android malicious software detection model;

the method specifically comprises the following steps:

s1: decompiling the apk file by using the apktool to obtain android manifest.xml and classes.dex;

s2: extracting the Permission, intent, component and API call characteristics;

s3: quantizing the features, wherein the output value is a one-hot vector, if the features exist, the vector is marked as 1, otherwise, the vector is marked as 0;

s4: forming a feature vector set by all feature vectors, reducing the dimension of the feature vector set by adopting a feature selection algorithm, and selecting an optimal feature subset;

s5: performing parameter optimization on the XGboost integrated learning framework by using an ant colony optimization algorithm, quickly finding out a global optimal solution, obtaining an optimal target value after multiple iterations, and obtaining an optimal parameter combination shrinkage step length shrinkage of the XGboost and a minimum sample weight threshold value min _ child _ weight in a child node;

s6: randomly extracting 10% of the optimized feature vectors as a test set, and inputting the rest 90% of the optimized feature vectors as a training set into an optimized XGboost integrated learning frame for optimized learning;

s7: and evaluating the classification result from the true rate, the false positive rate and the classification precision, and judging whether the Android malicious software detection model generated by the XGboost algorithm optimized based on the ant colony algorithm meets the detection requirement.

2. The Android malicious software detection method based on the XGboost machine learning algorithm as claimed in claim 1, wherein the specific steps of using the ant colony optimization algorithm to perform parameter optimization on the XGboost ensemble learning frame are as follows:

A. setting the contraction step length shrinkage of the XGboost classifier parameter and the upper and lower limits of the minimum sample weight threshold min _ child _ weight in the child node, the maximum iteration times MaxIter, the ant colony scale M and the information evaporation coefficient Rho;

B. initializing populations, namely initializing shrinkage and min _ child _ weight as a position vector of each ant;

C. executing ant colony search;

D. XGboost training is carried out;

E. calculating the objective function value and the pheromone value of each ant by using an XGboost classifier, and searching the current optimal ant;

F. judging whether a termination condition is met: if the iteration times are larger than the MaxIter, outputting an ant colony optimal value and corresponding shrinkage and min _ child _ weight values, executing the step G, and if not, adding 1 to the iteration times, and executing the step C;

G. updating the pheromone;

H. and using the output shrinkage and min _ child _ weight in a detection model of the Android malicious software.

Images