Decoding Mobile App Performance with

Explainable AI (XAI)
Archana D Deepthi Sri R Deva Dharani P
Assistant Professor(Senior Grade) Department of ECE Department of ECE
Department of ECE PSG Institute of Technology and Applied PSG Institute of Technology and Applied
PSG Institute of Technology and Applied Research Research
Research Tamilnadu, India Tamilnadu, India
Tamilnadu, India 22l112@psgitech.ac.in 22l113@psgitech.ac.in
archana@psgitech.ac.in

Haripriya S Vimala Varshini C P

Department of ECE Department of ECE
PSG Institute of Technology and Applied PSG Institute of Technology and Applied
Research Research
Tamilnadu, India Tamilnadu, India
22l118@psgitech.ac.in 22l154@psgitech.ac.in

Abstract—The document discusses the development of a robust results. This ensemble model enhances prediction
mobile application software for small, wireless computing accuracy by combining the strengths of individual models.
devices, emphasizing the importance of a clear strategy for app
success. It introduces Interpretable AI to elucidate prediction
To ensure the results are understandable, these predictions are
results on mobile app success factors and assist developers. An
Ensemble Machine Learning Algorithm is used, along with
further processed through an interpretable AI model. This
Explainable AI models like LIME and SHAP, achieving a approach adheres to the principles of explainable artificial
prediction accuracy of 96%. intelligence (XAI), which aims to make the mechanics behind
AI predictions transparent and comprehensible to both
Keywords: Explainable Artificial Intelligence, Ensemble specialists and non-specialists, moving away from the "black
Machine Learning, Blackbox. box" nature of traditional machine learning. The remainder of
the paper discusses our literature review, methodologies,
I. INTRODUCTION experimental outcomes, conclusions, and cited references,
This paper explores the creation and utilization of mobile offering a comprehensive insight into the predictive
applications, which are specialized software applications techniques and their effectiveness in the context of mobile
designed to operate on handheld devices such as smartphones, app development.
tablets, and laptops. Mobile applications offer a comparable
range of services and products as those available on personal
computers but are typically tailored with a more limited set of
features tailored to mobile contexts. With mobile users
surpassing desktop users, businesses increasingly focus on
mobile strategies to enhance user engagement and improve
conversion rates. The convenience of mobile apps over mobile
websites has led to their preference among users, emphasizing
the importance of mobile channels for reaching optimal
business potential.

Evaluating the success of these mobile applications post-

launch involves monitoring their performance on platforms
like the Google Play Store, which poses significant challenges
for developers. To address this, we have implemented an
interpretable model that employs various Machine Learning
algorithms including linear regression, Support Vector Fig. 1 Role of mobile applications in every sector
Machine (SVM), k-Nearest Neighbor, Random Forest, and K-
Means. Our findings indicate that using an Ensemble Machine
Learning algorithm, which integrates the predictions of
decision trees, SVMs, and linear regression, yields the most
 II. LITERATURE SURVEY Key ensemble techniques include the Bayes Optimal
The expansion of digital storage in both public and private Classifier, boosting, Bootstrap Aggregating (bagging), and
sectors encompasses various data types, including spoken, the voting ensemble method. These methods effectively
written, and visual content, notably mobile application combine the strengths of different models to mitigate their
reviews. Systems aimed at predicting outcomes from these individual weaknesses.The process begins with the
reviews transform unstructured data into structured insights, development of two or more standalone models that are
enhancing transparency for consumers and developers. trained on the same dataset. Subsequently, a Voting
However, initial studies faced challenges due to limited Classifier is employed to aggregate the predictions of these
linguistic diversity and inadequate Unicode support in text models. This classifier computes the average of the outputs
mining techniques. Research in this field has grown, from each model, and the final prediction is determined by
incorporating diverse computational methods to forecast the the majority vote of these averages.In terms of weighting the
performance of digital platforms, ERP systems, and desktop predictions from each sub-model, assigning appropriate
software. Key methodologies include genetic algorithms, weights can be complex and may require heuristic methods.
symbolic logic, and neural networks. Notable studies include: To streamline this process, this project employs a
- Chien-Ho et al. (2007), who used genetic algorithms and combination of diverse algorithms, including Random
support vector machines to predict project success based on Forest, Support Vector Classification (SVC), and Logistic
project parameters such as stakeholder values and operational Regression, to effectively handle the weight allocation. The
metrics. ensemble's performance is evaluated using the
- Min-Yuan Cheng et al. (2008), who developed predictive cross_val_score function with a k-fold cross-validation
models based on construction costs, effort metrics, and strategy, which helps in assessing the model robustness and
project obligations to estimate project success. ensuring reliable predictions.
- Xiaohong Tai et al. (2009), who analyzed project risks
related to staffing, design flaws, and project management to
forecast application success.
- Ahmet Okutan and colleagues (2014), who utilized software
metrics like RFC, LOC, and CBO to predict software defects.
- Guillamue et al. (2015), who investigated how
misjudgments in project scope, management, and stakeholder
engagement could lead to project failures.
A study presented at the 2020 Springer Conference by
researchers from the University of Technology Sydney and
Islamic Azad University employed neural networks and
algorithms such as SVMs, MLPs, and Random Forests to
predict mobile app success but noted challenges in result
interpretability. In response to these issues, there has been a
shift towards ensemble machine learning models, combining
multiple predictive models to enhance accuracy and
robustness. However, these models often lack interpretability, Fig. 2 Working of Voting ensemble method
leading to increased focus on explainable artificial
IV. EXPLAINABLE ARTIFICIAL INTELLIGENCE
intelligence (XAI). XAI aims to make AI decision-making
processes transparent and understandable, promoting The remarkable advances in machine learning have
responsible AI development. This project has embraced an catalyzed the proliferation of artificial intelligence
ensemble machine learning approach coupled with XAI applications. Despite these advancements, the operations
frameworks to tackle accuracy and explainability concerns, within many AI systems remain largely opaque, presenting
with the goal of refining mobile app success prediction and challenges in understanding how decisions are made. This
offering clearer insights into influencing factors. opacity has spurred the development of Explainable
Artificial Intelligence (XAI), which seeks to demystify the
inner workings of AI models.XAI is designed to make AI
III. ENSEMBLE MACHINE LEARNING
models more transparent, ensuring that their functions are
Ensemble methods in machine learning leverage multiple not only highly effective but also accessible and
base models to create a superior predictive model. This understandable, even to those without technical expertise.
approach enhances decision-making accuracy at each data The goals of XAI include:
split, offering a robust alternative to relying on a single 1) Enhancing the explainability and interpretability of
model. By integrating the outputs of various machine learning models while maintaining a high level of learning
models, ensemble techniques ensure more reliable and performance.
accurate predictions. For this project, we've opted for an
2) Allowing non-experts to grasp the mechanisms behind AI
ensemble learning strategy because the reliability of our
decisions.
interpretive insights hinges on the accuracy of these
3) Managing the complexities introduced by the next wave
predictions.
of AI innovations effectively.
4)To illustrate these concepts in a straightforward way,
Figure 3 delineates the principles of XAI. In practical
applications, XAI methodologies help elucidate the rationale
behind model outputs, offering clear insights into the factors
influencing results.In this project, we've integrated two
prominent XAI frameworks to tackle different aspects of
model explanation:
1. SHAP (Shapley Additive Explanations): Used for global
interpretation, SHAP values provide a comprehensive view
of how each feature in a model contributes to the prediction
outcome, based on cooperative game theory.
2.LIME(Local Interpretable Model-agnostic Explanations): Table. 1 Features present in dataset along with the description
Focused on local interpretation, LIME explains predictions
on individual instances, making it possible to see how
changes in input features affect the output within a specific
context.
These tools bridge the gap between advanced machine
learning techniques and user-friendly outputs, enabling
stakeholders to make informed decisions based on the AI’s
findings.

Fig. 3 Common Machine learning model and Blackbox nature

V. METHODOLOGY
This section outlines the methodology and procedural steps
adopted in our proposed approach for analyzing mobile app
data from the Google Play Store. The detailed structure and
composition of the dataset, including variables like Ratings,
Reviews, Size, App Type, and Genre, are comprehensively
described in section VI of this paper.
1.Data Collection: The data was sourced directly from the
Google Play Store. This dataset's diverse features facilitate an
in-depth analysis of app performance metrics.
2.Data Pre-processing: The initial dataset undergoes a series
of pre-processing steps to ensure data quality and relevance
for further analysis. These steps include cleaning,
normalization, and handling missing values, detailed in
section VII.
3.Feature Selection: After pre-processing, we employ the
XGBoost (eXtreme Gradient Boosting) algorithm for feature
selection. This method helps identify and retain the most
influential features that significantly impact model outcomes,
enhancing the efficiency and accuracy of our predictions.

Fig. 4 Methodology of proposed approach

 VI. DATASET DESCRIPTION
The dataset sourced from the Google Play Store includes
approximately 10,000 apps, each characterized by a
comprehensive set of features that provide a detailed view of
app properties and user interactions. These features are
instrumental in performing analyses and building predictive
models. The features included in the dataset are as follows:
Table. 1 Features present in dataset along with the description

VII. DATA PRE-PROCESSING

Preparing data is essential in the data analysis process to
ensure it's accurate and suitable for building machine
learning models. For the Google Play Store dataset, which
contains around 10,000 apps, several steps are necessary to
refine the raw data into a usable format for analysis and
modeling. Here are the typical pre-processing steps involved
in managing this dataset:

Fig. 5 Proposed approach

4.Dataset Split : The refined dataset is then divided into

training and testing sets, maintaining a 70/30 split. This
separation supports model training and validation processes,
ensuring that our findings are robust and generalizable.
5. Model Training: Voting Ensemble Method
The training data is subjected to a Voting Ensemble
Fig. 6 Steps followed in Pre-processing
classification approach. This method integrates multiple
algorithms—namely Logistic Regression, Decision Tree, and
Support Vector Machines (SVM)—to form a powerful
predictive model. The ensemble technique leverages the
strengths of each algorithm, potentially reducing bias and
variance.
6. Model Explanation: XAI Implementation: Upon completing
the classification, we apply Explainable Artificial Intelligence
(XAI) techniques, specifically using the LIME and SHAP
models for local and global interpretability, respectively.
These models allow us to understand and explain the decision-
making processes of our ensemble classifier, shedding light on
how specific features influence predictions.
7.Results Documentation: The outcomes of the classification
and the insights from the explainability analysis are Fig. 7 Flow chart for the steps followed in Pre-processing the google play
store dataset
thoroughly documented and discussed in section IX of this
paper. This final section provides a detailed review of the In Google Colab, a widely used Python environment for
model’s performance, the significance of the features machine learning and data analysis, we can efficiently
identified, and the implications of the model's explanatory preprocess a dataset sourced from the Google Play Store.
power on stakeholders' decision-making processes. Through This process involves leveraging Python libraries like
this structured approach, we aim to not only predict app pandas and numpy to refine the data for subsequent analysis
performance effectively but also provide transparency and or machine learning tasks. Here's a breakdown of the
insight into the predictive processes, aligning with the broader specific preprocessing steps implemented:
objectives of responsible AI development. 1.Checking for NULL Values and Data Types: Using
pandas functions, we identify any NULL values within each
column and confirm the data type of each feature.
2. Analyzing Unique Values and Handling NULLs:
We examine the unique values in each column to understand
data distribution and integrity. Rows with NULL values are
dropped to maintain data quality:
3.Converting Data Types: The Review column is converted
from an object type to int64 to enable numerical operations:
4.Cleaning the Installs Column: Special characters like ‘+’
and ‘,’ are removed from the Installs column, and these
entries are converted to integers:
5.Standardizing the Price Column: The '$' sign is eliminated
from the Price column, and the result is converted to a float
for accurate financial calculations
6.Normalizing Size Values: The Size column is standardized
by removing 'M' (megabytes) and 'k' (kilobytes), converting
all entries to a consistent unit (megabytes)
7. Log Normalizing and Encoding: Logarithmic
normalization is applied to skewed numerical data to reduce
scale disparities
8.One Hot Encoding: Remaining categorical variables are
converted into numerical format using One Hot Encoding,
creating new columns indicating the presence of each Fig. 8 Feature importance graph using XGBoost algorithm
possible value from the original data:
By following these steps in Google Colab using pandas and The XGBoost algorithm, known for its precision in feature
numpy, the dataset is refined into a clean, well-structured selection, has pinpointed a set of crucial features from the
format suitable for advanced data analysis and machine Google Play Store dataset. These features, which have a
learning applications. significant influence on model outcomes, include: Reviews,
Size, Category, Rating, Installs, Price, Type.
VIII.FEATURE SELECTION
XGBoost, short for eXtreme Gradient Boosting, has
emerged as a potent algorithm in Kaggle competitions and IX. EXPERIMENTAL RESULTS
real-world machine learning applications, particularly with Once the data has been pre-processed and divided into
structured or tabular data. It represents an advanced training and testing sets, and the most important features
implementation of gradient boosted decision trees, have been identified, we employ an Ensemble Machine
renowned for its high speed and exceptional performance. Learning technique to categorize the outcomes as either
XGBoost offers various gradient boosting techniques, success (1) or failure (0). Our approach utilizes an ensemble
including standard Gradient Boosting, Stochastic Gradient voting method that integrates several algorithms, including
Boosting—which introduces randomness by training each Support Vector Machines (SVM), Decision Trees, and
tree on a random subset of data to mitigate overfitting—and Logistic Regression, to enhance prediction accuracy and
Regularized Gradient Boosting, which incorporates robustness. This strategy leverages the strengths of different
regularization to improve model generalization. With its models to achieve a more reliable classification of app
ability to efficiently handle large-scale data and provide success.
sophisticated options for model tuning and regularization,
XGBoost is favored for feature selection and model
development in both machine learning and Explainable
Artificial Intelligence (XAI) frameworks. Its primary
strengths lie in its swift execution and the robustness of the
models it generates.

Fig. 9 Working on Ensemble method

After applying the Voting Ensemble method to predict

outcomes, we proceed to implement Explainable Artificial
Intelligence (XAI) techniques to clarify the decision-making
process of the machine learning model. We utilize two
prominent XAI tools: SHAP (Shapley Additive
Explanations) and LIME (Local Interpretable Model-
agnostic Explanations). SHAP is utilized to offer global
interpretability, providing insights into the overall behavior
of the model across all data points, while LIME focuses on the app did not succeed. Positive influencers like reviews
local interpretability, explaining predictions on an individual and installs, which generally support an app's success, were
instance level. These tools play a crucial role in enhancing outweighed by negative factors. The category, size, and
transparency and understanding of the model's decision- rating of the app negatively influenced its performance,
making process, addressing the increasing demand for driving it towards failure. This visualization helps pinpoint
accountability in AI applications. the key areas where the app struggled, providing actionable
insights for developers to focus on in future iterations or
similar applications.

Fig. 12 Paint splash app

Fig. 10 SHAP Value(impact on the model output)
Figure 12 illustrates the analysis of the "Paint Splash!" app's
Figure 10 illustrates the global interpretation of the dataset failure. In the graph, orange indicates features that positively
used for predicting mobile app success. In the graph, the influence the model's prediction of failure, while blue
most influential features are shown on the left side. The indicates features that have a negative influence. From this
SHAP plot provides a visualization of how each feature analysis, it is clear that the failure of the "Paint Splash!" app
contributes to shifting the prediction from the base value, can be attributed to the outcomes depicted in the Local
which is the average output of the model, towards the actual Interpretable graph. Key factors that typically influence an
predicted outcome. In this depiction, features that decrease app’s success or failure include its category, price, the
the prediction are colored blue, while features that increase number of reviews it receives, and its overall user rating.
it are shown in red.The results of the LIME model, used for These elements have played a significant role in the case of
local interpretation, are also presented. LIME provides a "Paint Splash!".
detailed list of justifications, explaining the impact of each
feature on individual predictions. This output is crucial for
understanding the specific contribution of each feature and
identifying which ones have the most significant influence
on the prediction outcome. This dual approach of using
SHAP and LIME enhances our understanding of both the
collective and individual impacts of features within the
dataset.

Fig. 13 Little Teddy bear coloring book game app

Figure 13 analyzes the success factors for the app "Little

Teddy Bear Coloring Book Game." In the graph, features
that positively impact the model's prediction of success are
shown in orange, while those that negatively impact the
prediction are shown in blue. The Local Interpretable
graph's output clearly indicates that reviews are a significant
positive driver toward the app's success. Conversely, factors
Fig. 11 Smoke effect photo maker app such as category, price, number of installs, and rating appear
Figure.11 showcases the local interpretive analysis of the to negatively influence the app’s success.
app "Smoke Effect Photo Maker - Smoke Editor" and its
failure in the market. In the graph, features contributing
positively towards the model's prediction are marked in
orange, whereas those pushing the prediction negatively are
highlighted in blue. From the graph's output, it's clear why
 [3].Lundberg, S. M., & Lee, S. I. (2017). A unified approach
to interpreting model predictions. In Advances in Neural
Information Processing Systems (pp. 4765-4774).
[4].Ribeiro, M. T., Singh, S., & Guestrin, C. (2016). "Why
Should I Trust You?": Explaining the Predictions of Any
Classifier. In Proceedings of the 22nd ACM SIGKDD
International Conference on Knowledge Discovery and Data
Mining (pp. 1135-1144).
[5].Guidotti, R., Monreale, A., Ruggieri, S., Turini, F.,
Giannotti, F., & Pedreschi, D. (2018). A survey of methods
for explaining black box models. ACM Computing Surveys
(CSUR), 51(5), 1-42.
Fig. 14 Viber Messenger app

Figure 14 presents the analysis of why the app "Viber

Messenger" failed. In the Local Interpretable graph used for
this analysis, orange represents features that positively
influence the model's prediction of failure, while blue
represents features that negatively influence the prediction.
The graph clearly shows that the app's price contributed
positively to its success, suggesting that pricing strategy was
not a factor in its failure. However, the app's category, size,
reviews, and rating are shown in blue, indicating these
factors collectively contributed to pushing the app towards
failure.
X. CONCLUSION AND FUTURE WORKS
In this work, we introduce a novel framework designed to
demystify the operations of black box machine learning
(ML) models for mobile app developers, incorporating
elements of explainability. This framework aids developers
in predicting the potential success of new apps using data
preprocessing followed by classification through an
ensemble method. The ensemble method utilized here is the
voting ensemble, which combines multiple machine learning
algorithms to classify the dataset into either success (1) or
failure (0) of the apps.
Our study specifically focuses on the Google Play Store, the
world's largest app marketplace, using a dataset comprising
approximately 10,000 apps. This dataset forms the basis for
our Global Interpretation model, which means the
interpretations and predictions are strictly applicable to the
apps within this dataset. Any changes in the dataset could
lead to different results in the Global Interpretation.
For detailed case studies, we have confined our analysis to a
Local Interpretation of the Models for a select sample of five
apps, as discussed in section IX of our paper. The accuracy
achieved in these interpretations was 96%. Moving forward,
the aim of our project is to enhance this accuracy and
broaden the scope of our interpretations by integrating more
Explainable Artificial Intelligence (XAI) models, thus
providing more comprehensive insights across a wider array
of apps.

REFERENCES
[1].Gunning, D., & Aha, D. (2019). DARPA's Explainable
Artificial Intelligence (XAI) Program. AI Magazine, 40(2),
44-58.
[2].Adadi, A., & Berrada, M. (2018). Peeking inside the
black-box: A survey on explainable artificial intelligence
(XAI). IEEE Access, 6, 52138-52160.