CN108369665A - （It is mobile）Application program uses the prediction being lost in - Google Patents

Abstract

The present invention provides a kind of attrition prediction model, usage behavior data and user characteristics to predict whether given user will be lost in (that is, stopping using) from an application program.Initially, the training set of user's interaction can be related to the loss probability value of various sequences of User Activity.Then, for active user, action of the user in browsing app can be recorded, other than user characteristics, it can also use the information to predict the probability that the user will be lost in, to implement the probability of (or on the contrary, remain loyal and be continuing with the application program) in a manner of " loss nips in the blossom ".In some embodiments, user's action of a grouping can be identified as the subsequence of the out-of-sequence row of known flow.For executing the movable user of these subsequences, can send real-time messages, preferential or promotion with influence they without being lost in.

Description

Prediction of (Mobile) App Usage Churn

Cross References to Related Applications

This application claims the benefit of US Provisional Patent Application No. 62/265,552, filed December 10, 2015, the disclosure of which is incorporated by reference into this application as if fully set forth herein.

technical field

The present invention relates generally to personal electronic devices such as smartphones, tablets, and computers, and more particularly to methods for predicting what percentage of the population that attempts a given application on a user device will continue to use it over various subsequent time periods and with what regularity Metrics and Models.

Background technique

This application generally relates to application software commonly referred to as an "app". Apps are computer software designed to help users perform specific tasks. Apps can execute on various computing devices, such as on mobile devices including smartphones. For example, mobile apps are software applications designed to run on smartphones, tablets, and other mobile devices. These apps are available through application distribution platforms, often run by owners of mobile operating systems, such as the Apple App Store, Google Phone, Windows Phone Store, and BlackBerry App World. Mobile devices such as smartphones and tablets are designed to facilitate the installation and operation of apps.

Churn prediction is the method of determining the percentage of the population (i.e. users) who will stop using a given service or product after initially trying it. All mobile app publishing companies are interested in the percentage of users who continue to use an app a week after they install it.

Churn predictions have traditionally been based on consumer characteristics such as age, gender, and zip code. However, these categories are too broad to roughly predict how a user of an app will enjoy using it and whether he or she will continue to use it, continue experimenting with it, or decide that it is of no use to him or her.

So often how users interact with a new application determines both whether they will continue and whether they have a sufficient level of interest to actually learn how to use it in a meaningful way. It's not just a question of user age, gender, and residential area. It's more about their personality, their needs, and how they relate to a particular software application during the window of time that their "first impression" of the application makes on them.

What is needed in the art is a method to more accurately predict churn based on actual user activity involving an application.

Contents of the invention

The present invention proposes a churn prediction model that uses behavioral data as well as user characteristics to predict whether a given user will churn from an application (ie, stop using it). Initially, a training set of user interactions can be associated with churn probability values for various sequences of user activities. Then, for real-time users, it is possible to record the user's actions in browsing the app. In addition to user characteristics, this information can also be used to predict the probability that the user will churn, so as to "eliminate churn in the bud" to achieve (Or the other way around, stay loyal and keep using the app). In some embodiments, a grouping of user actions can be identified as a subsequence of known churn sequences. For users performing these sub-sequences of activity, real-time messages, offers or promotions can be sent to reach them without churn. In an exemplary embodiment of the present invention, user data may be uploaded from the user's device to a dedicated or cloud server. For example, the latest aggregated data for a given app can be used on these servers to perform churn analysis or more detailed churn analysis.

Description of drawings

FIG. 1 is a graph of exemplary distinguishing values and associated distinguishing patterns for churn levels, according to an exemplary embodiment of the present invention;

Figure 2 is a diagram of an exemplary differentiation scheme for loyalty levels according to an exemplary embodiment of the present invention;

3-14 are exemplary screenshots (or portions thereof) in an exemplary beta application called "AVG WiFi Assistant (Assistant)" for testing churn and loyalty predictions according to exemplary embodiments of the present invention. );

Figure 3 depicts an exemplary launch screen;

Figure 4 depicts an exemplary OnboardWiFi (airborne wireless network) screen;

Figure 5 depicts an exemplary OnboardVPN (airborne VPN) screen;

Figure 6 depicts an exemplary guidance bubble displayed in relation to the Onboarding (airborne) screen of Figure 4 to help the user understand the functionality of the Wifi Assistant;

Figure 7 depicts an exemplary home screen;

FIG. 8 depicts an exemplary screen displayed to a user when the user selects a WiFi hotspot;

FIG. 9 is an exemplary secure hotspot screen displayed to a user upon selecting a WiFi hotspot as in FIG. 8;

Figure 10 depicts an exemplary WiFi settings screen for setting parameters of a selected WiFi hotspot;

Figure 11 depicts an exemplary WiFi Settings Advanced Screen;

Figure 12 depicts an exemplary upgrade screenshot pair;

FIG. 13 depicts an exemplary sidebar menu screen, such as accessed by a user from the screen shown in FIG. 7;

Figure 14 depicts an exemplary About screen;

Figure 15 is an exemplary table containing various feature vectors according to an exemplary embodiment of the present invention; and

Figure 16 depicts an exemplary mobile device on which a churn prediction module may be deployed according to an exemplary embodiment of the present invention.

Detailed ways

In an exemplary embodiment of the present invention, a churn prediction model that uses both behavioral data and user characteristics may be provided. Specifically, how a user navigates within an app can be recorded, and this data, in addition to a set of user characteristics, can be used to predict the probability that a user will churn (i.e. stop using the app). Estimating this value can be valuable in a number of ways, which might include, for example:

1. Allows the app developer/designer to reshape the flow of the app. That is, using an exemplary embodiment of the present invention, over a period of time it can become clear at which point or points in the app a user is about to churn or will likely churn. This can show that the app's user interface can be improved and even inform the developer/designer how to improve it;

2. Provide insight into when certain actions (e.g. coupons, promotions, etc.) should be initiated to keep customers using or engaging with the app.

I. Concept definition

In order to better understand certain concepts that underlie the various exemplary embodiments of this invention, the following definitions of key terms are provided:

Behavioral data refers to data about how a given app is used, i.e. how customers interact with the app. In particular, an ordered series of events (ie, a sequence) representing various actions that can be taken by a user of the app may be considered. For example, a chat app could consist of the following screens: Welcome (w), Sign Up (s), Tutorial (t), Address Book (a), Help (h), and Chat (c). For a given user i, an ordered series of events might be: (w, s, c, c, c, h, a), while for another user j it might be (w, s, t, a, c , c). Typically, this data is collected by enabling analytics on each event so that when an event occurs it is logged and sent to the database.

More formally, we let E = {e ₁ ,...,e _n } be the set of n different events that a user can perform, comprising an alphabet of different screen identifiers - e.g. {w, s, t, a, h, c}. Thus, the set E is the totality of all possible events (ie screens that can be accessed and interactions that can be performed on each screen). Then we let the sequence Y=<y ₁ ,...,y _m > be an ordered list of events actually performed by the user, ie, y _i ∈ E for 1≤i≤m. For behavioral data, a transaction can be represented by a tuple (an ordered set of values) (u _i , Y ), where u _i is a user identifier and Y is a sequence of events. Thus, the tuple associates a given user u _i with a given sequence of interactive events.

Customer Profile Data refers to all personal and demographic data available about a User. For example, age, gender, country of residence, income, phone make/model, operating system version, whether the user is on the PRO version (or other version type or designator), etc. Available customer profile data varies from app to app, depending primarily on which data the app collects. For example, for a subscription-based app, there will be payment information and an email address; for a dating app, on the other hand, gender, age, and city or town of residence are usually all known.

More formally, for each customer, there is a binary vector of length p: p:X={x ₁ ,...,x _p }. where for 1≤i≤p, each _xi represents the presence/absence of a binarized feature i. Like behavioral data, a transaction is a tuple (u _i ,X), where u _i is a user identifier and X is a feature vector. Here, a tuple associates a given user with a given set of customer attributes. Figure 15 is a table containing various exemplary feature vectors. As can be seen with reference thereto, each row in the table contains data from a different client or user of the exemplary WiFi Assistant application described below in connection with FIGS. 3-14. The data includes the user's country, city and country, the language they speak, and information about the user's device, such as the make, model, type, and operating system used.

II. Churn Prediction: A Two-Stage Process

In an exemplary embodiment of the invention, the churn prediction application may consist of two phases:

1. Learning phase, where a churn model is learned, derived or generated from data - based on historical data, a statistical model to predict churn is learned or generated.

2. During the deployment phase, in a live setting, a churn model is used to predict whether a user is likely to churn. After a period of time, the model can then be retrained with updated data.

In an exemplary embodiment of the invention, the entire analysis can be run with the now larger data set including updated data to find a new list of distinguishing patterns. However, in some embodiments it may also be beneficial to use only newer data after a period of time.

These stages are described below.

A. Learning Churn Model

1. Introduction and Definitions

In order to derive or learn a statistical model for churn prediction from interaction data, multiple historical data transactions are required. That is, behavioral and customer profile data from multiple customers is required. Also, for this "learning set", it is known whether a customer churns or not, so this data can be correlated with known outcomes.

Let D={(u ₁ ,Y ₁ ),…,(u _n ,Y _n )} be the transaction database of behavior characteristic data, and let A={(u ₁ ,X ₁ ),…,(u _n ,X _n )} as a transaction database for user characteristic data. As mentioned above, each such transaction database element is a set of tuples. Similarly, U={u ₁ , . . . , u _n is the database of users. Furthermore, in the training data, because it is historical, a binary class label (such as "churn" or "loyalty") is also known for each user. That is, with known label features C:U→Churned/loyal for all users in the historical dataset. Use D ^churn , D ^loyal to denote the subsets of users in D who have lost/loyal respectively, that is, D ^churn ={(u _i ,Y _i )|(u _i ,Y _i )∈D and C(u _i )= churn}. Furthermore, we need the notion of corresponding transactions between the behavioral transaction database and the feature transaction database for and in }. Intuitively, A[D'] contains all transactions of A where the user identifier is in both A and D'.

As a result, the triplet (Y _i ,X _i ,C(u _i )) provides behavioral and characteristic data of user u _i and also indicates whether the user has churned.

The overall goal of churn prediction is to learn (Y _i ,X _i ,C(u _i )) = probability of churn, i.e. learn the probability that user u _j with behavioral and feature data X _j and Y _j will churn.

2. Common methods of churn prediction

A common approach to learning statistical models for outcome prediction is to use logistic regression, Bayesian or other predictive model on an input set A and a map C of class labels. More precisely, in an exemplary embodiment of the present invention, the churn prediction model M can be derived by using an off-the-shelf prediction model Φ applied on A and its corresponding classification label, namely M=Φ(A,C), For example, with Bayesian, the probability of churn for user j - i.e.,

where P(churn) is the prior probability of churn on the training dataset, and P(X _j |churn) can be estimated using maximum likelihood estimation on A.

3. Consider behavioral data

Intuitively, groups of users can first be divided into multiple (possibly partially overlapping) groups. For example, segmentation can be done based on how customers use the app, i.e. users with similar behavior are grouped together. Therefore, a statistical model for churn prediction can be learned for each group separately.

In an exemplary test run by the inventors, grouping of users was done based on behavioral characteristics of the users. In particular, users with similar behaviors are grouped together.

Note that, as used above, the term or concept "similar behavior" is used in the sense of similar browsing behavior, ie, how a user navigates through an application. Frequent sequential patterns are one way of capturing similarities, for example; when enough people visit a screen in the sequence A-B-C, this can be recognized as a frequent pattern. Only the frequent patterns that are actually discriminative can then be further grouped, as non-discriminative patterns are not informative for prediction purposes.

Furthermore, the grouping is optimized to select those groups that deviate from the overall churn probability, that is, those whose overall churn probability is much higher/lower than the overall churn probability. To compute such groupings, well-known techniques such as frequent sequence mining, hidden Markov models or variable-length Markov chains can be used.

What follows is a detailed description of how frequent sequence mining techniques are used to obtain such off-ensemble churn probabilities.

4. Discover frequent sequence recognition patterns

For two sequences Y={y ₁ ,...,y _m }, Z={z ₁ ,...,z _k }, Z is said to be a subsequence of Y, expressed as Z≤Y, if and only if: in Y There exists a sequence of length k such that:

y _i =z ₁ ,...,y _i+k =z _k , where 1≤i≤mk.

The cover of a sequence Z in a transaction database D consists of the bins or clusters of transactions in D that support Z:

cover(Z,D)={(u _i ,Y)|(u _i ,Y)∈D,Z<D}.

The support for sequence Z in transaction database D is the number of transactions masked by Z in D, supp(Z,D)=|cover(Z,D)|. Frequent sequence mining is about deriving all sequences with support greater than a user-provided minimum support threshold. Distinguishing frequent sequence patterns are patterns that distinguish between classes, i.e. patterns that are more common in the churn group than in the loyal customer group. Formally, the discriminant value of the frequent sequence pattern P is equal to:

max(supp((P,D ^churned )/supp(P,D ^loyal ),supp((P,D ^loyal )/supp(P,D ^churned )).

Typically, discriminant values are user-defined parameters that can be used to fine-tune an algorithm. An exemplary discriminant value may be 0.6.

Given the previous definitions and discussions, the pseudocode for learning the churn model is presented next. The output of the algorithm is all frequently recognized patterns and a set of models collection.

1. Learning loss model (A, D, C)

1.1. ＝{}//Initialize the set of loss models to empty

1.2. = Discovery of discriminative frequent patterns with respect to (D)

1.3. For For each frequent sequence pattern P in:

1.3.1.M _P = learning loss model on A [cover(P,D)]}

1.3.2.

1.4. Back

In an exemplary embodiment of the invention, a set of models emerges because for each pattern a classification model is built. A schema is backed by some part of the data, and on that part, an associated model can be built.

B. Deploy Churn Model

To predict whether user u _j will churn, it is assumed that part of u _j 's customer characteristics, i.e., X _j and behavioral characteristics, i.e. Y _j , are available. The behavioral features of _uj are then used to identify groups with similar behavioral distributions; this group has some pattern P as a common descriptor.

For example, according to the behavioral feature Y _j , for example, we can first find the best match from the set of patterns P, where the longest subsequence means the best match. Therefore, for all p in P, we test whether p is a subsequence of Y _j . For example, from all matches, we choose the longest one. If there are multiple matching largest subsequences, we choose the most discriminative one. If again the unlikely case of multiple maximal subsequences occurs, all equivalence distinguishes, we can use both (and average the results).

Then, the model trained for pattern P, ie _MP , can be used to determine the probability of churn _{for Xj} . It is worth noting that Y _j varies with time, and the selected forecasting model can also vary with time.

In fact, for all patterns discovered during the learning phase, it is desired to find the best match to the current behavioral feature sequence _Yj . For "best", a simple heuristic can be used: the longest sequence that is a subsequence of _Yj is the most suitable. When multiple equally long sequences fit, we can compute a prediction score for each of them. Then select the maximum score for the churn category and the maximum score for the loyalty category and return. In terms of pseudocode:

1. Predict churn (X _j , Y _j , )

1.1.maxmatch=0

1.2.Out={}

1.3. For All P in (# start with longest pattern first)

1.3.1. If ((P≤X _j ) and (|P|≥maxmatch))

1.3.1.1.Out＝Out∪{calculate the probability of churn and loyalty (M _P ,Y _j )}

1.3.1.2.maxmatch=|P|

1.4. (Churn, Loyalty) = Max from Out

1.5. Return (Churn, Loyalty)

In addition to predicting whether a user will churn, another aspect of exemplary embodiments of the present invention is to influence user access to certain screens, ie, change user behavior regarding app usage. Therefore, we want to change the way a user uses the app based on the insights we gain from the model and apply it to his or her behavior. For example, suppose we have detected two groups of users of the above chat app, and the first group has a behavior pattern (w(welcome), s(register), c(chat)), the overall churn probability of this group is equal to 90%. On the other hand, the second group has behavioral patterns (W(Welcome), S(Signup), T(Tutorial)) with an overall churn probability of 10%. So, if the user visited stage w, s, we want to influence the user to now visit tutorial screen t. This influence can be done both online and offline. Offline (ie, pre-) effects can be made, for example, by changing the user interface to make the tutorial screens more visible. Online influence (meaning real-time) can be achieved by automatically sending out promotions.

Therefore, in addition to customer characteristics, churn prediction methods are also provided based on app usage data.

III. Example

A. Examples of Recognition Patterns

churn pattern

Figure 1 depicts examples of differentiating patterns for churn categories. The discriminant value is equal to the probability that the sequence is from the "churn" category.

loyalty model

Figure 2 depicts an example of a discrimination pattern for the loyalty category, with a discrimination value equal to the probability that the sequence is from the 'loyalty' category. It can be easily noticed that Discriminant, _Loyalty = [1 – Discriminant, _Churn ].

B. Detailed Churn Prediction Example

Two examples of churn prediction according to exemplary embodiments of the present invention are described next.

Example 1:

Select the following behavioral data from the churned user group:

Networkinfo|GetStarted Pressed 0|OnboardWifi Start yes|OnboardVPN yes0|Bubble

HotspotAutomation Display|Bubble VPN Display|Bubble HotspotAutomation

Dismissed|Bubble VPN Dismissed|Home Button AddWifiNetwork|SecureHotspot yes

0|SecureHotspot ContinueNoVPN 0|Home Button AddWifiNetwork|HomeButton

AddWifiNetwork|Home Button SideMenu|Home Button SideMenu|SideWifiAssistant Off|Side

WifiAssistant On|Home Button AddWifiNetwork|Home ButtonAddWifiNetwork|Home Button

Upgrade|Upgrade Secured Hotspot WifiSettings

Initially, when only the user's first action is known, i.e. in this case, only the Networkinfo screen is known to have been visited, but there is no pattern matching this sequence. As a result, a churn prediction model based on the entire dataset (i.e., a global model) is used. The result is that this user will churn with a probability of 18.93%, so the loyalty is 81.07%.

Note that the global churn model is a churn model derived using all available data. As mentioned above, this model can be used as a fallback when it does not match any of the derived discriminative sequence patterns.

As the user accesses more screens, the results do not change until 14 screens are recorded. Here are short descriptions of some of the intermediate stages:

Networkinfo|GetStarted Pressed 0＝>no match, global model P(churn)＝18.93％

Networkinfo|GetStarted Pressed 0|OnboardWifi Start yes|OnboardVPN yes0＝>no match,

Global model P(churn) = 18.93%

This is because none of the diff patterns we exported matched (partially) the first 13 screens. However, after the 13th screen, we have the following input:

Networkinfo|GetStarted Pressed 0|OnboardWifi Start yes|OnboardVPN yes0|Bubble

HotspotAutomation Display|Bubble VPN Display|Bubble HotspotAutomation

Dismissed|Bubble VPN Dismissed|Home Button AddWifiNetwork|SecureHotspot yes

0|SecureHotspot ContinueNoVPN 0|Home Button AddWifiNetwork|HomeButton

AddWifiNetwork|Home Button SideMenu|

There is now one that matches one of the distinguishing patterns from the churn category:

Home Button AddWifiNetwork|Home Button AddWifiNetwork|Home ButtonSideMenu

By choosing a churn prediction model trained only on data instances that support this rule, we get: P(churn) = 100%.

The remaining screens added by the user do not result in any different patterns matching the input sequence, so the best matching rule for this user remains:

Home Button AddWifiNetwork|Home Button AddWifiNetwork|Home ButtonSideMenu

Example 2:

The second example describes the various screens visited by loyal users:

networkinfo|GetStarted Pressed 0|OnboardWifi Start yes|BubbleHotspotAutomation

Display|Bubble VPN Display|Bubble HotspotAutomation Dismissed|BubbleVPN

Dismissed|SecureHotspot AlwaysOn Checked|SecureHotspot yes 0|SecureHotspot

ContinueNoVPN 0|Home Button SideMenu|Home VPN On

As in Example 1, after the first three screens have been accessed, there is no matching discernment pattern here either. Therefore, the following initial churn predictions were made:

networkinfo|GetStarted Pressed 0|OnboardWifi Start yes＝>

no match, global model P(churn)=28.36%.

With the addition of the next screen, a match is found in the pattern:

OnboardWifi Start yes|OnboardVPN yes 0|Bubble HotspotAutomationDisplay

And the corresponding model estimates P(churn) = 0.04%. Adding another screen produces two matching distinguish modes: the former "OnboardWifi Start yes|OnboardVPN yes 0|Bubble HotspotAutomationDisplay" and the latter:

OnboardWifi Start yes|Bubble HotspotAutomation Display|Bubble VPNDisplay.

Since the latter is the longer pattern (three events vs. two events), the latter pattern is selected and used for prediction following the above rules, so now P(churn) = 0.06%.

IV. Exemplary Application Flow: Screens and Their Corresponding Names

Exemplary application flows are described next with reference to FIGS. 3-14. In the following instructions, events are sent to Google Analytics ("GA"). Every event sent to Google Analytics is an event in our behavioral database. In an exemplary embodiment, other mobile tracking solutions may be used such as, for example, Adobe Omniture, Flurry Analytics, or any in-house developed software that tracks and sends app user interactions. In an exemplary embodiment of the present invention, user data may be uploaded from the user's device to a dedicated or cloud server. For example, a churn analysis using the latest aggregated data for a given app or a more detailed churn analysis can be performed on these servers.

Exemplary application flow is described with reference to the screenshots of FIGS. 3-14. For each screen, a list of user interactions is provided, and data based on such interactions sent to the analysis environment (eg, GA) is provided. Data capturing user actions can be sent to GA every time a user visits or interacts with a screen, as described below.

start up

When the user opens the screen shown in Figure 3, we send GA the name of the page he is viewing, in this case:

activation

When the user presses the "GetStarted" button, the information sent to GA is:

ο "GetStarted", "Pressed_0", "EventCounter", null

ο"NetworkInfo", "mobile operator type_mobile operator name", "EventCounter", null

For example, Mobile Carrier Type can include the following information:

·GSM or CDMA or None or SIP

OnboardWiFi

Figure 4 depicts an exemplary Onboarding WiFi screen. When the user sees this screen, the name of the page he is viewing is sent to GA, in this case:

Onboard Wifi

When the user taps "Start WiFi Automation" in Figure 4, the information sent is:

ο "OnboardWifi", "Start_Yes", "EventCounter", null

When the user clicks "Maybe later (maybe later)" in Figure 4, the information sent is:

ο "OnboardWifi", "Start_Later", "EventCounter", null

OnboardVPN

Figure 5 depicts an example Onboarding VPN screen. When the user sees this screen, we send the name of the page he is viewing to GA, in this case:

OnboardVPN

When the user clicks "Got it", the information sent to GA is:

ο "OnboardVPN", "Yes_0", "EventCounter", null

tutoring bubbles

As a part of the Onboarding process shown in FIG. 4 and FIG. 5 , for example, a tutorial bubble may be displayed to help the user understand the functions of the WiFi assistant. An exemplary tutoring bubble is shown in FIG. 6 .

When the user sees these bubbles, the message sent is:

ο "Bubble", "HotspotAutomation_Display", "EventCounter", null

ο "Bubble", "VPN_Displayed", "EventCounter", null

ο "Bubble", "AssistantOff_Display", "EventCounter", null

When the user abandons these bubbles, the message sent is:

ο "Bubble", "HotspotAutomation_Dismissed", "EventCounter", null

ο "Bubble", "VPN_Dismissed", "EventCounter", null

ο "Bubble", "AssistantOff_Dismissed", "EventCounter", null

the main screen

Figure 7 depicts an exemplary home screen. When the user opens this screen, the name of the page he is viewing will be sent to GA, in this case:

Home

When the user clicks "Wifi off", the information sent is:

ο "Home", "Wifi_On", "EventCounter", null

When the user clicks "Wifi is on", the information sent is:

ο "Home", "Wifi_Off", "EventCounter", null

When the user turns on the VPN, the information sent is:

ο "Home", "VPN_On", "EventCounter", null

(without first sending the user to connect to the hotspot and display the SecureHotspot screen)

When the user closes the VPN, the information sent is:

ο "Home", "VPN_Off", "EventCounter", null

The buttons "WiFi Off" and "WiFi On" are clickable, allowing interaction data to be created.

On the home screen as shown in FIG. 7, the user may click on the icon shown in the upper left of FIG. 7 as three horizontal bars.

When the user clicks on the icon in the upper left corner to access the sidebar menu, the message sent is:

ο "Home", "Button_SideMenu", "EventCounter", null

When the user clicks "Go Pro", the information sent is:

ο "Home", "Button_Upgrade", "EventCounter", null

When the user clicks "+", the information sent is:

ο "Home", "Button_AddWifiNetwork", "EventCounter", null

hotspot interaction

Figure 8 depicts an exemplary hotspot connection screen. For example, if a user enters a Starbucks in Amsterdam, Netherlands, the user sees this screen.

When the user clicks on the hotspot and selects "connect to network", the information sent is:

ο "Home", "Hotspot_Connect", "EventCounter", null

When the user clicks on the hotspot and selects "forget network", the information sent is:

ο "Home", "Hotspot_Forget", "EventCounter", null

When the user clicks on the hotspot and selects the "Modify Network" network, the information sent is:

ο "Home", "Hotspot_WifiSettings", "EventCounter", null

When the user clicks on the hotspot and selects "disconnect from the network", the information sent is:

ο "Home", "Hotspot_Disconnect", "EventCounter", null

security hotspot

Figure 9 depicts an exemplary secure hotspot screen. For example, if this is the first time a user chooses to connect to a hotspot (eg, Starbucks Amsterdam), the user will see this screen. When this screen is shown, the name of the page the user is viewing is sent to GA, in this case:

SecureHotspot

When the user clicks the "Secure Hotspot" button, the information sent is:

ο"SecureHotspot", "Yes", "EventCounter", null

When the user does not check the "Always use VPN for this hotspot" box, the message sent is:

ο "SecureHotspot", "AlwaysOn_Unchecked", "EventCounter", null

When the user checks the "Always use VPN for this hotspot" box, the message sent is:

ο "SecureHotspot", "AlwaysOn_Checked", "EventCounter", null

When the user clicks the "No VPN Connection" button, the information sent is:

ο "SecureHotspot", "ContinueNoVPN", "EventCounter", null

WiFi settings

Figure 10 depicts an exemplary WiFi setup screenshot. When the user opens this screen, the name of the page he is viewing is sent to GA, in this case:

Wifi Settings

User interaction with WiFi settings:

When the user changes WiFi automation to OFF, the message sent is:

ο "WifiSettings", "WiFi Auto_Off", "EventCounter", null

When the user changes WiFi automation to ON, the message sent is:

ο "WifiSettings", "WiFi Auto_On", "EventCounter", null

· When the user changes the VPN automation to OFF, the information sent is:

ο "WifiSettings", "VPN Auto_Off", "EventCounter", null

· When the user changes the VPN automation to ON, the information sent is:

ο "WifiSettings", "VPN Auto_On", "EventCounter", null

When the user clicks the upgrade (Upgrade), the information sent is:

ο "WifiSettings", "Upgrade", "EventCounter", null

When the user clicks "...", the information sent is:

ο "WifiSettings", "More", "EventCounter", null

When the user clicks "Forget Network (Forget Network)", the information sent is:

ο "WifiSettings", "More_Forget", "EventCounter", null

When the user clicks Advanced, the information sent is:

ο "WifiSettings", "More_Advanced", "EventCounter", null

When the user clicks the log (Log), the information sent is:

ο "WifiSettings", "More_Log", "EventCounter", null

WiFi Settings Advanced

Figure 11 depicts an exemplary WiFi settings high-level screenshot. When the user opens this screen, the name of the page he is viewing is sent to GA, in this case:

WifiSettingsAdvanced

Advanced user interaction with WiFi settings:

When the user unchecks the option to turn on Wifi automatically, the message sent is:

ο "WifiSettings", "Advanced_AutoOnDisabled", "EventCounter", null

When the user checks to automatically turn on Wifi, the information sent is:

ο "WifiSettings", "Advanced_AutoOnEnabled", "EventCounter", null

When the user unchecks automatically turn off Wifi, the message sent is:

ο "WifiSettings", "Advanced_AutoOffDisabled", "EventCounter", null

When the user checks to automatically turn off Wifi, the information sent is:

ο "WifiSettings", "Advanced_AutoOffEnabled", "EventCounter", null

When the user clicks "Clear ID", the information sent is:

ο "WifiSettings", "Advanced_ClearID", "EventCounter", null

upgrade

Figure 12 depicts a pair of exemplary WiFi setup high-level screenshots. When the user opens this screen, the name of the page he is viewing is sent to GA, in this case:

upgrade

User interaction with upgrades:

When the user clicks "Secure Me", the information sent is:

ο "Upgrade", "Payment_Display",,, EventCounter", null

When the user completes the payment process, the information sent is:

ο "Upgrade", "Payment_Ok",,, EventCounter", null

When the user clicks "Cancel Subscription (cancel subscription)", the information sent is:

ο "Upgrade", "Cancel",,, EventCounter", null

When a user clicks on a secure hotspot, the information sent is:

ο "Upgrade", "SecuredHotspot_WifiSettings",,, EventCounter", null

side menu

Figure 13 depicts an exemplary sidebar menu screenshot. For example, as mentioned above, the sidebar menu can be accessed by the user tapping the icon in the upper left of the home screen. When the user opens this screen, the name of the page he is viewing is sent to GA, in this case:

side

User interaction with the sidebar menu:

When the user clicks "Go Pro" (see the upper right of Figure 13), the information sent is:

ο "Side", "Button_Upgrade", "EventCounter", null

· When the user changes the Wifi Assistant automation to OFF, the information sent is:

ο "Side", "WifiAssistant_Off", "EventCounter", null

· When the user changes the WiFi assistant automation to ON, the information sent is:

ο "Side", "WifiAssistant_On", "EventCounter", null

When the user clicks "Share", the information sent is:

ο "Side", "Button_Share", "EventCounter", null

(also a screen event is sent as,, Share "screen)

When the user clicks "More Info (more information)", the information sent is:

ο "Side", "Button_About", "EventCounter", null

When the user clicks "News (news)", the information sent is:

ο "Side", "Button_News", "EventCounter", null

When the user clicks "Rate us (evaluate us)", the information sent is:

ο "Side", "Button_RateUs", "EventCounter", null

about

Figure 14 depicts an exemplary About screen shot. For example, a user may navigate to this screen from the sidebar menu screen shown in FIG. 13 . When a user opens the about screen, the name of the page he is viewing is sent to GA, in this case:

About

ο When the user clicks on these options, send a screen event

Wifi Interaction from Android System Settings

Wifi interaction outside of Wifi Assistant

· When the user turns off WiFi from the Android settings, the WiFi assistant pauses, sending the following message:

ο "System", "Assistant_Pause", "EventCounter", null

When the user turns on WiFi from the Android settings, we also turn on WiFi in the WiFi assistant and send:

ο "System", "Assistant_Resume", "EventCounter", null

This event is also sent every time WiFi Assistant is resumed from a suspended state. In an exemplary embodiment of the present invention, this requires filtering in the analytics backend whenever recovering from a button press in the UI.

As can be seen from the detailed application flow given above, for an exemplary application such as "AVG WiFi Assistant", an exemplary implementation of the present invention is achieved by capturing the screens accessed by the user and the user's interactions on each screen accessed The proposed system and method can use this behavior to operate on a database of many users and predict churn or loyalty to the application. If the method is applied to many apps of a single type, such as gaming apps or social media apps, where a correlation can be established between the types of screens accessed (e.g. all smartphone apps have an opening screen, a home screen and user-preferred screen collections), or applied to a newer version of an existing program with some changes, can use relatively small or even no training set and data and prediction methods from all similar applications to predict the same as Churn or loyalty of users interacting with new applications. For example, this approach could be an improvement over simply using a small training set of the total percentage for the new app as described above.

The combination of user characteristics and behavior data in the model

In the exemplary model generation process described above, the distinguishing schema was behavioral data, ie the sequences of screens visited and events attended on such screens. Most commonly, there is a difference in behavior about an app versus a propensity to churn from that app. However, this is not classification. For propensity to churn, sometimes user characteristics are more predictive, or combinations of user characteristics and various behavioral interaction sequences are more predictive. It is thus noticed that two customers with the same behavior sequence as for an app may have completely different churn probabilities. Some applications are more friendly or suitable for one customer group than another. For example, women in their 30s prefer dating apps to fantasy football gambling apps. Likewise, bodybuilding apps appeal more to younger men. Thus, in such demographically specific apps, it is often a combination of user characteristics and transient behavioral sequences that can best differentiate churn propensity. Accordingly, any optimal clustering of customer characteristics and behavioral data may be useful in various exemplary embodiments of the present invention, and all such clustering and resulting differentiating patterns for churn or loyalty are useful. is expected and is within the scope of the present invention.

V. Non-limiting software and hardware embodiments

Exemplary Mobile Devices and Systems

FIG. 16 shows a high-level block diagram of a mobile device 1601 . It will further be appreciated that the arrangement shown in Figure 16 is illustrative and that variations and modifications are possible. The mobile device 1601 can include a controller 1602 , a wireless module 1604 , a location module 1606 , a churn prediction module 108 , a computer readable medium (CRM) 1610 , a display module 1612 and an input module 1614 . Mobile device 1601 may include additional modules. In some embodiments, mobile device 1601 may be of sufficient size, size, and weight to enable the device to be easily moved by a user. For example, mobile device 1601 may be pocket sized.

The controller 1602 , which may be implemented as one or more integrated circuits, is capable of controlling and managing the overall operation of the mobile device 1601 . For example, controller 1602 may perform various tasks such as retrieving various assets that may be stored in CRM 1610, accessing functions of various modules (e.g., interacting with other Bluetooth-enabled devices through the Various software programs on 1610 (eg, operating system and application programs) and the like. In some embodiments, the controller 1602 may include one or more processors (eg, microprocessors or microcontrollers) configured to execute machine-readable instructions. For example, controller 1602 may include a single-chip application processor. Controller 1602 may be further connected to CRM 1610 in any suitable manner.

Wireless module 1604 may include any suitable wireless communication technology. For example, wireless module 1604 may include Bluetooth modules, radio frequency (RF) modules, WiFi modules, and/or the like. The Bluetooth module may include any suitable combination of hardware for wirelessly communicating with other Bluetooth-enabled devices and allowing for the exchange of RF signals between the controller 1602 and the other Bluetooth-enabled devices. In some embodiments, the Bluetooth module may perform such wireless communication in accordance with the Bluetooth Basic Rate/Enhanced Data Rate (BR/EDR) and/or Bluetooth Low Energy (LE) standards. Generally, the Bluetooth protocol allows point-to-point wireless communication between multiple devices over a short distance (eg, 30 meters). Bluetooth has grown in popularity since its introduction and is now used in a variety of different devices. To allow Bluetooth to be used in a wider variety of applications, a low-power version of the technology has been introduced Specification Version 4.0. In general, Bluetooth Low Energy (LE) enables devices to communicate wirelessly with low power consumption. use bluetooth devices can typically run for more than a year without needing to recharge their batteries.

For example, a Bluetooth module may include a Bluetooth-only Suitable hardware to perform device discovery, connection establishment, and communication (eg, single-mode operation). As another example, a Bluetooth module may include a Bluetooth-based /EDR and Bluetooth Appropriate hardware for device discovery, connection establishment, and communication for both (eg, dual mode operation). As yet another example, a Bluetooth module may include only Bluetooth-based Suitable hardware for device discovery, connection establishment and communication for /EDR.

The RF module may include any suitable combination of hardware for performing wireless communications with wireless voice and/or data networks. For example, the RF module may include an RF transceiver that enables a user of the mobile device 1601 to place telephone calls over a wireless voice network.

The WiFi module may include any suitable combination of hardware for performing WiFi-based communications with other WiFi-enabled devices. For example, the WiFi module may be compatible with IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, and/or IEEE802.11n.

The location module 1606 may include any suitable location technology that uses one or more wireless signals to determine a current location. In some embodiments, the positioning module 1606 includes a global positioning system (GPS) module. In some embodiments, the positioning module 1606 includes one or more of: a WiFi location module, a cellular location module, a crowdsourced WiFi location module, a time-of-flight (ToF) location module, and the like.

Churn prediction module 1608 may include code that, when executed, predicts the probability that a user will churn from or remain loyal to an app based on the user's interaction with a given app that is also stored and operational on the mobile device. For example, using the methods described above, the churn prediction module 1608 can send the predictions to a backend server operated by the app's publisher. The app publisher can then send out various attempts to the user as described above to persuade him or her to take action in order to reduce the likelihood that he or she will churn. In addition, the churn prediction module 1608 can continuously download updated aggregate user data and algorithm updates to fine-tune its predictive model, and similarly, can also perform device-side collection and aggregation of app usage data and send it to the backend server.

CRM 1610 may be implemented, for example, using magnetic disks, flash memory, random access memory (RAM), hybrid types of memory, optical disk drives, or any other storage medium that can store program code and/or data. CRM 1610 may store software programs executable by controller 102, including operating systems, application programs, and associated program code (eg, code for churn prediction module 1608).

A software program (also referred to herein as software or an app) may include any program executable by the controller 1602 . In some embodiments, certain software programs may be installed on the mobile device 1601 by their manufacturers, while other software programs may be installed by the user. Examples of software programs may include operating systems, navigation or other mapping applications, locator applications, productivity applications, video game applications, personal information management applications, applications for playing media assets and/or navigating databases of media assets programs, applications for controlling the phone interface to make and/or receive calls, and so on. Although not specifically shown, one or more application modules (or instruction sets) may be provided for launching and executing one or more application programs, such as various software components stored in medium 1610, to execute mobile device 1601 various functions.

Display module 1612 may be implemented using any suitable display technology, including CRT displays, LCD displays (eg, touch screen), plasma displays, direct or rear projection DLP, microdisplays, and the like. In various implementations, the display module 1612 can be used to visually display user interfaces, images, and/or the like.

The input module 1614 can function as a touch screen (eg, an LCD-based touch screen), a voice command system, a keyboard, a computer mouse, a trackball, a wireless remote control, buttons, and the like. An input module 1614 may allow a user to provide input to invoke functions of the controller 1602 . In some embodiments, the input module 1614 and the display module 1612 may be combined or integrated. For example, mobile device 1601 may include an LCD-based touch screen that displays images and also captures user input. Illustratively, a user may tap his or her finger on the area of the touch screen surface where icons are displayed. The touch screen can capture the tap and, in response, launch a software program associated with the icon. After launching the software program, the graphical user interface of the application program can be displayed on the touch screen for presentation to the user.

Various exemplary embodiments of the present invention as described above may be implemented as one or more program products, software applications, etc., for use with a computer system, such as a smartphone or other mobile user device. The terms program, software application, etc. as used herein are defined as a sequence of instructions designed for execution on a computer system or data processor. A program, computer program, or software application may include a subroutine, function, procedure, object method, object implementation, executable application, applet, servlet, source code, object code, shared Libraries/Dynamic Load Libraries and/or other sequences of instructions designed to be executed on a computer system.

The program of the program product or software may define functions of the embodiments (including the methods described herein) and may be contained on various computer-readable media. Exemplary computer-readable media include, but are not limited to: (i) information permanently stored on non-writable storage media (e.g., a read-only memory device within a computer such as a CD-ROM disk readable by a CD-ROM drive); (ii) ) variable information stored on a writable storage medium, such as a floppy disk drive or a floppy disk within a hard drive; or (iii) information transmitted to a computer by a communication medium, such as through a computer or telephone network, including wireless communications. The latter embodiment specifically includes information downloaded from the Internet and other networks. Such computer-readable media, when carrying computer-readable instructions that direct the functions of the present invention, represent embodiments of the present invention.

In general, routines that execute embodiments of the present invention, whether implemented as part of an operating system or as a particular application, component, program, module, object, or sequence of instructions, may be referred to herein as "programs." A computer program generally comprises a plurality of instructions which are to be translated by a local computer into a machine readable format and thus executable instructions. In addition, programs include variables and data structures that either reside locally to the program or reside in memory or storage devices. In addition, the various procedures described herein can be identified based on the application in which they are implemented in a particular embodiment of the invention. It should be appreciated, however, that any specific procedural terminology below is used for convenience only, and thus the invention should not be limited to use only in any specific application identified and/or implied by such terminology.

It is also clear that a computer program can be organized into an often infinite number of ways in which routines, procedures, methods, modules, objects, etc., and in which program functionality can be distributed among various software layers (e.g., operating system, libraries, APIs, applications, applets, etc.). It should be understood that the invention is not limited to the specific organization and assignment or program functionality described herein.

The present invention can be realized by hardware, software, or a combination of hardware and software. A system according to a preferred embodiment of the present invention can be implemented in a centralized fashion in one computer system, or in a distributed fashion where different elements are distributed over several interconnected computer systems including cloud-connected computing systems and devices. Any kind of computer system or other apparatus adapted for carrying out the methods described herein is suitable, and preferably the invention is implemented in a smartphone, tablet or other personal electronic device. A typical combination of hardware and software could be a general purpose computer system with a computer program that, when being loaded and executed, controls the computer system such that it carries out the methods described herein. On the user equipment side, for example, a typical combination of hardware and software may be a receiver equipped with one or more data processors with a computer program that, when loaded and executed, controls the data processors so that they perform the the method described.

Each computer system may include, inter alia, one or more computers and at least one signal-bearing medium that allows the computers to read data, instructions, messages or message packets, and other signal-bearing information from the signal-bearing medium. Signal bearing media may include nonvolatile memory such as ROM, flash memory, disk drive memory, CD-ROM, and other permanent storage. In addition, computer media may include, for example, volatile memory such as RAM, buffers, cache memory, and network circuits. Additionally, signal-bearing media may include signal-bearing information in transitory state media (eg, network links and/or network interfaces, including wired or wireless networks), which allow a computer to read such signal-bearing information.

Although specific embodiments of the invention have been disclosed, those of ordinary skill in the art will appreciate that changes may be made in the specific embodiments without departing from the spirit and scope of the invention. Therefore, the scope of the present invention is not limited to the specific embodiments. The foregoing description and drawings are by way of example only and are not intended to limit the invention in any way, except as set forth in the following claims. For example, while the present disclosure is described in terms of predicting churn or loyalty probabilities for applications on mobile phones, as noted above, its techniques and systems are applicable to any type of application on any type of user device. It is particularly noted that those skilled in the art could readily combine the various technical aspects of the various elements of the various exemplary embodiments that have been described above in various other ways, all of which are considered to be in within the scope of the present invention.

Claims (21)

1. a kind of method that processor for predicting customer loss is realized, the method includes：

Following data is collected using one or more data processors：

User data corresponding with the user of the application program run on a user device, the user data include that user is basic Attribute information, and

User interactive data associated with application program, including at least one of the following：(i) which user interface had accessed Screen, (ii) is with which kind of sequence, and which event (iii) participates on every screen；And

Similar users group is determined for the user based on the user data using the data processor；

Using data processor one or more differentiation patterns are determined from user interactive data；

At least one difference pattern is selected according to the one of definition group of rule,

It calculates customer loss or is loyal to the probability of application program；With

At least one：The probability is stored on the user equipment, and the probability of the user is sent to service Device.

2. according to the method described in claim 1, wherein based on any one distribution similar users group in following：(i) it is instructing Practice the upper cluster completed of one concentrated or (ii) is clustered by the update that back-end server executes.

3. according to the method described in claim 1, wherein, similar users are distributed based on the initial clustering completed on training set Group is periodically updated using all available user data.

4. according to the method described in claim 1, wherein, the differentiation pattern includes following one：User circle that user accesses The sequence of user interface screen and the action taken in each user interface screen that the sequence of face curtain or user access Row.

5. according to the method described in claim 1, wherein, selected differentiation pattern is selected based on length, is longest Pattern.

6. according to the method described in claim 1, wherein, in multiple differentiation patterns, selecting longest recognition mode, and wherein If there is the differentiation pattern of multiple equal lengths, then selection has the differentiation pattern that highest is lost in probability.

7. will according to the method described in claim 1, calculating user after wherein being interacted with user interface screen in each user The probability to be lost in.

8. according to the method described in claim 7, probability more than the definition level being wherein lost in response to instruction, sends out to user Message is sent to instruct user to access one or more particular user interface screens to reduce the probability of loss.

9. a kind of includes the non-transitory computer-readable medium of one or more instruction sequences, one or more of sequence of instructions Row cause when executed by one or more processors：

User data corresponding with the user of the application program run on a user device is collected, the user data includes using Family essential attribute information, and

User interactive data associated with the application program, including at least one of the following：

(i) which user interface screen is had accessed,

(ii) with which kind of sequence, and

(iii) which event is carried out on every screen；

The similar users group of user is determined based on user data；

One or more differentiation patterns are determined according to user interactive data；

At least one differentiation pattern is selected according to the one of definition group of rule；With

It calculates customer loss or is loyal to the probability of application program.

10. a kind of computer system, including：

One or more processors；With

One or more of addressable memories of processor, the memory storage can be by one or more of processors The instruction of execution, to collect：

(i) user data corresponding with the user of the application program run on a user device, the user data include using Family essential attribute information, and

(ii) user interactive data associated with application program,

It is that user determines similar users group based on user data；

One or more differentiation patterns are determined from user interactive data；

At least one differentiation pattern is selected according to the one of definition group of rule,

It calculates customer loss or is loyal to the probability of application program；With

At least one：

Probability storage on a user device, and by the probability of the user is sent to server.

11. computer system according to claim 10, wherein a upper cluster completed based on (i) in training set Or (ii) newer cluster for being executed by back-end server distributes similar users group.

12. computer system according to claim 10, wherein being distributed based on the initial clustering completed on training set Similar users group is updated periodically using all available subscribers data.

13. computer system according to claim 10, which is characterized in that the differentiation pattern includes following one：With The user interface screen and taken in each user interface screen that the sequence for the user interface screen that family accesses or user access Action sequence.

14. computer system according to claim 10, which is characterized in that selected differentiation pattern be based on length come Selection.

15. computer system according to claim 10, wherein in multiple differentiation patterns, longest is selected to distinguish pattern, And wherein if there is the differentiation pattern of multiple equal lengths, then selection has the differentiation pattern that highest is lost in probability.

16. computer system according to claim 10, wherein being counted after each user interacts with user interface screen Calculate the probability that user will be lost in.

17. computer system according to claim 16 will disappear wherein in response to the loss probability more than definition level Breath is sent to user to instruct user to access one or more particular user interface screens to reduce the probability of loss.

18. computer system according to claim 10, further comprises at least one of the following：The probability is deposited Storage is sent to server on the user equipment, and by the probability of user.

19. computer system according to claim 10, wherein the calculating probability executes on the server, and In response to the loss probability on definition level, it is one or more special to instruct user to access to send a message to user equipment User interface screen is determined to reduce the probability of loss.

20. according to the method described in claim 1, the wherein described calculating probability executes on a user device, and user Data are to upload to proprietary or cloud server from the user equipment.

21. according to the method for claim 20, wherein being come using the newest collective data of set app on the server Execute more detailed churn analysis.