CN110808872A - Method and device for realizing flow experiment and electronic equipment - Google Patents

Abstract

The embodiment of the present invention discloses a method, device and electronic device for realizing a traffic experiment. The method includes: determining a target experimental layer where the traffic experiment is located, and the target experimental layer includes multiple experimental domains; attribute information, respectively determine the experimental domain corresponding to each user traffic; determine the experimental parameter thresholds of the traffic experiment according to each user traffic and the policy set of the corresponding experimental domain, and implement the corresponding traffic experiment in the experimental domain. The present invention divides the target experimental layer into multiple experimental domains, so that the same experimental layer can simultaneously exist multiple flow tests with mutually exclusive preset conditions, thereby avoiding the inability of multiple preset conditions and mutual exclusion in the same experimental layer in the traditional technology. The problem of the flow experiment, thereby improving the experimental efficiency of the experimental layer.

Description

A method, device and electronic device for realizing flow experiment

technical field

The invention relates to the field of computer technology, and in particular, to a method, device and electronic device for realizing a flow experiment.

Background technique

Traffic experiment, usually divides part of user traffic, and then modifies one or more experimental parameter thresholds during program execution to test the experience data of this part of user traffic under different versions, functions or strategies, and finally according to user traffic feedback Experience data and compare the effects/performance of different versions, functions or strategies to obtain the optimal version, function or strategy. For example, before a new website function or a new strategy on the site is launched with full traffic, it is usually necessary to conduct traffic experiments such as algorithm tuning, user experience comparison, and page version effect comparison, so as to continuously optimize the design scheme.

In traditional traffic experiments, conditions for delineating user traffic can usually be set at the experimental layer, and then user traffic that matches the conditions can be selected to complete the traffic experiment. However, since the conditions for delineating user traffic in the same experiment layer are unique, the same experiment layer can only conduct traffic experiments with the same delineation user traffic conditions in the same time period, and multiple delineation user traffic conditions cannot be mutually exclusive at the same time. flow experiment, resulting in lower experiment efficiency in the experimental layer.

SUMMARY OF THE INVENTION

The purpose of the embodiments of the present invention is to provide a method, a device and an electronic device for realizing a traffic experiment, so as to solve the problem that in the prior art, the same experimental layer cannot simultaneously exist multiple traffic experiments with mutually exclusive traffic conditions delineating users in the same time period. This leads to the problem of low experimental efficiency in the experimental layer.

In order to solve the above-mentioned technical problems, the embodiments of the present invention are implemented as follows:

In a first aspect, a method for a traffic experiment provided by an embodiment of the present invention includes:

determining the target experiment layer where the traffic experiment is located, where the target experiment layer includes multiple experiment domains;

According to the acquired attribute information of each user traffic, the experimental domain corresponding to each user traffic is determined respectively;

According to each user traffic and the policy set of the corresponding experimental domain, the experimental parameter thresholds of the traffic experiment are determined, and the corresponding traffic experiment is implemented in the experimental domain.

In an implementation manner, the experiment domain is used to implement a benchmark flow experiment and at least one subflow experiment, wherein the benchmark flow experiment is a control experiment, and is used for effect comparison with the subflow experiment. The traffic experiment is a sub-experiment under the strategy set of the experiment domain.

In an implementation manner, the determining the experimental domain corresponding to the user traffic according to the attribute information of each user traffic includes: for each user traffic, the attribute information of the user traffic is determined. Perform a hash operation to determine the corresponding hash operation result;

A modulo operation is performed on the hash operation result to determine the experimental domain corresponding to each user traffic. In an implementation manner, the experimental parameter threshold of the traffic experiment is determined according to each user traffic and the policy set of the corresponding experimental domain, and the corresponding traffic experiment is implemented in the experimental domain, including:

Determine whether the attribute information of each user traffic matches the preset conditions of the corresponding experimental domain;

When the attribute information of the user traffic matches the preset condition of the corresponding experimental domain, determining a sub-traffic experimental area in the experimental domain corresponding to the user traffic;

Judging whether there is a sub-flow experiment in the sub-flow experiment area;

If the sub-flow experiment exists in the sub-flow experiment area, obtain a first policy set corresponding to the sub-flow experiment, and determine the experimental parameter thresholds of the flow experiment according to the first policy set, so as to realize the The traffic experiment corresponding to the experimental domain;

If the sub-flow experiment does not exist in the sub-flow experiment area, obtain a preset second policy set of the experiment domain, and determine the experimental parameter threshold of the flow experiment according to the preset second policy set , in order to realize the flow experiment corresponding to the experiment domain.

In an implementation manner, determining the experimental parameter thresholds of the traffic experiment according to each user traffic and the policy set corresponding to the experimental domain, and implementing the corresponding traffic experiment in the experimental domain, further comprising:

When the attribute information of the user traffic does not match the preset conditions of the corresponding experimental domain, obtain a preset second strategy set of the experimental domain; determine the experiment of the traffic experiment according to the preset second strategy set Parameter threshold to realize the flow experiment corresponding to the experiment domain.

In an implementation manner, the method further includes: saving the identifier of the experiment domain corresponding to each user flow and the policy set of the experiment domain into the session storage corresponding to the user flow, and displaying the page.

In a second aspect, an embodiment of the present invention provides a device for a flow experiment, the device comprising:

The first determination module is used to determine the target experiment layer where the flow experiment is located, and the target experiment layer includes a plurality of experiment domains;

The second determining module is configured to respectively determine the experimental domain corresponding to each user traffic according to the acquired attribute information of each user traffic;

The processing module is used to determine the experimental parameter thresholds of the traffic test according to each user traffic and the policy set corresponding to the experimental domain, and implement the corresponding traffic experiment in the experimental domain.

In an implementation manner, the experiment domain is used to implement a benchmark flow experiment and at least one subflow experiment, wherein the benchmark flow experiment is a control experiment, and is used for effect comparison with the subflow experiment. The traffic experiment is a sub-experiment under the strategy set of the experiment domain.

In an implementation manner, the second determining module is used for:

For each user traffic, hash operation is performed on the attribute information of the user traffic, and a corresponding hash operation result is determined;

A modulo operation is performed on the hash operation result to determine the experimental domain corresponding to each user traffic.

In an implementation manner, the processing module is used for:

Determine whether the attribute information of each user traffic matches the preset conditions of the corresponding experimental domain;

When the attribute information of the user traffic matches the preset condition of the experimental domain, determining a sub-traffic experimental area in the experimental domain corresponding to the user traffic;

Judging whether there is a sub-flow experiment in the sub-flow experiment area;

If the sub-flow experiment exists in the sub-flow experiment area, obtain a first policy set corresponding to the sub-flow experiment, and determine the experimental parameter thresholds of the flow experiment according to the first policy set, so as to realize the The traffic experiment corresponding to the experimental domain;

If the sub-flow experiment does not exist in the sub-flow experiment area, a preset second policy set of the experiment domain is obtained, and the experimental parameter thresholds of the flow experiment are determined according to the preset second policy set, so as to The traffic experiment corresponding to the experiment domain is implemented.

In an implementation manner, the processing module further includes: when the attribute information of the user traffic does not match the preset conditions of the corresponding experimental domain, acquiring a preset second strategy set of the experimental domain; The preset second strategy set determines the experimental parameter thresholds of the traffic experiment, so as to realize the traffic experiment corresponding to the experimental domain.

In an implementation manner, the apparatus further includes a display module, configured to save the identifier of the experimental domain corresponding to each user traffic and the policy set of the experimental domain in the session storage corresponding to the user traffic, and perform display of the page.

In a third aspect, an embodiment of the present invention provides a flow experiment device, including a processor, a memory, and a computer program stored on the memory and executable on the processor, the computer program being executed by the processor The steps of the method for implementing the traffic experiment provided by the foregoing embodiment are implemented when executed.

In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, characterized in that, a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the traffic experiment provided by the foregoing embodiment is implemented steps of the method.

It can be seen from the technical solutions provided by the above embodiments of the present invention that the embodiment of the present invention divides the target experimental layer into at least two experimental domains, and allocates each user traffic to the corresponding experimental domain according to the attribute information of each user traffic. The strategy set of each experimental domain determines the experimental parameter thresholds of the traffic experiment, so as to realize the corresponding traffic experiment of the experimental domain. In this way, since the preset conditions of each experimental domain are independent of each other and do not affect each other, at least two experiments can exist in the same experimental layer at the same time. A traffic experiment with mutually exclusive preset conditions avoids the problem that multiple traffic experiments with mutually exclusive user traffic conditions cannot exist simultaneously on the same experimental layer in the traditional technology, thereby improving the experimental efficiency of the experimental layer.

Description of drawings

In order to explain the embodiments of the present invention or the technical solutions in the prior art more clearly, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only These are some embodiments described in the present invention. For those of ordinary skill in the art, other drawings can also be obtained according to these drawings without any creative effort.

Fig. 1a is the schematic diagram of the experimental platform in a kind of realization method of flow experiment of the present invention;

Figure 1b is a schematic flowchart of a method for implementing a flow experiment of the present invention;

1c is a schematic diagram of an experiment domain divided in a target experiment layer in a method for implementing a flow experiment of the present invention;

2 is a schematic structural diagram of a device for realizing a flow experiment of the present invention;

FIG. 3 is a schematic structural diagram of a device for realizing a flow experiment of the present invention.

Detailed ways

Embodiments of the present invention provide a method, a device, and an electronic device for implementing a flow experiment.

In order to make those skilled in the art better understand the technical solutions of the present invention, some concepts are briefly introduced first.

(1) Traffic experiment

The traffic experiment is to divide part of the traffic, and test the effect comparison between different situations, versions, and functions by modifying one or more experimental parameter thresholds when the program is executed. For example, if you need to conduct a traffic experiment on the effect of a page element in a web page being set to different image styles, you can set different image styles in different sub-traffic experiments in the experimental domain; for example, sub-traffic experiments Both sub-flow experiment 1 and sub-flow experiment 2 are located in experiment domain 1, and sub-flow experiment 1 corresponds to picture style 1, and sub-flow experiment 2 corresponds to picture style 2.

Then, when a user A browses the web page, if the user traffic enters the sub-traffic experiment 1 of the experimental domain 1, the parameters of the page will be configured according to the corresponding traffic experiment parameter thresholds of the sub-traffic experiment 1, that is, The page element is displayed as picture style 1, and correspondingly, the page element that user A sees on the page will be displayed according to picture style 1.

In addition, if another user B is also browsing the web page, and the user traffic of user B is assigned to sub-traffic experiment 2, the page element that user B sees on the page will be displayed according to picture style 2 .

That is to say, users who enter different sub-flow experiments will see different page display effects. Correspondingly, the user's operation behavior on the page can be recorded, for example, including which links the user clicked, executed What kind of specific behavior (for example, booking an item, etc.), and when a specific behavior is detected, it can be traced back to what led the behavior, for example, whether a page element was clicked, and if so, then Think that the behavior is guided by that page element, etc. In short, according to the user traffic entering each sub-traffic experiment, the user behavior data under various traffic experimental conditions can be analyzed and compared, so as to provide a data basis for system optimization.

(2) Experimental domain

The experimental domain is the experimental space divided in the experimental layer, including the thresholds of the traffic experimental parameters corresponding to each sub-traffic experiment and the default experimental parameter thresholds, which are used to realize the traffic experiment. Among them, there are at least two experimental domains in the same experimental layer, and the experimental conditions of each experimental domain are mutually exclusive.

In practical applications, there may be some special traffic experiments. For example, if the traffic experiment is for some specific users, when implementing the traffic experiment, you need to add experimental conditions, such as the gender, age, region, device used, browser type, operating system type, etc. For conditional traffic experiments, not all traffic in the experimental layer will pass through the traffic experiment.

For example, a conditional traffic experiment is set up in an experiment layer (for example, the degree of 30-40-year-old women's preference for a certain dress), the condition of the traffic experiment is that the user's gender is female and the age is 30-40 years old, then When a user traffic enters the experimental layer, it will first determine whether the user traffic meets the above conditions. If not, the relevant parameters will not be configured according to the experimental parameter thresholds of the traffic experiment.

At this time, in order to make full use of user traffic, in theory, multiple traffic experiments with mutually exclusive experimental conditions can be placed in one experimental layer. For example, two experiments with conditions can be placed in an experiment layer, and the two conditions are "female user" and "male user". When a user's traffic enters an experiment layer, the experiment platform can first determine whether it matches one of the two conditions. The experimental conditions of the experiment are matched. If there is a conditional matching flow experiment, the flow experiment will be passed. If there is no condition matching flow experiment, the flow experiment will not be passed, or the relevant parameters can be configured with default values.

However, in fact, because the conditions for delineating user traffic in the same experimental layer are unique, the same experimental layer can only conduct traffic experiments with the same experimental conditions in the same time period, and there cannot be multiple traffic experiments with mutually exclusive experimental conditions at the same time. Based on this, the embodiment of the present invention divides the experimental layer into multiple experimental domains, so that the same experimental layer can perform multiple flow experiments with different experimental conditions in the same time period, thereby improving the experimental efficiency of the experimental layer.

(3) Experimental platform

The experimental platform is the experimental system of the traffic experiment. Generally, the experimental platform can include multiple experimental layers. As shown in Figure 1a, it is an experimental platform including three experimental layers, wherein each experimental layer includes multiple experimental domains, for example, experimental layer 1 includes experimental domain A1, experimental domain A1, experimental domain A1, and experimental domain A1; The experiment layer 2 includes an experiment domain B1, an experiment domain B2, and an experiment domain B3; the experiment layer 3 includes an experiment domain C1 and an experiment domain C2.

In practical applications, experimenters can create and manage pages, experimental domains (including modules and/or layers, etc.), traffic experiments, etc. on the experimental platform, and record the identifiers of each experimental domain and the sub-traffic included in each experimental domain. Data such as the number and the identification of each sub-flow are stored in the database. There is a permission management system on the experimental platform, which can control the traffic experiment permissions at all levels of pages, modules, and experimental layers. In addition, through the experimental platform, you can also enter the traffic experiment effect data report system to view the effect data report of the traffic experiment (wherein, the traffic experiment effect data report system performs correlation analysis and calculation on the experimental information carried by the traffic and the effect brought by the traffic, and obtains The effect of the experiment is obtained, and the effect data is organized into a data report).

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

Example 1

As shown in FIG. 1, an embodiment of the present invention provides a method for implementing a traffic experiment. The execution subject of the method includes but is not limited to a server, a personal computer, a notebook computer, a tablet computer, a smart phone, etc. An intelligent electronic device that performs predetermined processing procedures such as numerical calculations and/or logical calculations. The server may be a single network server or a server group composed of multiple network servers, or a cloud composed of a large number of computers or network servers based on cloud computing (Cloud Computing).

It should be noted that the intelligent electronic device can operate alone to implement the present invention, or it can be connected to a network to implement the present invention through interactive operation with other intelligent electronic devices in the network. Wherein, the network where the computer equipment is located includes but is not limited to the Internet, a wide area network, a metropolitan area network, a local area network, a VPN network, and the like. The method may specifically include the following steps:

Step S101, determining a target experimental layer where the traffic experiment is located, where the target experimental layer includes multiple experimental domains;

In the experiment layer, the conditions of program execution can be refined into parameters and defined on the layer. At this time, the traffic experiment is to modify the parameter values of the layer and inject the modified values into the program execution context to achieve the test goal. Also, each layer can define one or more parameters, and each parameter has a default value. The experiment layer includes multiple experiment domains, and an experiment domain can set one or more parameters of the layer to a value different from the default value.

In one implementation, any experimental layer can be selected as the target experimental layer.

Step S102, according to the acquired attribute information of each user traffic, determine the experimental domain corresponding to each user traffic respectively.

User traffic refers to the network access users who access the services targeted by the traffic experiment. The business includes modules with different functions in a website or software. For example, in a browser, the business includes a big information business, a search business, an advertising business, and the like. For example, when the business of the traffic experiment is an advertising business, the user traffic refers to network access users accessing the advertising business.

Attribute information refers to characteristic information possessed by user traffic, including but not limited to identification information and/or related information. Among them, the identification information is used to uniquely characterize the identity information of user traffic, such as user ID, cookie, mobile phone number, etc., and related information such as gender, age, location, education, etc.

In an implementation manner, the attribute information corresponding to the user traffic can be obtained by analyzing the access request sent by the user traffic or the information simultaneously reported by the requesting terminal when sending the access request.

For example, when the experimental platform publishes a traffic experiment, the traffic experiment configuration file can be sent to the machine where the application is located through a file synchronization system (such as Zookeeper, etc.) (an application often has multiple machines), and the splitter component can read the experiment configuration file. , and store the experimental domain information (including module information, layer information, etc.) and parameter information related to the traffic experiment in the cache. When user traffic accesses the web server on the application machine, the user traffic can be intercepted by the interceptor on the web server and handed over to the diverter component for processing, thereby obtaining attribute information of the user traffic.

Wherein, the experiment domain is an experiment space divided in the target experiment layer, and the experiment domain is used to implement a benchmark flow experiment and at least one sub-flow experiment, wherein the benchmark flow experiment is a control experiment, which is used to compare with the sub-flow experiment. Flow experiment to compare the effect.

As shown in Figure 1c, it is a schematic diagram of the experimental domain divided in the target experimental layer, wherein the target experimental layer includes experimental domain A1, experimental domain A2, experimental domain A3, ... experimental domain An and other n experimental domains and the rest A flow storage area, and each experiment domain includes a baseline flow experiment and at least one subflow experiment.

For example, taking the experiment domain A2 as an example, the experiment domain A2 may include a base flow experiment BaseExp, a subflow experiment Exp1, a subflow experiment Exp2, and a remaining flow sub-storage area RemainFlow.

Among them, the flow experiment is not placed in the benchmark flow experiment area. The benchmark flow experiment reflects the current effect and is used to compare the effect of the other subflow experiments in the experimental domain. You can compare the subflow experiments in the experimental domain. Effect.

Among them, the remaining traffic storage area is used to store the remaining experimental traffic on the target experimental layer, used to create a new experimental domain or collect the heavy volume of the traffic experiment (usually, any experimental layer includes 100% of the user traffic).

For example, suppose an experimental layer includes two experimental domains, and traffic tests are performed in the two experimental domains respectively. The traffic experiment corresponding to one experimental domain needs to allocate 20% of the user traffic, and the traffic experiment corresponding to the other experimental domain needs to allocate 20% of the user traffic. If the allocated user traffic is 50%, the remaining 30% of the user traffic in the experiment layer is unallocated. At this time, the 30% user traffic can be stored in the remaining traffic storage area, so that the experiment layer needs to create a new experiment in the future When a domain is created, the part of the traffic experiment can be assigned to the newly created experiment domain.

Alternatively, in order to improve the utilization rate of user traffic, the user traffic corresponding to the completed traffic test can be collected into the remaining traffic storage area, so that a new traffic test can be implemented in the subsequent process.

It should be noted that when creating a new experimental domain, it is necessary to ensure that the user traffic weight corresponding to the current experimental domain remains unchanged, and the user traffic included in the remaining traffic storage area meets the user traffic required for the newly created experimental domain. Create a new experimental domain.

In an implementation manner, in order to enable user traffic to be randomly allocated to different traffic experiments, a hash algorithm can be configured at the experimental layer. The attribute information is hashed to determine the experimental domain corresponding to each user traffic.

For example, a hash operation may be performed on the attribute information of each user flow to determine the corresponding hash operation result; a modulo operation may be performed on the hash operation result to determine the experimental domain corresponding to each user flow.

Among them, when using the hash algorithm to perform hash operation on the attribute information of user traffic to determine the experimental domain corresponding to the user traffic, the following formula can be used but not limited to [1]:

m=hash(uid/pv+domain-layerid)% 1000[1]

Among them, m represents the identifier of the experimental domain corresponding to the user traffic, uid represents the attribute information of the user, pv represents the information carried in the access request sent by the user, and domain-layerid represents the identification information of the target experimental layer.

In this embodiment of the present invention, in the same experimental layer, the identification of the experimental domain into which user traffic enters may be determined by the identification of user traffic. In other words, the identification of the experimental domain to be entered is fixed. That is to say, no matter whether the user initiates a visit from a newly opened page, or a re-initiated visit such as refreshing the page or flipping the page, as long as the user ID remains unchanged, the corresponding user traffic will enter a certain experimental layer in each experimental layer. In an experimental domain, obviously, the page display effect that the user sees after refreshing can be the same as before refreshing, thereby avoiding the problem of interference between each experimental domain.

In addition, in this embodiment of the present invention, by considering user attributes when determining the experimental domain corresponding to user traffic, and using user attributes to allocate user traffic, it is possible to support more dimensional and more refined traffic experiments, and user traffic is matched according to its own user attributes The corresponding traffic experiment content realizes the personalized demand traffic experiment for different users.

Step S103: Determine the experimental parameter thresholds of the traffic experiment according to each user traffic and the policy set corresponding to the experimental domain, and implement the corresponding traffic experiment in the experimental domain.

In an implementation manner, the traffic test corresponding to the experimental domain can be implemented through the following steps:

Determine whether the attribute information of user traffic matches the preset conditions of the experimental domain;

The preset conditions refer to the experimental conditions for delineating user traffic, which can describe the characteristic information of the user traffic for the traffic test, such as the gender, age, region, device used, browser type, operating system type, etc. of the user traffic.

For example, when testing the audience of a certain dress, the preset condition of the traffic test can be set to the traffic of female users aged 30-40.

Following the above example, if a user's request to access the web server is parsed, and the attribute information of the user's traffic is obtained as follows: Gender: Male, Age: 19, it means that the attribute information of the user's traffic is inconsistent with the preset conditions of the experimental domain. match.

Or, if a user's request to access the web server is parsed, and the attribute information of the user's traffic is obtained as: gender: female, age: 32, it means that the attribute information of the user's traffic matches the preset conditions of the experimental domain.

When the attribute information of the user traffic matches the preset conditions of the corresponding experimental domain, determine the sub-traffic experimental area in the experimental domain corresponding to the user traffic;

Following the above example, if it is determined that the attribute information of the user traffic matches the preset conditions of the corresponding experimental domain, the sub-traffic experimental area in the experimental domain corresponding to the user traffic can be further determined.

In an implementation manner, the sub-traffic experimental area in the experimental domain corresponding to the user traffic can be determined according to the following formula [2]:

n=hash(uid/pv+exp-layerid)% 1000 [2]

Among them, n represents the identifier of the sub-flow experiment area corresponding to the user traffic, uid represents the attribute information of the user, pv represents the information carried in the access request sent by the user, and exp-layerid represents the identification information of the current sub-flow experiment.

Determine whether there is a sub-flow experiment in the sub-flow experiment area;

Through the above steps, after determining the sub-flow experiment area in the experiment domain corresponding to the user traffic, it is necessary to determine whether there is a sub-flow experiment in the sub-flow experiment area.

If there is a sub-flow experiment in the sub-flow experiment area, obtain the first strategy set corresponding to the sub-flow experiment, and determine the experimental parameter thresholds of the flow experiment according to the first strategy set, so as to realize the flow experiment corresponding to the experiment domain;

The first strategy set includes experimental parameters corresponding to the sub-flow experiments and thresholds corresponding to the experimental parameters.

If there is no sub-flow experiment in the sub-flow experiment area, a preset second strategy set of the experiment domain is obtained, and experimental parameter thresholds of the flow experiment are determined according to the preset second strategy set, so as to realize the flow experiment corresponding to the experiment domain.

The second strategy set includes preset default experimental parameters corresponding to the sub-flow experiment and thresholds corresponding to the experimental parameters.

Or, in an implementation manner, when the attribute information of the user traffic does not match the preset conditions of the corresponding experimental domain, the preset second policy set of the experimental domain can be obtained;

The experimental parameter thresholds of the traffic experiment are determined according to the preset second policy set, so as to realize the traffic experiment corresponding to the experimental domain.

An embodiment of the present invention provides a method for implementing a traffic experiment. By dividing a target experimental layer into at least two experimental domains, and assigning each user traffic to a corresponding experimental domain according to attribute information of each user traffic, finally based on each experiment The policy set of the domain determines the experimental parameter thresholds of the traffic experiment, so as to realize the traffic experiment corresponding to the experimental domain. In this way, since the preset conditions of each experimental domain are independent of each other and do not affect each other, there can be at least two traffic experiments with mutually exclusive preset conditions in the same experimental layer at the same time. The problem of multiple traffic experiments with mutually exclusive user traffic conditions cannot exist at the same time, thereby improving the experimental efficiency of the experimental layer.

In an implementation manner, the embodiment of the present invention further includes: saving the identifier of the experiment domain corresponding to the user traffic and the policy set of the experiment domain into the session storage corresponding to each user traffic, and displaying the page.

For example, when each user traffic enters the corresponding experimental domain or sub-traffic experimental area to conduct traffic experiments, etc., the information about the experimental domain, including the identification of the experimental domain and the strategy of the experimental domain, can be recorded in the session storage corresponding to the user traffic. collection, in order to achieve the purpose of judging the whereabouts of user traffic.

Embodiment 2

The above is the implementation method of the traffic experiment provided by the embodiment of the present invention. Based on the same idea, the embodiment of the present invention also provides a device for realizing the traffic experiment, as shown in FIG. 2 .

The apparatus 20 for the flow experiment includes: a first determination module 201, a second determination module 202 and a processing module 203, wherein:

The first determination module is used to determine the target experiment layer where the flow experiment is located, and the target experiment layer includes a plurality of experiment domains;

The second determining module is configured to respectively determine the experimental domain corresponding to each user traffic according to the acquired attribute information of each user traffic;

The processing module is used to determine the experimental parameter thresholds of the traffic test according to each user traffic and the policy set corresponding to the experimental domain, and implement the corresponding traffic experiment in the experimental domain.

In an implementation manner, the experiment domain can be used to implement a benchmark flow experiment and at least one subflow experiment, wherein the benchmark flow experiment is a control experiment and can be used to compare effects with a subflow experiment, and the subflow experiment is an experiment domain. Sub-experiments under the strategy set.

In an implementation manner, the second determining module may be configured to perform a hash operation on the attribute information of the user traffic for each user traffic to determine a corresponding hash operation result; perform a modulo operation on the hash operation result to determine Each user traffic corresponds to the experimental domain.

In one implementation, the processing module can be used to:

Determine whether the attribute information of each user traffic matches the preset conditions of the experimental domain;

When the attribute information of the user traffic matches the preset conditions of the corresponding experimental domain, determine the sub-traffic experimental area in the experimental domain corresponding to the user traffic;

Determine whether there is a sub-flow experiment in the sub-flow experiment area;

If there is a sub-flow experiment in the sub-flow experiment area, obtain the first strategy set corresponding to the sub-flow experiment, and determine the experimental parameter thresholds of the flow experiment according to the first strategy set, so as to realize the flow experiment corresponding to the experiment domain;

If there is no sub-flow experiment in the sub-flow experiment area, a preset second strategy set of the experiment domain is obtained, and experimental parameter thresholds of the flow experiment are determined according to the preset second strategy set, so as to realize the flow experiment corresponding to the experiment domain.

In an implementation manner, the processing module may further include: when the attribute information of the user traffic does not match the preset conditions of the experimental domain, acquiring a preset second policy set of the experimental domain; determining the traffic according to the preset second policy set The experimental parameter threshold of the experiment to realize the flow experiment corresponding to the experimental domain.

In an implementation manner, the apparatus may further include a display module, which may be configured to save the identifier of the experimental domain corresponding to the user traffic and the policy set of the experimental domain to the session storage corresponding to the user traffic, and display the page.

An embodiment of the present invention provides a device for implementing a traffic experiment. By dividing a target experimental layer into at least two experimental domains, and assigning each user traffic to a corresponding experimental domain according to the attribute information of each user traffic, finally based on each experiment The strategy set of the domain determines the experimental parameter thresholds of the traffic experiment to realize the traffic experiment corresponding to the experimental domain. In this way, since the preset conditions of each experimental domain are independent of each other and do not affect each other, there can be at least two preset conditions in the same experimental layer at the same time. Setting the traffic test with mutually exclusive conditions avoids the problem that multiple traffic tests with mutually exclusive user traffic conditions cannot exist simultaneously in the same experimental layer in the same time period in the traditional technology, thereby improving the experimental efficiency of the experimental layer.

Embodiment 3

3 is a schematic diagram of the hardware structure of a device for implementing a flow experiment according to various embodiments of the present invention,

The device 300 of the traffic experiment includes but is not limited to: a radio frequency unit 301 , a network module 302 , an audio output unit 303 , an input unit 304 , a sensor 305 , a display unit 306 , a user input unit 307 , an interface unit 308 , a memory 309 , and a processor 310 , and the power supply 311 and other components. Those skilled in the art can understand that the device structure of the flow experiment shown in FIG. 3 does not constitute a limitation on the device of the flow experiment, and the device of the flow experiment may include more or less components than those shown in the figure, or a combination of some components, or a different arrangement of components. In the embodiment of the present invention, the devices for the flow experiment include but are not limited to mobile phones, tablet computers, notebook computers, handheld computers, vehicle terminals, wearable devices, and pedometers.

The processor 310 is used to determine the target experimental layer where the traffic experiment is located, and the target experimental layer includes multiple experimental domains; according to the acquired attribute information of each user traffic, the experimental domain corresponding to each user traffic is determined respectively; A user traffic and a policy set of the corresponding experimental domain determine the experimental parameter thresholds of the traffic experiment, and implement the corresponding traffic experiment in the experimental domain.

In an implementation manner, the experiment domain can be used to implement a benchmark flow experiment and at least one subflow experiment, wherein the benchmark flow experiment is a control experiment and can be used to compare effects with the subflow experiment.

In an implementation manner, the processor may also be configured to perform a hash operation on the attribute information of the user traffic for each user traffic to determine a corresponding hash operation result; perform a modulo operation on the hash operation result to determine each An experimental domain corresponding to user traffic.

In an implementation manner, the processor is further configured to: determine whether the attribute information of the user traffic matches the preset condition of the corresponding experimental domain; when the attribute information of the user traffic matches the preset condition of the corresponding experimental domain, determine whether The sub-traffic experiment area in the experiment domain corresponding to user traffic; determine whether there is a sub-flow experiment in the sub-flow experiment area; if there is a sub-flow experiment in the sub-flow experiment area, you can obtain the first policy set corresponding to the sub-flow experiment The strategy set determines the experimental parameter thresholds of the traffic experiment to realize the traffic experiment corresponding to the experimental domain; if there is no sub-traffic experiment in the sub-traffic experiment area, the preset second strategy set of the experimental domain is obtained and determined according to the preset second strategy set The experimental parameter threshold of the flow experiment is used to realize the flow experiment corresponding to the experimental domain.

In an implementation manner, the processor is further configured to obtain a preset second strategy set of the experiment domain when the attribute information of the user traffic does not match the preset condition of the corresponding experiment domain; according to the preset second strategy The set determines the experimental parameter thresholds of the flow experiment to realize the flow experiment corresponding to the experimental domain.

In an implementation manner, the processor is further configured to save the identifier of the experiment domain corresponding to the user traffic and the policy set of the experiment domain into the session storage corresponding to the user traffic, and display the page.

An embodiment of the present invention provides a device for a traffic experiment. By dividing the target experimental layer into at least two experimental domains, and assigning each user traffic to the corresponding experimental domain according to the attribute information of each user traffic, finally based on the attribute information of each experimental domain The strategy set determines the experimental parameter thresholds of the traffic experiment to realize the traffic experiment corresponding to the experimental domain. In this way, since the preset conditions of each experimental domain are independent of each other and do not affect each other, at least two preset conditions can exist at the same experimental layer at the same time. The mutually exclusive traffic test avoids the problem that multiple traffic experiments with mutually exclusive user traffic conditions cannot exist in the same experimental layer in the same time period in the traditional technology, thereby improving the experimental efficiency of the experimental layer.

It should be understood that, in this embodiment of the present invention, the radio frequency unit 301 may be used for receiving and sending signals during sending and receiving of information or during a call. Specifically, after receiving the downlink data from the base station, it is processed by the processor 310; The uplink data is sent to the base station. Generally, the radio frequency unit 301 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, and the like. In addition, the radio frequency unit 301 can also communicate with the network and other devices for traffic experiments through a wireless communication system.

The device for the traffic experiment provides the user with wireless broadband Internet access through the network module 302, such as helping the user to send and receive emails, browse web pages, and access streaming media.

The audio output unit 303 may convert audio data received by the radio frequency unit 301 or the network module 302 or stored in the memory 309 into audio signals and output as sound. Also, the audio output unit 303 may also provide audio output related to a specific function performed by the device 300 of the traffic experiment (eg, call signal reception sound, message reception sound, etc.). The audio output unit 303 includes a speaker, a buzzer, a receiver, and the like.

The input unit 304 is used to receive audio or video signals. The input unit 304 may include a graphics processor (Graphics Processing Unit, GPU) 3041 and a microphone 3042, and the graphics processor 3041 captures images of still pictures or videos obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode data is processed. The processed image frames may be displayed on the display unit 306 . The image frames processed by the graphics processor 3041 may be stored in the memory 309 (or other storage medium) or transmitted via the radio frequency unit 301 or the network module 302 . The microphone 3042 can receive sound and can process such sound into audio data. The processed audio data can be converted into a format that can be transmitted to a mobile communication base station via the radio frequency unit 301 for output in the case of a telephone call mode.

The apparatus 300 for the flow experiment also includes at least one sensor 305, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor includes an ambient light sensor and a proximity sensor, wherein the ambient light sensor can adjust the brightness of the display panel 301 according to the brightness of the ambient light, and the proximity sensor can turn off the display panel when the device 300 in the flow experiment moves to the ear 3061 and/or backlight. As a kind of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in all directions (usually three axes), and can detect the magnitude and direction of gravity when stationary, and can be used to identify the device posture of the flow experiment (such as horizontal and vertical screen switching, related games, magnetometer attitude calibration), vibration recognition related functions (such as pedometer, tapping), etc.; the sensor 305 may also include a fingerprint sensor, a pressure sensor, an iris sensor, a molecular sensor, a gyroscope, a barometer, a hygrometer, Thermometers, infrared sensors, etc. will not be repeated here.

The display unit 306 is used to display information input by the user or information provided to the user. The display unit 306 may include a display panel 3061, and the display panel 5061 may be configured in the form of a Liquid Crystal Display (LCD), an Organic Light-Emitting Diode (OLED), or the like.

The user input unit 307 can be used to receive input numerical or character information, and generate key signal input related to user settings and function control of the device for flow experiments. Specifically, the user input unit 507 includes a touch panel 3071 and other input devices 3072 . The touch panel 3071, also referred to as a touch screen, can collect the user's touch operations on or near it (such as the user's finger, stylus, etc., any suitable object or accessory on or near the touch panel 3071). operate). The touch panel 5071 may include two parts, a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into contact coordinates, and then sends it to the touch controller. To the processor 310, the command sent by the processor 310 is received and executed. In addition, the touch panel 3071 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic waves. In addition to the touch panel 3071 , the user input unit 307 may also include other input devices 3072 . Specifically, other input devices 3072 may include, but are not limited to, physical keyboards, function keys (such as volume control keys, switch keys, etc.), trackballs, mice, and joysticks, which will not be repeated here.

Further, the touch panel 3071 can be covered on the display panel 3061. When the touch panel 3071 detects a touch operation on or near it, it transmits it to the processor 310 to determine the type of the touch event, and then the processor 310 determines the type of the touch event according to the touch The type of event provides corresponding visual output on display panel 3061. The touch panel 3071 and the display panel 3061 are used as two independent components to realize the input and output functions of the device for flow experiments, but in some embodiments, the touch panel 3071 and the display panel 3061 can be integrated to achieve The input and output functions of the device in the flow experiment are not limited here.

The interface unit 308 is an interface for connecting an external device to the device 300 of the flow experiment. For example, external devices may include wired or wireless headset ports, external power (or battery charger) ports, wired or wireless data ports, memory card ports, ports for connecting devices with identification modules, audio input/output (I/O) ports, video I/O ports, headphone ports, and more. The interface unit 308 may be used to receive input from external devices (eg, data information, power, etc.) and transmit the received input to one or more elements within the apparatus 300 for flow experiments or may be used in the flow experiment. Data is transferred between the device 300 and an external device.

The memory 309 may be used to store software programs as well as various data. The memory 309 may mainly include a stored program area and a stored data area, wherein the stored program area may store an operating system, an application program (such as a sound playback function, an image playback function, etc.) required for at least one function, and the like; Data created by the use of the mobile phone (such as audio data, phone book, etc.), etc. Additionally, memory 309 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device.

The processor 310 is the control center of the equipment of the flow experiment, and uses various interfaces and lines to connect various parts of the equipment of the whole flow experiment, by running or executing the software programs and/or modules stored in the memory 309, and calling the software programs and/or modules stored in the memory 309. 309, various functions and processing data of the device performing the traffic experiment, so as to perform overall monitoring of the device in the traffic experiment. The processor 310 may include one or more processing units; optionally, the processor 310 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, etc., and the modem The modulation processor mainly handles wireless communication. It can be understood that, the above-mentioned modulation and demodulation processor may not be integrated into the processor 310.

The device 300 for the flow experiment may also include a power supply 311 (such as a battery) for supplying power to various components. Power management and other functions.

Optionally, an embodiment of the present invention further provides a flow experiment device, including a processor 310, a memory 309, a computer program stored in the memory 309 and executable on the processor 310, and the computer program is executed by the processor. When 310 is executed, each process of the foregoing method embodiment of the traffic experiment is implemented, and the same technical effect can be achieved. To avoid repetition, details are not repeated here.

Embodiment 4

Embodiments of the present invention further provide a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium. When the computer program is executed by a processor, each process of the above-mentioned method embodiment of the traffic experiment can be realized, and the same can be achieved. The technical effect, in order to avoid repetition, will not be repeated here. The computer-readable storage medium is, for example, a read-only memory (Read-Only Memory, ROM for short), a random access memory (Random Access Memory, RAM for short), a magnetic disk, or an optical disk.

An embodiment of the present invention provides a computer-readable storage medium, by dividing a target experimental layer into at least two experimental domains, and assigning each user traffic to a corresponding experimental domain according to attribute information of each user traffic, and finally based on each experimental domain In this way, since the preset conditions of each experimental domain are independent of each other and do not affect each other, there can be at least two preset conditions in the same experimental layer that are mutually exclusive at the same time. It avoids the problem that multiple traffic experiments with mutually exclusive user traffic conditions cannot exist in the same experimental layer in the same time period in the traditional technology, thereby improving the experimental efficiency of the experimental layer.

As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block in the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to the processor of a general purpose computer, special purpose computer, embedded processor or other programmable data processing device to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing device produce Means for implementing the functions specified in a flow or flow of a flowchart and/or a block or blocks of a block diagram.

These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory result in an article of manufacture comprising instruction means, the instructions The apparatus implements the functions specified in the flow or flow of the flowcharts and/or the block or blocks of the block diagrams.

These computer program instructions can also be loaded on a computer or other programmable data processing device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process such that The instructions provide steps for implementing the functions specified in the flow or blocks of the flowcharts and/or the block or blocks of the block diagrams.

In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

Memory may include non-persistent memory in computer readable media, random access memory (RAM) and/or non-volatile memory in the form of, for example, read only memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.

Computer-readable media includes both persistent and non-permanent, removable and non-removable media, and storage of information may be implemented by any method or technology. Information may be computer readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), Flash Memory or other memory technology, Compact Disc Read Only Memory (CD-ROM), Digital Versatile Disc (DVD) or other optical storage, Magnetic tape cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include transitory computer readable media, such as modulated data signals and carrier waves.

It should also be noted that the terms "comprising", "comprising" or any other variation thereof are intended to encompass a non-exclusive inclusion such that a process, method, article or device comprising a series of elements includes not only those elements, but also Other elements not expressly listed, or which are inherent to such a process, method, article of manufacture, or apparatus are also included. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional identical elements in the process, method, article of manufacture, or device that includes the element.

As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The above descriptions are merely embodiments of the present invention, and are not intended to limit the present invention. Various modifications and variations of the present invention are possible for those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the scope of the claims of the present invention.

Claims (10)

1. A method for implementing a flow experiment is characterized by comprising the following steps:

determining a target experiment layer where a flow experiment is located, wherein the target experiment layer comprises a plurality of experiment domains;

respectively determining an experimental domain corresponding to each user flow according to the acquired attribute information of each user flow;

and determining an experimental parameter threshold value of the flow experiment according to each user flow and the strategy set of the experimental domain corresponding to the user flow, and realizing the corresponding flow experiment in the experimental domain.

2. The method of claim 1, wherein the experimental domain is used to implement a baseline flow experiment and at least one sub-flow experiment, wherein the baseline flow experiment is a control experiment used for effect comparison with the sub-flow experiment, and the sub-flow experiment is a sub-experiment under a strategy set of the experimental domain.

3. The method according to claim 1, wherein the determining, according to the obtained attribute information of each user traffic, an experimental domain corresponding to each user traffic respectively comprises:

performing hash operation on attribute information of each user flow to determine a corresponding hash operation result;

and performing modular operation on the hash operation result to determine an experiment domain corresponding to each user flow.

4. The method of claim 1, wherein determining the threshold of the experiment parameter of the flow experiment according to each user flow and the policy set of the experiment domain corresponding to the user flow, and implementing the corresponding flow experiment in the experiment domain comprises:

judging whether the attribute information of each user flow is matched with the preset conditions of the corresponding experimental domain;

when the attribute information of the user traffic is matched with the preset conditions of the corresponding experimental domain, determining a sub-traffic experimental region in the experimental domain corresponding to the user traffic;

judging whether a sub-flow experiment exists in the sub-flow experiment area;

if the sub-flow experiment exists in the sub-flow experiment area, acquiring a first strategy set corresponding to the sub-flow experiment, and determining an experiment parameter threshold of the flow experiment according to the first strategy set so as to realize the flow experiment corresponding to the experiment domain;

if the sub-flow experiment region does not have the sub-flow experiment, acquiring a preset second strategy set of the experiment domain, and determining an experiment parameter threshold of the flow experiment according to the preset second strategy set so as to realize the flow experiment corresponding to the experiment domain.

5. The method of claim 4, wherein determining the threshold of the experiment parameter of the flow experiment according to each user flow and the policy set of the experiment domain corresponding to the user flow, and implementing the corresponding flow experiment in the experiment domain, further comprises:

when the attribute information of the user flow is not matched with the preset condition of the corresponding experimental domain, acquiring a preset second strategy set of the experimental domain;

and determining an experiment parameter threshold value of the flow experiment according to the preset second strategy set so as to realize the flow experiment corresponding to the experiment domain.

6. The method of claim 1, further comprising:

and storing the identification of the experiment domain corresponding to each user flow and the strategy set of the experiment domain into a session storage corresponding to each user flow, and displaying a page.

7. An implementation device of a flow experiment is characterized in that the device comprises:

the device comprises a first determination module, a second determination module and a third determination module, wherein the first determination module is used for determining a target experiment layer where a flow experiment is located, and the target experiment layer comprises a plurality of experiment domains;

the second determining module is used for respectively determining an experimental domain corresponding to each user flow according to the acquired attribute information of each user flow;

and the processing module is used for determining the experimental parameter threshold of the flow test according to each user flow and the strategy set of the experimental domain corresponding to the user flow, and realizing the corresponding flow experiment in the experimental domain.

8. The apparatus of claim 7, wherein the determining module comprises:

for each user flow, performing hash operation on the attribute information of the user flow, and determining a corresponding hash operation result;

and performing modular operation on the hash operation result to determine the experiment domain corresponding to each user flow.

9. An electronic device comprising a processor, a memory, and a computer program stored on the memory and executable on the processor, the computer program, when executed by the processor, implementing the steps of the method of implementing a flow experiment according to any one of claims 1 to 6.

10. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of a method of carrying out a flow experiment according to one of claims 1 to 6.

Images