CN107871190B

CN107871190B - Service index monitoring method and device

Info

Publication number: CN107871190B
Application number: CN201610849587.2A
Authority: CN
Inventors: 马小鹏; 马涛; 李金辉
Original assignee: Alibaba Group Holding Ltd
Current assignee: Alibaba Group Holding Ltd
Priority date: 2016-09-23
Filing date: 2016-09-23
Publication date: 2021-12-14
Anticipated expiration: 2036-09-23
Also published as: CN107871190A

Abstract

The application discloses a business index monitoring method and a business index monitoring device, which can automatically predict the upper and lower limit thresholds of the data to be monitored of the business index to be monitored by adopting a mode of carrying out statistical analysis on historical monitoring sample data of the business index to be monitored, and determine whether the data to be monitored is abnormal data or not based on the predicted upper and lower limit thresholds; or, a variable point detection mode can be adopted to identify abnormal data in the time sequence data to be monitored of the service index to be monitored. Because a fixed same-loop ratio threshold value does not need to be set in a manual mode, the situations of false alarm, missed alarm and the like can be avoided on the basis of reducing the monitoring cost and expanding the monitoring range, and the sensitivity and the accuracy of service index monitoring are improved.

Description

Service index monitoring method and device

Technical Field

The present application relates to the technical field of service index monitoring, and in particular, to a service index monitoring method and apparatus.

Background

For various services such as internet advertisement services, in order to ensure normal operation of the services, real-time or timed monitoring of service indexes is often required.

Specifically, in the industry, a mode of manually configuring a same-loop ratio threshold of each service index is usually adopted (that is, operation and maintenance personnel manually configure service monitoring items one by one, and configure a fixed same-loop ratio threshold for each monitoring index of each service monitoring item), each service index is monitored, and when a same-loop ratio and a loop ratio change range simultaneously exceed corresponding static thresholds, an abnormal point is considered, so that the following problems may exist:

the first problem is that: because the service monitoring items and the service indexes are various, and each service monitoring item can have a plurality of monitoring time periods with different characteristics, 10-20 thresholds are generally required to be configured for one service monitoring item, so that the process of configuring the thresholds is complex and tedious, and the monitoring cost is high.

The second problem is that: when a new service monitoring item is added, if operation and maintenance personnel are not notified in time, each service index of the new service monitoring item is in a monitoring blind area state, and monitoring careless mistakes may occur.

The third problem is that: since the same ratio is that the detection value is compared with the last week at the same time, the loop ratio is that the detection value is compared with the last day at the same time, and the flow characteristics of weekend holidays and the like are far away from other dates, for the monitoring mode of manually setting the threshold value of the same loop ratio, the situations of false alarm, false alarm and the like are easy to occur, for example, the false alarm or the false alarm often occurs on the alternate dates of holidays and working days.

That is, the existing service index monitoring method has the problems of low accuracy, high monitoring cost, and the like, and therefore, it is urgently needed to provide a new service index monitoring method to solve the above problems.

Disclosure of Invention

The embodiment of the application provides a method and a device for monitoring a service index, which are used for solving the problems of low accuracy, high monitoring cost and the like in the conventional service index monitoring mode.

In one aspect, an embodiment of the present application provides a service index monitoring method, including:

acquiring data to be monitored of a service index to be monitored;

for each acquired data to be monitored, judging whether the data to be monitored meets the following conditions according to a set upper limit threshold and a set lower limit threshold corresponding to the data to be monitored: the value of the data to be monitored is not lower than the corresponding upper threshold or not higher than the corresponding lower threshold;

if the judgment result is yes, determining the data to be monitored as candidate abnormal data;

the upper threshold and the lower threshold corresponding to the data to be monitored are obtained by performing statistical analysis on historical monitoring sample data of the service index to be monitored at one or more historical synchronization time points corresponding to the time point of the data to be monitored.

On the other hand, an embodiment of the present application provides another service index monitoring method, including:

acquiring time sequence data to be monitored of a service index to be monitored;

performing variable point detection on the time sequence data to be monitored based on a set variable point detection algorithm to judge whether a variable point exists in the time sequence data to be monitored;

and if so, taking data corresponding to the time point corresponding to the change point in the time sequence data to be monitored as candidate abnormal data in the time sequence data to be monitored.

In another aspect, an embodiment of the present application provides a service index monitoring apparatus, including:

the data acquisition unit is used for acquiring the data to be monitored of the service index to be monitored;

the statistical analysis unit is used for performing statistical analysis on historical monitoring sample data of the to-be-monitored service index at one or more historical synchronization time points corresponding to the time point of the to-be-monitored data to obtain an upper threshold and a lower threshold corresponding to the to-be-monitored data;

the index judgment unit is used for judging whether the data to be monitored meets the following conditions or not according to the upper threshold and the lower threshold which are obtained by the analysis of the statistical analysis unit and correspond to the data to be monitored aiming at each data to be monitored obtained by the data obtaining unit: the value of the data to be monitored is not lower than the corresponding upper threshold or not higher than the corresponding lower threshold;

and the abnormality determining unit is used for determining the data to be monitored as candidate abnormal data if the judgment result aiming at the data to be monitored is determined to be yes according to the judgment result of the index judging unit.

In another aspect, an embodiment of the present application provides another service index monitoring apparatus, including:

the data acquisition unit is used for acquiring the time sequence data to be monitored of the service index to be monitored;

the variable point detection unit is used for carrying out variable point detection on the time sequence data to be monitored acquired by the data acquisition unit based on a set variable point detection algorithm so as to judge whether a variable point exists in the time sequence data to be monitored;

and the abnormity determining unit is used for taking data corresponding to the time point corresponding to the change point in the time series data to be monitored as candidate abnormal data in the time series data to be monitored if the change point exists in the time series data to be monitored according to the detection result of the change point detecting unit.

On the other hand, an embodiment of the present application further provides another service index monitoring apparatus, including:

a memory for storing software programs and modules;

a processor for executing the software programs and modules stored in the memory to perform the following operations:

acquiring data to be monitored of a service index to be monitored; and aiming at each acquired data to be monitored, judging whether the data to be monitored meets the following conditions according to a set upper limit threshold and a set lower limit threshold corresponding to the data to be monitored: the value of the data to be monitored is not lower than the corresponding upper threshold or not higher than the corresponding lower threshold;

the upper threshold and the lower threshold corresponding to the data to be monitored are obtained by the processor through statistical analysis of historical monitoring sample data of the service index to be monitored at one or more historical synchronization time points corresponding to the time point of the data to be monitored.

In another aspect, an embodiment of the present application further provides another service index monitoring apparatus, including:

a memory for storing software programs and modules;

acquiring time sequence data to be monitored of a service index to be monitored; performing variable point detection on the time sequence data to be monitored based on a set variable point detection algorithm to judge whether variable points exist in the time sequence data to be monitored or not; and if so, taking data corresponding to the time point corresponding to the change point in the time sequence data to be monitored as candidate abnormal data in the time sequence data to be monitored.

The beneficial effect of this application is as follows:

the embodiment of the application provides a business index monitoring method and a business index monitoring device, which can automatically predict the upper and lower limit thresholds of the data to be monitored of the business index to be monitored by adopting a mode of carrying out statistical analysis on historical monitoring sample data of the business index to be monitored, and determine whether the data to be monitored is abnormal data or not based on the upper and lower limit thresholds obtained by prediction; or, a variable point detection mode can be adopted to identify abnormal data in the time sequence data to be monitored of the service index to be monitored. Because a fixed same-loop ratio threshold value does not need to be set in a manual mode, the situations of false alarm, missed alarm and the like can be avoided on the basis of reducing the monitoring cost and expanding the monitoring range, and the sensitivity and the accuracy of service index monitoring are improved.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

Fig. 1 is a schematic flowchart illustrating a possible flow of a service indicator monitoring method according to an embodiment of the present application;

FIG. 2 is a schematic flow chart illustrating a possible dynamic threshold prediction according to an embodiment of the present disclosure;

FIG. 3 is a schematic diagram illustrating a possible process of variable point detection according to an embodiment of the present disclosure;

fig. 4 is a schematic diagram illustrating a possible application scenario of a service index monitoring method according to an embodiment of the present application;

fig. 5 is a schematic structural diagram of a service indicator monitoring apparatus according to a second embodiment of the present application;

fig. 6 is a schematic structural diagram of another service indicator monitoring apparatus according to a second embodiment of the present application;

fig. 7 is a schematic structural diagram of another service indicator monitoring apparatus according to a second embodiment of the present application;

fig. 8 is a schematic structural diagram of another service indicator monitoring apparatus according to the second embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application clearer, the present application will be described in further detail with reference to the accompanying drawings, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

The first embodiment is as follows:

in order to solve the problems of low accuracy, high monitoring cost and the like in the conventional service index monitoring mode, an embodiment of the application provides a service index monitoring method, which is applicable to monitoring various service indexes of various services such as internet advertisement services and the like. Fig. 1 is a schematic flow chart of a service index monitoring method according to an embodiment of the present disclosure. As shown in fig. 1, the service index monitoring method may include two monitoring schemes that can be operated independently:

one is a service index monitoring scheme based on dynamic threshold prediction, which may specifically include S11: predicting a dynamic threshold value; s12: judging whether the threshold value is punctured or not; s13: and determining candidate abnormal data and the like. In S11, the upper and lower limit thresholds of the data to be monitored may be automatically predicted in a manner of performing statistical analysis on historical monitoring sample data of the service index to be monitored at one or more historical synchronization time points corresponding to the time point of the data to be monitored; in S12, it may be determined whether the value of the data to be monitored exceeds a corresponding threshold based on the upper and lower threshold of the data to be monitored obtained through prediction, for example, whether the value of the data to be monitored satisfies the following condition: not lower than the corresponding upper threshold or not higher than the corresponding lower threshold; accordingly, in S13 of the monitoring scheme, candidate abnormal data may be determined according to the determination result of S12, for example, data to be monitored, which has a value not lower than the corresponding upper threshold or not higher than the corresponding lower threshold, may be used as the candidate abnormal data.

Another service index monitoring scheme based on the change point detection algorithm may specifically include S10: detecting a change point to judge whether the change point exists in the time sequence data to be monitored of the service index to be monitored; s13: and determining candidate abnormal data and the like. In S10, performing a change point detection on the to-be-monitored time series data of the to-be-monitored service indicator based on a set change point detection algorithm to determine whether a change point exists in the to-be-monitored time series data; accordingly, in S13 of the monitoring scheme, candidate abnormal data may be determined according to the determination result of S10, such as data corresponding to the time point corresponding to the change point in the time-series data to be monitored, as the candidate abnormal data in the time-series data to be monitored.

In the first monitoring scheme, the threshold value can be dynamically generated, operation and maintenance personnel are not needed to participate, and the dynamic threshold value can adapt to difference changes of holidays, weekends and workdays, flow characteristic differences of different monitoring time periods and the like to a certain extent, so that compared with a mode of manually setting a static threshold value, the sensitivity and the accuracy of service index monitoring can be improved on the basis of reducing monitoring cost and expanding a monitoring range.

In the second monitoring scheme, the change point detection does not need to depend on manual setting of the same-loop ratio threshold value, and continuous micro-change of data can be identified, so that the sensitivity and the accuracy of service index monitoring can be improved by avoiding occurrence of missing report and the like caused by continuous micro-change accumulation on the basis of reducing monitoring cost and expanding monitoring range.

That is to say, in the solution described in the embodiment of the present application, a statistical analysis mode may be adopted for historical monitoring sample data of the service index to be monitored, the upper and lower threshold values of the data to be monitored of the service index to be monitored are automatically predicted, and based on the upper and lower threshold values obtained by prediction, whether the data to be monitored is abnormal data is determined; or, a variable point detection mode can be adopted to identify abnormal data in the time sequence data to be monitored of the service index to be monitored. Because a fixed same-loop ratio threshold value does not need to be set in a manual mode, the situations of false alarm, missed alarm and the like can be avoided on the basis of reducing the monitoring cost and expanding the monitoring range, and the sensitivity and the accuracy of service index monitoring are improved. In addition, after the abnormal data of the service index to be monitored is identified, alarming and/or abnormal reason analysis and other operations can be performed on the abnormal data, so that economic loss brought to a user by the existence of the abnormal data is avoided as much as possible, and the application experience of the user is improved.

Furthermore, it should be noted that the two monitoring schemes provided in the embodiments of the present application may be used in combination with each other, besides being operated independently. For example, while or after monitoring the data to be monitored based on the dynamic threshold prediction, the data to be monitored may also be subjected to the point change detection, or the data to be monitored may be subjected to the point change detection first, and then the data to be monitored may be monitored based on the dynamic threshold prediction, etc., so as to further improve the sensitivity and accuracy of the service index monitoring, which is not limited herein.

The steps involved in the above two monitoring schemes of the embodiment of the present application will be further described in detail with reference to fig. 2 and 3.

Alternatively, in the first monitoring scheme, before performing S11, the data to be monitored of the service index to be monitored may be obtained generally first.

Optionally, in the embodiment of the present application, the data to be monitored of the service index to be monitored may be obtained in the following manner:

receiving to-be-monitored data of a to-be-monitored service index pushed by a service server for storing to-be-monitored service index data; or, actively acquiring the data to be monitored of the service index to be monitored from the service server.

Namely, the required data to be monitored can be acquired from the corresponding service server by adopting two modes of passive receiving or active acquisition. In addition, the acquired data to be monitored may be one or more data to be monitored of one or more service indexes to be monitored, which is not described in detail herein.

The Service server may be an online storage server or an online storage system such as Tair, Treasure, HBase, UPS, or an offline storage server or an offline storage system such as ODPS (Open Data Processing Service). In addition, taking the service to be monitored as an internet advertisement service (also referred to as an internet information delivery service) as an example, the service index to be monitored may include any one or more of the following indexes: the number of new user registrations of the service to be monitored, and feedback indexes such as Click Rate (CTR, Click Through Rate), single Click gain (CPC, Cost Per Click), thousand exposure gains (RPM), and the like are all described in detail.

Further, when acquiring the data to be monitored of the service index to be monitored, a corresponding time period to be monitored may be further specified (the specific value of the time period may be flexibly adjusted according to the actual situation), so as to acquire the data of the service index to be monitored in the set time period to be monitored from the service server, and use the data as the data to be monitored of the service index to be monitored.

In addition, it should be noted that the acquired data to be monitored may be time series data to be monitored, where a time series (or called dynamic series) refers to a series formed by arranging numerical values of the same index according to the time sequence of occurrence; in addition, in order to facilitate dynamic threshold tuning, the obtained time series data to be monitored can be time series data with positive distribution characteristics that are obvious enough, that is, time series data with small numerical value fluctuation (for example, the difference between adjacent data is smaller than a set first threshold) or corresponding small standard deviation (for example, the corresponding standard deviation is smaller than a set second threshold); the first threshold and the second threshold can be flexibly set according to actual conditions.

For example, taking the example of actively obtaining the data to be monitored of the service index to be monitored from the service server, if the data of the service index to be monitored in the time period of 05: 00-06: 00 of a certain day needs to be monitored, the time sequence data of the service index to be monitored in the time period of 05: 00-06: 00 of the certain day can be actively obtained from the service server, and the time sequence data is used as the data to be monitored of the service index to be monitored.

In addition, optionally, after the data to be monitored of the service index to be monitored is obtained, the data to be monitored can be formatted into a format required by subsequent data processing according to actual requirements, which is not described in detail herein.

Accordingly, after the data to be monitored of the service index to be monitored is obtained, the operation described in S11 may be executed. Specifically, as shown in fig. 2 (fig. 2 is a schematic flow chart of dynamic threshold prediction), in S11, for each data to be monitored of the service index to be monitored, an upper threshold and a lower threshold corresponding to the data to be monitored may be obtained through the following steps:

s21: historical monitoring sample data of the service index to be monitored at one or more historical synchronization time points corresponding to the time point of the data to be monitored is obtained.

Optionally, historical monitoring sample data corresponding to the data to be monitored may be acquired from historical data of the service index to be monitored in a set historical time period (the historical data may be acquired from a corresponding service server); the specific value of the set historical time period can be flexibly adjusted according to the actual situation, which is not described in detail herein.

For example, if the upper and lower limit thresholds of the data of the service index a to be monitored at 9:00am at 25/09/2016 are to be predicted, one or more index values of the service index a to be monitored at 9:00am in the past 1-2 months (which may be longer or shorter, and may be adjusted as needed) may be obtained, and each obtained index value is used as historical monitoring sample data of the service index a to be monitored at 9:00am at 25/09/2016.

In addition, the historical contemporaneous time point of each time point generally refers to the historical time point corresponding to the time point under the same time standard (such as 24 hours system or 12 hours system). For example, for a time point of 2016, 9:00am 31 month, 9:00am, the historical contemporaneous time point for that time point can be 2016, 9:00am 30 month 03 month, 20 day 9:00am 2016, 9:00am 31 month 2015, and so on.

Furthermore, it should be noted that, the historical monitoring sample data of the to-be-monitored service indicator at one or more historical synchronization time points corresponding to the time point of the to-be-monitored data (i.e., the historical monitoring sample data corresponding to the to-be-monitored data) may generally be the historical monitoring sample data whose corresponding time segment attribute is consistent with the corresponding time segment attribute of the to-be-monitored data.

That is, in order to distinguish the difference between samples and prevent the inaccuracy of the upper and lower threshold prediction caused by the excessive standard deviation, a data sample having the same characteristics as the data to be monitored can be selected as required when selecting the sample, so as to improve the subsequent prediction effect.

For example, taking the click rate of the internet advertisement service, the single click gain, the thousand exposure gains, and other feedback indexes as examples, because the traffic characteristics of the working day and the holiday are greatly different, when determining the historical monitoring sample data of each data to be monitored of the index, the time segment attribute (such as the working day, the holiday, and the like) of each data to be monitored can be determined first, and according to the time segment attribute of each data to be monitored, the historical data of which the corresponding time segment attribute is consistent with the time segment attribute corresponding to the data to be monitored is selected as the historical monitoring sample data of the data to be monitored.

In addition, in order to further improve the accuracy of the prediction of the upper and lower limit thresholds, after the historical monitoring sample data corresponding to the data to be monitored is acquired, noise data (i.e., data with abnormal occurrence) in the historical monitoring sample data can be filtered, so that the positive-too-distribution characteristics of the sample sequence are sufficiently obvious.

Optionally, before obtaining the historical monitoring sample data corresponding to the data to be monitored, the historical data of the service index to be monitored in the set historical time period may be formatted into a format required by subsequent data processing according to actual requirements, so as to obtain the historical monitoring sample data of the data to be monitored based on the formatted historical data.

S22: and determining the average value of the historical monitoring sample data of the to-be-monitored service index at one or more historical synchronization time points corresponding to the time point of the to-be-monitored data.

Optionally, determining an average value of historical monitoring sample data of the to-be-monitored service index at one or more historical synchronization time points corresponding to the time point of the to-be-monitored data may be specifically implemented as:

calculating a weighted average or an arithmetic average of historical monitoring sample data of the to-be-monitored service index at one or more historical synchronization time points corresponding to the time point of the to-be-monitored data;

and taking the calculated weighted average or arithmetic average as the average of historical monitoring sample data of the service index to be monitored at one or more historical synchronization time points corresponding to the time point of the data to be monitored.

That is, the average value of the historical monitoring sample data of the data to be monitored can be calculated by adopting a method of calculating a weighted average value or a method of calculating an arithmetic average value.

For example, if the historical monitoring sample data of the data to be monitored changes regularly in a long term (for example, continuously rises or continuously falls), an average value of the historical monitoring sample data of the data to be monitored can be calculated by calculating a weighted average value, wherein a weight corresponding to each sample data in the historical monitoring sample data of the data to be monitored can be set according to the amplitude of the regular change, and if the current time point is closer, the weight is set in a higher manner; or, if the historical monitoring sample data of the data to be monitored is relatively stable and not easy to change in the long term, the average value of the historical monitoring sample data of the data to be monitored can be calculated by adopting a mode of calculating an arithmetic average value.

In addition, it should be noted that, an average value of the historical monitoring sample data of the to-be-monitored service index at one or more historical synchronization time points corresponding to the time point of the to-be-monitored data may also be referred to as a reference value of the to-be-monitored data, which is not described herein again.

S23: taking the average value multiplied by a first set coefficient as an upper limit threshold corresponding to the data to be monitored, and taking the average value multiplied by a second set coefficient as a lower limit threshold corresponding to the data to be monitored, wherein the first set coefficient is not less than 1, and the second set coefficient is not more than 1.

In addition, the first setting coefficient and the second setting coefficient are usually different from each other, and are not described in detail herein.

As shown in fig. 2, after the execution of S21 and S22, S23 may not be executed, but the following steps may be executed:

s24: and determining the standard deviation of historical monitoring sample data of the to-be-monitored service index at one or more historical synchronization time points corresponding to the time point of the to-be-monitored data.

The standard deviation of the historical monitoring sample data corresponding to the data to be monitored can be calculated based on the average value calculated in step S22 and the existing standard deviation calculation formula, which is not described herein again.

S25: and taking the sum of the average value and the standard deviation multiplied by a third set coefficient as an upper limit threshold corresponding to the data to be monitored, and taking the difference between the average value and the standard deviation multiplied by a fourth set coefficient as a lower limit threshold corresponding to the data to be monitored, wherein the third set coefficient and the fourth set coefficient are not less than 0.

That is to say, the upper and lower limit thresholds corresponding to the data to be monitored can be obtained by adjusting according to a direct coefficient of the reference value of the data to be monitored, and in addition, can also be obtained by adjusting according to the reference value of the data to be monitored and the standard variance of the historical monitoring sample data of the data to be monitored, which is not limited herein.

Further, it should be noted that, after the upper and lower threshold values corresponding to the data to be monitored are obtained according to the prediction in the above steps for each data to be monitored of the service index to be monitored, the operations in S12 and S13 may be performed for the data to be monitored to determine whether the data to be monitored is the candidate abnormal data. That is, since there are usually a plurality of data to be monitored, S11, S12 and S13 can be performed alternately, which is not described herein.

In addition, it should be noted that, in the embodiment of the present application, the upper and lower threshold values corresponding to each piece of data to be monitored may be obtained by prediction based on historical monitoring sample data of the piece of data to be monitored in advance, or may be obtained by prediction based on historical monitoring sample data of the piece of data to be monitored in real time during the monitoring process; and after the upper and lower limit thresholds corresponding to the data to be monitored are obtained through prediction, the data to be monitored can be updated in real time or at regular time according to the latest sample data, which is not described in detail herein.

As can be seen from the above, in the first monitoring scheme described in S11, S12, and S13, since a statistical analysis may be performed on historical monitoring sample data of the service index to be monitored, the upper and lower limit thresholds of the data to be monitored of the service index to be monitored may be automatically predicted, and based on the predicted upper and lower limit thresholds, whether the data to be monitored is abnormal data is determined. Therefore, compared with the mode of manually setting the static threshold, the sensitivity and the accuracy of the service index monitoring can be improved on the basis of reducing the monitoring cost and expanding the monitoring range.

However, since some index data in the service index monitoring can be found only after the continuous micro-change is accumulated to a certain degree, in this case, it is difficult to find the index data regardless of the static threshold or the dynamic threshold. Therefore, in order to solve the problem and make up for the deficiency of the dynamic threshold mode in detecting the continuous micro-change, in the solution described in the embodiment of the present application, a variable point detection algorithm (changepoint algorithm) may be further adopted to perform variable point detection on the data to be monitored of the service index to be monitored, so as to identify candidate abnormal data (that is, the data to be monitored of the service index to be monitored is monitored by using the second monitoring scheme including S10 and S13 in fig. 1), so as to prevent occurrence of missing report and the like caused by continuous micro-change accumulation, and improve accuracy of monitoring the service index.

Alternatively, since a change point detection algorithm may be generally used to detect a change point in a given complete time series data, in the second monitoring scheme, before performing S10, the time series data to be monitored of the service index to be monitored may be generally obtained first.

Optionally, similar to the foregoing related description, the time series data to be monitored of the service index to be monitored may be obtained from the corresponding service server in both a passive receiving manner and an active obtaining manner. And when acquiring the time sequence data to be monitored of the service index to be monitored, the corresponding time period to be monitored can be specified. In addition, taking the service to be monitored as an internet advertisement service as an example, the service index to be monitored may include any one or more of the following indexes: the registration number of new users of the service to be monitored, the click rate, the single click income, the thousands of exposure income and the like are not described in detail.

Furthermore, in order to improve the accuracy of the change point detection, the acquired time series data to be monitored can be time series data with sufficiently obvious positive distribution characteristics, that is, time series data with small numerical value fluctuation or small corresponding standard deviation. And after the time series data to be monitored are obtained, the time series data to be monitored can be formatted into a format required by subsequent data processing according to actual requirements, which is not described in detail.

Accordingly, after the time-series data to be monitored of the service index to be monitored is obtained, the operation described in S10 may be performed. Specifically, as shown in fig. 3 (fig. 3 is a schematic flow chart of the change point detection), S10 may specifically include the following steps:

s31: and determining a difference accumulation sum sequence corresponding to the time sequence data to be monitored.

Alternatively, a cumulative sum sequence of differences (i.e., a CUSUM sequence) corresponding to the time series data to be monitored may be determined by:

sequentially determining difference accumulation sum data of all data in the time sequence data to be monitored;

and obtaining a difference accumulation sum sequence corresponding to the time sequence data to be monitored according to the difference accumulation sum data of the data in the time sequence data to be monitored which are determined in sequence.

In addition, for any data in the time series data to be monitored, determining the difference value accumulation sum data of the data, the method can comprise the following steps:

calculating a difference value between the value of any data and the average value of the time sequence data to be monitored, and a difference value between the value of each data in the time sequence data to be monitored, which is positioned before the time point of any data, and the average value of the time sequence data to be monitored;

and taking the sum of the calculated difference values as difference value accumulation sum data of any data.

It should be noted that, since the time series data to be monitored are series data at continuous time points, an average value of the time series data to be monitored can be calculated by calculating an arithmetic average value; of course, the average value of the time series data to be monitored can also be calculated by adopting a method of calculating a weighted average value, which is not limited.

That is, taking the time-series data to be monitored as [ X1, X2, X3, X4, …, X10] as an example, the difference cumulative sum data of the respective data therein can be calculated in the following manner:

1) calculating an average value Xavg of the time-series data to be monitored, for example, the average value Xavg may be (X1+ X2+ X3+ X4+.. + X10)/10, or (X1 × m1+ X2 × m2+.. + X10 × m10)/10, wherein m1, m2, m3, …, and m10 are weights of X1, X2, X3, X4, …, and X10, respectively;

2) and sequentially checking the amplitude of each sample data in the time series data to be monitored deviating from the average value, and accumulating the sum of the amplitudes of all sample data deviating from the average value before the moment of each sample data in the process to obtain the following result:

S1＝(X1-Xavg)；

S2＝S1+(X2-Xavg)；

...

S10＝S9+(X10-Xavg)。

wherein, S1, S2, S3, … and S10 are respectively difference sum data of X1, X2, X3, X4, … and X10.

That is, the accumulated sum of difference values corresponding to each data is the accumulated sum of the data and the deviation of each data before the data from a target value (i.e., the average value of the time-series data to be monitored), so that even a slight fluctuation in the process average value can result in a steady increase (or decrease) in the accumulated deviation value, and the accumulated sum of difference values sequence can reflect the accumulated trend of slight changes of the data in the time-series data to be monitored over time.

Optionally, in order to improve the accuracy of the subsequent change point determination to further improve the accuracy of the service indicator monitoring, before determining the accumulated sum sequence of the difference values corresponding to the time series data to be monitored, the method may further include the following steps:

s30: and sequentially determining difference data between each data in the time sequence data to be monitored and the reference value of the data, and obtaining a difference sequence corresponding to the time sequence data to be monitored according to the sequentially determined difference data between each data in the time sequence data to be monitored and the reference value of the data.

The reference value of each data in the time series data to be monitored can be calculated by adopting the reference value calculation mode in the first monitoring scheme, so that the accuracy of the reference value is improved, and the accuracy of the subsequent change point detection is improved. Of course, in the second monitoring scheme, the reference value corresponding to each data may be set manually by experience, and is not limited thereto.

Accordingly, the determining a cumulative sum sequence of differences corresponding to the time-series data to be monitored of S31 may include:

and determining a difference accumulation sum sequence corresponding to the time sequence data to be monitored according to the difference sequence.

The specific implementation of determining the difference cumulative sum sequence corresponding to the time series data to be monitored according to the difference sequence is similar to the specific implementation of directly determining the difference cumulative sum sequence corresponding to the time series data to be monitored according to the time series data to be monitored, for example, the difference cumulative sum data of each data in the difference sequence may be sequentially determined, and the difference cumulative sum sequence corresponding to the time series data to be monitored may be obtained according to the difference cumulative sum data of each data in the difference sequence that is sequentially determined, which is not described in detail herein.

That is to say, in this embodiment, the difference accumulation sum sequence may be determined directly based on the acquired time series data to be monitored (i.e., the detection data), and the algorithm is applied to detect the change point, or the difference sequence between the detection data and the reference value may be obtained by sequentially subtracting each data in the time series data to be monitored from the corresponding reference value, and then the difference accumulation sum sequence is determined based on the difference sequence and the algorithm is applied to detect the change point, which is not limited.

S32: and judging whether a change point exists in the difference value accumulation sum sequence.

Optionally, the determining whether a change point exists in the difference accumulated sum sequence may include:

judging whether a state change point of changing a first derivative from a positive number to a negative number or from the negative number to the positive number exists in the difference value accumulation sum sequence;

if the difference accumulation sum sequence exists (if the state change point does not exist, the difference accumulation sum sequence does not exist), whether the positive and negative of the first derivative of a continuously set number of points (the set number can be flexibly adjusted according to actual conditions) including the point next to the state change point after the state change point is the same as the positive and negative of the first derivative of the state change point is judged, and if the positive and negative of the first derivative of the state change point are the same, the state change point is taken as the change point in the difference accumulation sum sequence (namely, the existence of the change point in the difference accumulation sum sequence can be determined).

For example, if it is determined that the change state of the first derivative of each point on the curve corresponding to the difference accumulation sum sequence from positive to negative or negative to positive lasts for 3 points, the point at which the change first occurs is considered as the change point.

This is because, since the difference accumulation sum sequence can reflect the accumulation trend of slight changes of data in the time series data to be monitored over time, if the sequence has no change point, the curve corresponding to the sequence has no significant inflection point, and if the significant inflection point is found, the sequence has the change point.

Optionally, before taking the state change point as a change point in the difference accumulation sum sequence, the method may further include:

calculating confidence degrees of the difference values accumulated and the variable points in the sequence; and are

And determining that the confidence of the change points in the accumulated difference sum sequence is not lower than a set confidence threshold (the confidence threshold can be flexibly adjusted according to the actual situation).

That is, in order to improve the accuracy of service monitoring, after confidence verification is performed on the candidate change point in the difference accumulation sum sequence, the candidate change point may be used as the final required change point.

Alternatively, the confidence of the change points in the difference cumulative sum sequence may be obtained by iteratively calculating the difference Sdiffi between the maximum value and the minimum value of the cumulative variable of the time-series data to be monitored. The iterative sample difference is mainly defined by randomly disordering the sequence of the sample sequence (namely, the time series data to be monitored), and if the number of times that the Sdiffi is smaller than the Sdiff obtained by the original sequence in the iteration n times is m, the cumulative sum of the differences of the time series data to be monitored and the confidence of the change point in the sequence (or also referred to as the confidence of the change point in the time series data to be monitored) is m/n × 100%. Since the sample sequence with a confidence above 95% has stable confidence in more iterations above 1000 times, when the confidence of the change point is above 95%, the difference values can be accumulated and the change point in the sequence can be taken as the actual change point.

Further, after the change point is identified in the above manner, the operation of step S13 may be performed to use data corresponding to the time point corresponding to the change point in the time-series data to be monitored as candidate abnormal data in the time-series data to be monitored.

As can be seen from the above, in the second monitoring scheme described in S10 and S13, abnormal data in the time-series data to be monitored of the service index to be monitored can be identified by means of variable point detection. Because the change point detection does not need to depend on manual setting of the same-loop ratio threshold value, and continuous micro-change of data can be identified, the sensitivity and the accuracy of service index monitoring can be improved by avoiding missing report and the like caused by continuous micro-change accumulation on the basis of reducing the monitoring cost and expanding the monitoring range.

Further, as shown in fig. 1, since dynamic threshold detection or variable point detection is affected when the distribution of the data samples is not an ideal normal distribution, in order to filter out some unnecessary false alarms and further improve the accuracy of monitoring the service index, the method may further include the following steps:

s14: for each candidate abnormal data, according to a set same-ratio threshold and a ring-ratio threshold (the two thresholds can be flexibly set according to actual conditions) corresponding to the candidate abnormal data, judging whether the candidate abnormal data meets the following conditions, if so, taking the candidate abnormal data as actual abnormal data, otherwise, taking the candidate abnormal data as non-abnormal data:

the same-ratio variation amplitude is not lower than the corresponding same-ratio threshold, and the ring-ratio variation amplitude is not lower than the corresponding ring-ratio threshold.

That is to say, in the embodiments described in the present application, regression verification may be performed on the candidate abnormal data in a manner of a stationary threshold of a same-ring ratio to prevent false alarm caused by an influence of poor quality of a sample in an automatic detection manner, which is not described herein again.

In addition, it should be noted that after the abnormal data of the service index to be monitored is identified, an alarm (sound alarm and/or information alarm) and/or an abnormal reason analysis and other operations may be performed on the abnormal data, so as to avoid economic loss brought to the user by the existence of the abnormal data as much as possible, and improve the application experience of the user.

Finally, taking the application of the service index monitoring method shown in fig. 1 to the field of internet advertisements as an example, a specific implementation flow of service index monitoring in an application scenario shown in fig. 4 (fig. 4 is a schematic diagram of a possible application scenario of the service index monitoring method in this case) is briefly described, where the scenario may include: an advertisement server 41 (or may be referred to as an information delivery server) and a monitoring server 42, wherein:

various service index data generated during the operation of the internet advertisement service, such as any one or more index data of the number of new users registered in the internet advertisement service, the click rate of the internet advertisement (or advertisement slot), the single click revenue and the thousand exposure revenue, may be stored in the advertisement server 41.

Accordingly, the monitoring server 42 may obtain the data to be monitored of the advertisement service index to be monitored, such as time sequence data with sufficiently obvious just-too distribution characteristics of the advertisement service index to be monitored, from the advertisement server 41 by passive receiving or active obtaining according to actual requirements, and monitor the obtained data to be monitored. Such as, for example,

the monitoring server 42 may determine, for each acquired data to be monitored, whether the data to be monitored meets the following conditions according to a set upper threshold and a set lower threshold corresponding to the data to be monitored: the value of the data to be monitored is not lower than the corresponding upper threshold or not higher than the corresponding lower threshold; if the judgment result is yes, determining the data to be monitored as candidate abnormal data; the upper threshold and the lower threshold corresponding to the data to be monitored are obtained by performing statistical analysis on historical monitoring sample data of the advertising service index to be monitored at one or more historical synchronization time points corresponding to the time point of the data to be monitored.

That is, the monitoring server 42 may automatically predict the upper and lower threshold values of the data to be monitored of the advertisement service index to be monitored by performing statistical analysis on historical monitoring sample data of the advertisement service index to be monitored, and determine whether the data to be monitored is abnormal data based on the predicted upper and lower threshold values. Because the threshold can be dynamically generated without the participation of operation and maintenance personnel, and the dynamic threshold can adapt to the difference change of holidays, weekends and workdays, the flow characteristic difference of different monitoring time periods and the like to a certain extent, compared with the mode of manually setting a static threshold, the method can avoid the occurrence of the situations of false report, missed report and the like on the basis of reducing the monitoring cost and expanding the monitoring range, and improve the sensitivity and the accuracy of monitoring the internet advertisement service index.

In addition, if the data to be monitored acquired by the monitoring server 42 from the advertisement server 41 is time series data to be monitored, the monitoring server 42 may further perform a change point detection on the time series data to be monitored based on a set change point detection algorithm to determine whether a change point exists in the time series data to be monitored; and if so, taking data corresponding to the time point corresponding to the change point in the time sequence data to be monitored as candidate abnormal data in the time sequence data to be monitored.

That is, the monitoring server 42 may also identify abnormal data in the monitored time-series data of the advertisement service index to be monitored by using the change point detection. Because the change point detection does not need to depend on manual setting of the same-loop ratio threshold value and can identify continuous micro-change of data, the situations of false alarm, missed alarm and the like can be avoided on the basis of reducing the monitoring cost and expanding the monitoring range, and the sensitivity and the accuracy of monitoring the internet advertisement service indexes are improved.

In addition, the advertisement server 41 and the monitoring server 42 may be communicatively connected through a communication network, which may be a local area network, a wide area network, or the like. The advertisement server 41 may be an online storage server or an online storage system such as Tair, Treasure, HBase, UPS, or an offline storage server or an offline storage system such as ODPS. Monitoring server 42 may be any server device capable of supporting processing operations such as service indicator monitoring.

Further, it should be noted that the application scenario shown in fig. 4 is only shown for facilitating understanding of the spirit and principle of the present application, and the embodiments of the present application are not limited in any way in this respect. On the contrary, the embodiments of the present application may be applied to any applicable scenarios, such as monitoring scenarios of other non-internet advertisement services, or application scenarios of business prediction, decision support, and the like, which are not described again.

Finally, it should be noted that the solutions described in the embodiments of the present application are not limited by language, software, or hardware. However, in order to improve the efficiency of data processing, a programming language such as JAVA, R language, etc. convenient for statistical analysis, and hardware with high performance may be preferably used for implementation, and this is not described in detail in this embodiment of the application.

Example two:

based on the same inventive concept as the first monitoring scheme in the first embodiment of the present application, the second embodiment of the present application provides a service index monitoring device, and specific implementation of the service index monitoring device may refer to the description related to the first monitoring scheme in the first embodiment of the present application, which is not repeated herein. Specifically, as shown in fig. 5, the service index monitoring apparatus 50 may include:

a data obtaining unit 51, configured to obtain data to be monitored of a service index to be monitored;

a statistical analysis unit 52, configured to perform statistical analysis on historical monitoring sample data of the to-be-monitored service indicator at one or more historical synchronization time points corresponding to the time point of the to-be-monitored data, for each to-be-monitored data acquired by the data acquisition unit 51, to obtain an upper threshold and a lower threshold corresponding to the to-be-monitored data;

the index determining unit 53 is configured to, for each piece of data to be monitored acquired by the data acquiring unit 51, determine whether the piece of data to be monitored satisfies the following conditions according to the upper threshold and the lower threshold corresponding to the piece of data to be monitored, which are obtained by analyzing by the statistical analysis unit 52: the value of the data to be monitored is not lower than the corresponding upper threshold or not higher than the corresponding lower threshold;

an abnormality determining unit 54 operable to determine the data to be monitored as candidate abnormal data if it is determined that the determination result for the data to be monitored is yes according to the determination result of the index determining unit 53.

Optionally, the data obtaining unit 51 is specifically configured to receive to-be-monitored data of a to-be-monitored service index pushed by a service server for storing to-be-monitored service index data; or, actively acquiring the data to be monitored of the service index to be monitored from the service server.

The service index to be monitored can include any one or more of the following indexes: the registration number, click rate, single click income, thousands of exposure income and the like of new users of the service to be monitored.

Optionally, the statistical analysis unit 52 is specifically configured to, for each to-be-monitored data of the to-be-monitored service indicator, determine an average value of historical monitoring sample data of the to-be-monitored service indicator at one or more historical synchronization time points corresponding to a time point of the to-be-monitored data; and taking the average value multiplied by a first set coefficient as an upper limit threshold corresponding to the data to be monitored, and taking the average value multiplied by a second set coefficient as a lower limit threshold corresponding to the data to be monitored, wherein the first set coefficient is not less than 1, and the second set coefficient is not more than 1.

Or,

the statistical analysis unit 52 is specifically configured to determine, for each data to be monitored of the service index to be monitored, an average value and a standard deviation of historical monitoring sample data of the service index to be monitored at one or more historical synchronization time points corresponding to a time point of the data to be monitored; and taking the sum of the average value and the standard deviation multiplied by a third set coefficient as an upper threshold corresponding to the data to be monitored, and taking the difference between the average value and the standard deviation multiplied by a fourth set coefficient as a lower threshold corresponding to the data to be monitored, wherein the third set coefficient and the fourth set coefficient are not less than 0.

The historical monitoring sample data of the service index to be monitored at one or more historical synchronization time points corresponding to the time point of the data to be monitored can be the historical monitoring sample data of which the corresponding time period attribute is consistent with the corresponding time period attribute of the data to be monitored.

In addition, the average value of the historical monitoring sample data of the to-be-monitored service index at one or more historical synchronization time points corresponding to the time point of the to-be-monitored data may be calculated by the statistical analysis unit 52 in the following manner:

Further, as shown in fig. 5, the service index monitoring apparatus 50 may further include:

the regression verification unit 55 is configured to, after the abnormality determination unit 54 determines that the data to be monitored is the candidate abnormal data, determine whether the data to be monitored meets the following conditions according to a set geometric threshold and a ring ratio threshold corresponding to the data to be monitored, if yes, take the data to be monitored as actual abnormal data, otherwise, take the data to be monitored as non-abnormal data:

Further, based on the same inventive concept as the first monitoring scheme in the first embodiment of the present application, the second embodiment of the present application further provides another service index monitoring apparatus, and specific implementation of the another service index monitoring apparatus may refer to the description related to the first monitoring scheme in the first embodiment of the present application, which is not repeated herein. Specifically, as shown in fig. 6, the other service index monitoring apparatus 60 may include:

a memory 61 operable to store software programs and modules;

a processor 62 operable to perform the following operations by executing software programs and modules stored in the memory 61:

the upper threshold and the lower threshold corresponding to the data to be monitored are obtained by the processor 61 performing statistical analysis on historical monitoring sample data of the service index to be monitored at one or more historical synchronization time points corresponding to the time point of the data to be monitored.

Optionally, the processor 61 may be specifically configured to receive to-be-monitored data of a to-be-monitored service index pushed by a service server for storing to-be-monitored service index data; or, actively acquiring the data to be monitored of the service index to be monitored from the service server.

Further, the processor 61 may be specifically configured to determine, for each to-be-monitored data of the to-be-monitored service indicator, an average value of historical monitoring sample data of the to-be-monitored service indicator at one or more historical synchronization time points corresponding to a time point of the to-be-monitored data; and taking the average value multiplied by a first set coefficient as an upper limit threshold corresponding to the data to be monitored, and taking the average value multiplied by a second set coefficient as a lower limit threshold corresponding to the data to be monitored, wherein the first set coefficient is not less than 1, and the second set coefficient is not more than 1.

Or,

the processor 61 is specifically configured to determine, for each data to be monitored of the service index to be monitored, an average value and a standard deviation of historical monitoring sample data of the service index to be monitored at one or more historical synchronization time points corresponding to a time point of the data to be monitored; and taking the sum of the average value and the standard deviation multiplied by a third set coefficient as an upper threshold corresponding to the data to be monitored, and taking the difference between the average value and the standard deviation multiplied by a fourth set coefficient as a lower threshold corresponding to the data to be monitored, wherein the third set coefficient and the fourth set coefficient are not less than 0.

In addition, the average value of the historical monitoring sample data of the to-be-monitored service index at one or more historical synchronization time points corresponding to the time point of the to-be-monitored data may be calculated by the processor 62 in the following manner:

Further, the processor 62 may be further configured to, after determining that the data to be monitored is the candidate abnormal data, determine whether the data to be monitored meets the following conditions according to a set same-ratio threshold and a ring-ratio threshold corresponding to the data to be monitored, if yes, use the data to be monitored as actual abnormal data, otherwise, use the data to be monitored as non-abnormal data:

That is, in one possible design, the other service index monitoring apparatus 60 may include a memory 61 and a processor 62 in a structure, and the processor 62 is configured to support and execute corresponding functions in the first monitoring scheme in the first embodiment of the present application. The memory 61 is used for coupling with the processor 62, and it stores program instructions and data necessary for the processor 62 to execute the corresponding functions in the first monitoring scheme in the first embodiment of the present application.

The storage 61 may include a memory 611 and an external storage 612, the memory 611 is used for temporarily storing the operation data in the processor 62 and data exchanged with the external storage 612 such as a hard disk, and the processor 62 exchanges data with the external storage 612 through the memory 611. The Memory 611 may be one of a Non-Volatile Random Access Memory (NVRAM), a Dynamic Random Access Memory (DRAM), a Static Random Access Memory (Static RAM, SRAM), a Flash Memory, and the like; the external memory 612 may be a hard disk, optical disk, USB disk, floppy disk, or tape drive, etc.

Additionally, the processor 62 may be a Central Processing Unit (CPU), a general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, transistor logic, a hardware component, or any combination thereof. Which may implement or perform the various illustrative logical blocks, modules, and circuits described in connection with the disclosure. The processor 62 may also be a combination of computing functions, e.g., comprising one or more microprocessors, a combination of a DSP and a microprocessor, or the like.

Furthermore, as will be appreciated by those skilled in the art, the memory 61 and the processor 62 may be communicatively coupled via a bus 63 as shown in FIG. 6; the structure shown in fig. 6 is merely an illustration, and the structure of the another service index monitoring apparatus 60 is not limited thereto. For example, the other traffic indicator monitoring apparatus 60 may also include more or fewer components than shown in fig. 6, or have a different configuration than shown in fig. 6, etc.

Further, based on the same inventive concept as the second monitoring scheme in the first embodiment of the present application, a further service index monitoring device is also provided in the second embodiment of the present application, and specific implementation of the further service index monitoring device may refer to the description related to the second monitoring scheme in the first embodiment of the present application, which is not repeated herein. Specifically, as shown in fig. 7, the further service index monitoring apparatus 70 may include:

the data acquisition unit 71 is configured to acquire to-be-monitored time series data of to-be-monitored service indexes;

a change point detection unit 72, configured to perform change point detection on the time series data to be monitored acquired by the data acquisition unit 71 based on a set change point detection algorithm, so as to determine whether a change point exists in the time series data to be monitored;

the anomaly determination unit 73 is configured to, if it is determined that a change point exists in the time-series data to be monitored according to the detection result of the change point detection unit 72, take data corresponding to the change point in the time-series data to be monitored as candidate anomaly data in the time-series data to be monitored.

Optionally, the data obtaining unit 71 is specifically configured to receive to-be-monitored time series data of a to-be-monitored service index pushed by a service server for storing to-be-monitored service index data; or, actively acquiring the time sequence data to be monitored of the service index to be monitored from the service server.

Alternatively, the change point detecting unit 72 may be specifically configured to determine a difference accumulation sum sequence corresponding to the time-series data to be monitored, and determine whether a change point exists in the difference accumulation sum sequence.

Specifically, the change point detecting unit 72 is specifically configured to sequentially determine difference accumulation sum data of each data in the time series data to be monitored; and obtaining a difference accumulation sum sequence corresponding to the time sequence data to be monitored according to the difference accumulation sum data of each data in the time sequence data to be monitored which is determined in sequence.

For any data in the time series data to be monitored, the change point detection unit 72 is specifically configured to determine a difference cumulative sum of the data by:

Optionally, as shown in fig. 7, the further service indicator monitoring apparatus 70 may further include a difference sequence determining unit 74:

the difference sequence determining unit 74 may be configured to sequentially determine difference data between each data in the time series data to be monitored and a reference value of the data before the change point detecting unit 72 determines a sum sequence of differences corresponding to the time series data to be monitored, and obtain a difference sequence corresponding to the time series data to be monitored according to the sequentially determined difference data between each data in the time series data to be monitored and the reference value of the data;

accordingly, the change point detecting unit 72 is specifically configured to determine a difference cumulative sum sequence corresponding to the time-series data to be monitored according to the difference sequence.

Wherein, for any data in the time series data to be monitored, the reference value of the data may be obtained by the change point detecting unit 72 by:

obtaining historical monitoring sample data of the service index to be monitored at one or more historical synchronization time points corresponding to the time point of the data;

and calculating the average value of the acquired historical monitoring sample data, and taking the calculated average value as the reference value of the data.

Further alternatively, the change point detecting unit 72 may be specifically configured to determine whether there is a state change point in the difference accumulation sum sequence where the first derivative changes from a positive number to a negative number, or changes from a negative number to a positive number; and if so, judging whether the positive and negative of the first derivative of a continuously set number of points including the point next to the state change point after the state change point is the same as the positive and negative of the first derivative of the state change point, and if so, taking the state change point as the change point in the difference accumulation sum sequence.

In addition, the change point detecting unit 72 may be further configured to calculate a confidence level of the change point in the difference accumulation sum sequence before the state change point is taken as the change point in the difference accumulation sum sequence; and determining that the confidence of the change points in the accumulated sum sequence of the difference values is not lower than a set confidence threshold.

Further, as shown in fig. 7, the another service index monitoring apparatus 70 may further include:

the regression verification unit 75 is configured to, for each candidate abnormal data in the time series data to be monitored, determine whether the candidate abnormal data satisfies the following conditions according to a set geometric threshold and a circular threshold corresponding to the candidate abnormal data, if yes, use the candidate abnormal data as actual abnormal data, otherwise, use the candidate abnormal data as non-abnormal data:

Further, based on the same inventive concept as the second monitoring scheme in the first embodiment of the present application, the second embodiment of the present application further provides another service index monitoring device, and specific implementation of the another service index monitoring device may refer to the description related to the second monitoring scheme in the first embodiment of the present application, which is not repeated herein. Specifically, as shown in fig. 8, the another service index monitoring apparatus 80 may include:

a memory 81 operable to store software programs and modules;

the processor 82, which may be used to perform the following operations by executing the software programs and modules stored in the memory:

Optionally, the processor 82 is specifically configured to receive to-be-monitored time series data of a to-be-monitored service indicator, which is pushed by a service server for storing to-be-monitored service indicator data; or, actively acquiring the time sequence data to be monitored of the service index to be monitored from the service server.

Optionally, the processor 82 is specifically configured to determine a difference accumulation sum sequence corresponding to the time-series data to be monitored, and determine whether a change point exists in the difference accumulation sum sequence.

Specifically, the processor 82 is specifically configured to sequentially determine a difference cumulative sum of each data in the time series data to be monitored; and obtaining a difference accumulation sum sequence corresponding to the time sequence data to be monitored according to the difference accumulation sum data of each data in the time sequence data to be monitored which is determined in sequence.

Wherein, for any data in the time series data to be monitored, the processor 82 is specifically configured to determine a difference cumulative sum of the any data by:

Optionally, the processor 82 may be further configured to, before determining the accumulated difference sum sequence corresponding to the time series data to be monitored, sequentially determine difference data between each data in the time series data to be monitored and a reference value of the data, and obtain the accumulated difference sequence corresponding to the time series data to be monitored according to the sequentially determined difference data between each data in the time series data to be monitored and the reference value of the data, so as to determine the accumulated difference sum sequence corresponding to the time series data to be monitored according to the difference sequence.

For any data in the time series data to be monitored, the reference value of the data may be obtained by the processor 82 through the following steps:

Further optionally, the processor 82 is specifically configured to determine whether there is a state change point in the difference accumulation sum sequence where the first derivative changes from positive to negative or from negative to positive; and if so, judging whether the positive and negative of the first derivative of a continuously set number of points including the point next to the state change point after the state change point is the same as the positive and negative of the first derivative of the state change point, and if so, taking the state change point as the change point in the difference accumulation sum sequence.

In addition, the processor 82 is further configured to calculate a confidence level of the change point in the difference accumulation sum sequence before the state change point is taken as the change point in the difference accumulation sum sequence; and determining that the confidence of the change points in the accumulated sum sequence of the difference values is not lower than a set confidence threshold.

Further, the processor 82 may be further configured to, for each candidate abnormal data in the time series data to be monitored, determine whether the candidate abnormal data meets the following conditions according to a set geometric threshold and a set circular threshold corresponding to the candidate abnormal data, if yes, use the candidate abnormal data as actual abnormal data, otherwise, use the candidate abnormal data as non-abnormal data:

That is, in one possible design, the other service index monitoring apparatus 80 may include a memory 81 and a processor 82 in a structure, where the processor 82 is configured to support execution of corresponding functions in the second monitoring scheme in the first embodiment of the present application. The memory 81 is used for coupling with the processor 82, and it stores program instructions and data necessary for the processor 82 to execute the corresponding functions in the second monitoring scheme in the first embodiment of the present application.

The memory 81 may include a memory 811 and an external memory 812, the memory 811 is used for temporarily storing the operation data in the processor 82 and the data exchanged with the external memory 812 such as a hard disk, and the processor 82 exchanges data with the external memory 812 through the memory 811. The memory 811 may be one of a nonvolatile memory, a dynamic random access memory, a static random access memory, a Flash memory, and the like; the external memory 812 may be a hard disk, optical disk, USB disk, floppy disk, or tape drive, etc.

Additionally, the processor 82 may be a Central Processing Unit (CPU), a general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, transistor logic, a hardware component, or any combination thereof. Which may implement or perform the various illustrative logical blocks, modules, and circuits described in connection with the disclosure. The processor 82 may also be a combination of computing functions, e.g., comprising one or more microprocessors, a combination of a DSP and a microprocessor, or the like.

Furthermore, as will be appreciated by those skilled in the art, the memory 81 and the processor 82 may be communicatively coupled via a bus 83 as shown in FIG. 8; the structure shown in fig. 8 is merely an illustration, and the structure of the another service index monitoring apparatus 80 is not limited thereto. For example, the other traffic indicator monitoring apparatus 80 may also include more or fewer components than shown in fig. 8, or have a different configuration than shown in fig. 8, etc.

As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, apparatus (device), or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (devices) and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

While the preferred embodiments of the present application have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all alterations and modifications as fall within the scope of the application.

It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the spirit and scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations as well.

Claims

1. A service index monitoring method is characterized by comprising the following steps:

acquiring data to be monitored of a service index to be monitored, wherein the service index to be monitored comprises any one or more of the following indexes: the registration number, the click rate, the single click income and the thousands of exposure income of new users of the service to be monitored;

if the judgment result is yes, determining that the data to be monitored is candidate abnormal data, judging whether the data to be monitored meets the following conditions according to a set same-ratio threshold value and a ring-ratio threshold value corresponding to the data to be monitored, if so, taking the data to be monitored as actual abnormal data, otherwise, taking the data to be monitored as non-abnormal data: the same-ratio variation amplitude is not lower than the corresponding same-ratio threshold, and the ring-ratio variation amplitude is not lower than the corresponding ring-ratio threshold;

2. The method of claim 1, wherein obtaining data to be monitored of a service indicator to be monitored comprises:

receiving to-be-monitored data of a to-be-monitored service index pushed by a service server for storing to-be-monitored service index data; or,

and actively acquiring the data to be monitored of the service index to be monitored from the service server.

3. The method of claim 1, wherein for each data to be monitored of the service index to be monitored, the upper threshold and the lower threshold corresponding to the data to be monitored are obtained by:

determining the average value of historical monitoring sample data of the to-be-monitored service index at one or more historical synchronization time points corresponding to the time point of the to-be-monitored data;

taking the average value multiplied by a first set coefficient as an upper limit threshold corresponding to the data to be monitored, and taking the average value multiplied by a second set coefficient as a lower limit threshold corresponding to the data to be monitored, wherein the first set coefficient is not less than 1, and the second set coefficient is not more than 1.

4. The method of claim 1, wherein for each data to be monitored of the service index to be monitored, the upper threshold and the lower threshold corresponding to the data to be monitored are obtained by:

determining the average value and standard deviation of historical monitoring sample data of the to-be-monitored service index at one or more historical synchronization time points corresponding to the time point of the to-be-monitored data;

and taking the sum of the average value and the standard deviation multiplied by a third set coefficient as an upper limit threshold corresponding to the data to be monitored, and taking the difference between the average value and the standard deviation multiplied by a fourth set coefficient as a lower limit threshold corresponding to the data to be monitored, wherein the third set coefficient and the fourth set coefficient are not less than 0.

5. The method of claim 3 or 4, wherein determining an average of historical monitoring sample data of the to-be-monitored service indicator at one or more historical contemporaneous time points corresponding to the time point of the to-be-monitored data comprises:

6. The method of claim 1, wherein the historical monitoring sample data of the service indicator to be monitored at one or more historical contemporaneous time points corresponding to the time point of the data to be monitored is the historical monitoring sample data whose corresponding time segment attribute is consistent with the corresponding time segment attribute of the data to be monitored.

7. A service index monitoring method is characterized by comprising the following steps:

acquiring time sequence data to be monitored of a service index to be monitored, wherein the service index to be monitored comprises any one or more of the following indexes: the registration number, the click rate, the single click income and the thousands of exposure income of new users of the service to be monitored;

if the time point exists, taking data corresponding to the time point corresponding to the change point in the time sequence data to be monitored as candidate abnormal data in the time sequence data to be monitored;

for each candidate abnormal data in the time sequence data to be monitored, judging whether the candidate abnormal data meets the following conditions according to a set same-ratio threshold and a ring-ratio threshold corresponding to the candidate abnormal data, if so, taking the candidate abnormal data as actual abnormal data, otherwise, taking the candidate abnormal data as non-abnormal data: the same-ratio variation amplitude is not lower than the corresponding same-ratio threshold, and the ring-ratio variation amplitude is not lower than the corresponding ring-ratio threshold.

8. The method of claim 7, wherein obtaining time series data to be monitored of the service indicator to be monitored comprises:

receiving to-be-monitored time sequence data of a to-be-monitored business index pushed by a business server for storing to-be-monitored business index data; or,

and actively acquiring the time sequence data to be monitored of the service index to be monitored from the service server.

9. The method of claim 7, wherein performing change point detection on the time-series data to be monitored based on a set change point detection algorithm to determine whether a change point exists in the time-series data to be monitored comprises:

and determining a difference value accumulation sum sequence corresponding to the time sequence data to be monitored, and judging whether a change point exists in the difference value accumulation sum sequence.

10. The method of claim 9, wherein determining a cumulative sum sequence of differences corresponding to the time series data to be monitored comprises:

11. The method of claim 10, wherein determining, for any data in the time series data to be monitored, a difference accumulation sum of the any data comprises:

12. The method of claim 9, wherein prior to determining a cumulative sum sequence of differences corresponding to the time series data to be monitored, the method further comprises:

sequentially determining difference data of each data in the time sequence data to be monitored and the reference value of the data, and obtaining a difference sequence corresponding to the time sequence data to be monitored according to the sequentially determined difference data of each data in the time sequence data to be monitored and the reference value of the data;

determining a cumulative sum sequence of differences corresponding to the time series data to be monitored, including:

13. The method according to claim 12, wherein for any data in the time series data to be monitored, the reference value of the data is obtained by:

14. The method of claim 9, wherein determining whether a change point exists in the accumulated sum sequence of difference values comprises:

and if so, judging whether the positive and negative of the first derivative of a continuously set number of points including the point next to the state change point after the state change point is the same as the positive and negative of the first derivative of the state change point, and if so, taking the state change point as the change point in the difference accumulation sum sequence.

15. The method of claim 14, wherein prior to taking the state change point as a change point in the sequence of accumulated sums of difference values, the method further comprises:

And determining that the confidence of the change points in the accumulated sum sequence of the difference values is not lower than a set confidence threshold.

16. A service indicator monitoring apparatus, comprising:

the data acquisition unit is used for acquiring data to be monitored of a service index to be monitored, wherein the service index to be monitored comprises any one or more of the following indexes: the registration number, the click rate, the single click income and the thousands of exposure income of new users of the service to be monitored;

an anomaly determination unit, configured to determine that the data to be monitored is candidate anomalous data if the determination result for the data to be monitored is yes according to the determination result of the index determination unit, determine whether the data to be monitored meets the following conditions according to a set same-ratio threshold and a ring-ratio threshold corresponding to the data to be monitored, if so, take the data to be monitored as actual anomalous data, and otherwise, take the data to be monitored as non-anomalous data: the same-ratio variation amplitude is not lower than the corresponding same-ratio threshold, and the ring-ratio variation amplitude is not lower than the corresponding ring-ratio threshold.

17. The apparatus of claim 16,

the statistical analysis unit is specifically configured to determine, for each piece of data to be monitored of the service index to be monitored, an average value of historical monitoring sample data of the service index to be monitored at one or more historical synchronization time points corresponding to a time point at which the data to be monitored is located; and taking the average value multiplied by a first set coefficient as an upper limit threshold corresponding to the data to be monitored, and taking the average value multiplied by a second set coefficient as a lower limit threshold corresponding to the data to be monitored, wherein the first set coefficient is not less than 1, and the second set coefficient is not more than 1.

18. The apparatus of claim 16,

the statistical analysis unit is specifically configured to determine, for each piece of data to be monitored of the service index to be monitored, an average value and a standard deviation of historical monitoring sample data of the service index to be monitored at one or more historical synchronization time points corresponding to a time point at which the piece of data to be monitored is located; and taking the sum of the average value and the standard deviation multiplied by a third set coefficient as an upper threshold corresponding to the data to be monitored, and taking the difference between the average value and the standard deviation multiplied by a fourth set coefficient as a lower threshold corresponding to the data to be monitored, wherein the third set coefficient and the fourth set coefficient are not less than 0.

19. A service indicator monitoring apparatus, comprising:

the data acquisition unit is used for acquiring the time sequence data to be monitored of the service indexes to be monitored, wherein the service indexes to be monitored comprise any one or more of the following indexes: the registration number, the click rate, the single click income and the thousands of exposure income of new users of the service to be monitored;

an anomaly determination unit, configured to, if it is determined that a change point exists in the time series data to be monitored according to the detection result of the change point detection unit, use data corresponding to the change point in the time series data to be monitored as candidate anomaly data in the time series data to be monitored, determine, for each candidate anomaly data in the time series data to be monitored, according to a set compare-with threshold and ring-compare threshold corresponding to the candidate anomaly data, whether the candidate anomaly data meets the following conditions, if yes, use the candidate anomaly data as actual anomaly data, otherwise, use the candidate anomaly data as non-anomaly data: the same-ratio variation amplitude is not lower than the corresponding same-ratio threshold, and the ring-ratio variation amplitude is not lower than the corresponding ring-ratio threshold.

20. The apparatus of claim 19,

the change point detection unit is specifically configured to determine a difference accumulation sum sequence corresponding to the time series data to be monitored, and determine whether a change point exists in the difference accumulation sum sequence.

21. The apparatus of claim 20,

the variable point detection unit is specifically configured to sequentially determine difference accumulation sum data of each data in the time series data to be monitored; and obtaining a difference accumulation sum sequence corresponding to the time sequence data to be monitored according to the difference accumulation sum data of each data in the time sequence data to be monitored which is determined in sequence.

22. The apparatus of claim 21, wherein the apparatus further comprises a difference sequence determining unit:

the difference sequence determining unit is used for sequentially determining difference data of each data in the time series data to be monitored and a reference value of the data before the change point detecting unit determines the accumulated sum sequence of the differences corresponding to the time series data to be monitored, and obtaining the difference sequence corresponding to the time series data to be monitored according to the sequentially determined difference data of each data in the time series data to be monitored and the reference value of the data;

the change point detection unit is specifically configured to determine a difference accumulation sum sequence corresponding to the time series data to be monitored according to the difference sequence.

23. The apparatus of claim 20,

the change point detection unit is specifically used for judging whether a state change point of which the first derivative changes from a positive number to a negative number or changes from the negative number to the positive number exists in the difference value accumulation sum sequence; and if so, judging whether the positive and negative of the first derivative of a continuously set number of points including the point next to the state change point after the state change point is the same as the positive and negative of the first derivative of the state change point, and if so, taking the state change point as the change point in the difference accumulation sum sequence.

24. The apparatus of claim 23,

the change point detection unit is also used for calculating the confidence degree of the change point in the difference accumulation sum sequence before the state change point is taken as the change point in the difference accumulation sum sequence; and determining that the confidence of the change points in the accumulated sum sequence of the difference values is not lower than a set confidence threshold.

25. A service indicator monitoring apparatus, comprising:

a memory for storing software programs and modules;

acquiring data to be monitored of a service index to be monitored; and aiming at each acquired data to be monitored, judging whether the data to be monitored meets the following conditions according to a set upper limit threshold and a set lower limit threshold corresponding to the data to be monitored: the value of the data to be monitored is not lower than a corresponding upper threshold or not higher than a corresponding lower threshold, wherein the service index to be monitored comprises any one or more of the following indexes: the registration number, the click rate, the single click income and the thousands of exposure income of new users of the service to be monitored;

26. The apparatus of claim 25,

the processor is specifically configured to determine, for each piece of data to be monitored of the service index to be monitored, an average value of historical monitoring sample data of the service index to be monitored at one or more historical synchronization time points corresponding to a time point at which the data to be monitored is located; and taking the average value multiplied by a first set coefficient as an upper limit threshold corresponding to the data to be monitored, and taking the average value multiplied by a second set coefficient as a lower limit threshold corresponding to the data to be monitored, wherein the first set coefficient is not less than 1, and the second set coefficient is not more than 1.

27. The apparatus of claim 25,

the processor is specifically configured to determine, for each piece of data to be monitored of the service index to be monitored, an average value and a standard deviation of historical monitoring sample data of the service index to be monitored at one or more historical synchronization time points corresponding to a time point at which the piece of data to be monitored is located; and taking the sum of the average value and the standard deviation multiplied by a third set coefficient as an upper threshold corresponding to the data to be monitored, and taking the difference between the average value and the standard deviation multiplied by a fourth set coefficient as a lower threshold corresponding to the data to be monitored, wherein the third set coefficient and the fourth set coefficient are not less than 0.

28. A service indicator monitoring apparatus, comprising:

a memory for storing software programs and modules;

acquiring time sequence data to be monitored of a service index to be monitored; performing variable point detection on the time sequence data to be monitored based on a set variable point detection algorithm to judge whether variable points exist in the time sequence data to be monitored or not; if the abnormal data exists, taking data corresponding to the time point corresponding to the change point in the time series data to be monitored as candidate abnormal data in the time series data to be monitored, wherein the service index to be monitored comprises any one or more of the following indexes: the registration number, the click rate, the single click income and the thousands of exposure income of new users of the service to be monitored;

29. The apparatus of claim 28,

the processor is specifically configured to determine a difference accumulation sum sequence corresponding to the time series data to be monitored, and determine whether a change point exists in the difference accumulation sum sequence.

30. The apparatus of claim 29,

the processor is specifically configured to sequentially determine difference accumulation sum data of each data in the time series data to be monitored; and obtaining a difference accumulation sum sequence corresponding to the time sequence data to be monitored according to the difference accumulation sum data of each data in the time sequence data to be monitored which is determined in sequence.

31. The apparatus of claim 30,

the processor is further configured to, before determining the difference accumulation sum sequence corresponding to the time series data to be monitored, sequentially determine difference data between each piece of data in the time series data to be monitored and a reference value of the piece of data, and obtain the difference sequence corresponding to the time series data to be monitored according to the sequentially determined difference data between each piece of data in the time series data to be monitored and the reference value of the piece of data, so as to determine the difference accumulation sum sequence corresponding to the time series data to be monitored according to the difference sequence.

32. The apparatus of claim 29,

the processor is specifically configured to determine whether a state change point at which a first derivative changes from a positive number to a negative number or from a negative number to a positive number exists in the difference accumulation sum sequence; and if so, judging whether the positive and negative of the first derivative of a continuously set number of points including the point next to the state change point after the state change point is the same as the positive and negative of the first derivative of the state change point, and if so, taking the state change point as the change point in the difference accumulation sum sequence.

33. The apparatus of claim 32,

the processor is further configured to calculate a confidence level of the change point in the difference accumulation sum sequence before the state change point is taken as the change point in the difference accumulation sum sequence; and determining that the confidence of the change points in the accumulated sum sequence of the difference values is not lower than a set confidence threshold.