CN111625625B

CN111625625B - Method, device, computer equipment and storage medium for determining exception log

Info

Publication number: CN111625625B
Application number: CN202010460929.8A
Authority: CN
Inventors: 许景禧
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2020-05-27
Filing date: 2020-05-27
Publication date: 2024-12-17
Anticipated expiration: 2040-05-27
Also published as: CN111625625A

Abstract

The application discloses a method, a device, computer equipment and a storage medium for determining an abnormal log, wherein the method comprises the steps of obtaining log data to be analyzed, clustering the logs according to data composition characteristics of data in the logs to obtain clustered log sets and data composition characteristics corresponding to each log set, determining at least one abnormal detection rule matched with the data composition characteristics corresponding to the log sets for each log set, and determining abnormal logs with the abnormality in the log sets and abnormal data in the abnormal logs by utilizing the at least one abnormal detection rule corresponding to the log sets for each log set. The scheme of the application can reduce the complexity of determining the abnormal log from a large amount of log data.

Description

Method, device, computer equipment and storage medium for determining exception log

Technical Field

The present application relates to the field of data processing technologies, and in particular, to a method, an apparatus, a computer device, and a storage medium for determining an exception log.

Background

During the running of a software product (e.g., an application, or a system platform containing multiple applications, etc.), a log characterizing its running condition is recorded. By analyzing the log of the software product, the problems of the software product can be found in time.

The software product generally includes program modules for implementing different functional modes, and due to the differences of the different program modules, the log forms generated by the different program modules are different, so that the log data generated by the software product is large in data size and complex and various. Meanwhile, since the abnormal expression forms of different program modules in the software product are different, the abnormal analysis modes of the log data of the different program modules are also different. Based on the above, in order to locate an abnormal log representing that the software product has an abnormality from the log data of the software product, each log in the log data needs to be manually analyzed one by one, which results in higher complexity of determining the abnormal log.

Disclosure of Invention

In view of this, the present application provides a method, apparatus, computer device, and storage medium for determining an exception log to reduce the complexity of determining an exception log from a large amount of log data.

To achieve the above object, in one aspect, the present application provides a method for determining an exception log, including:

acquiring log data to be analyzed, wherein the log data comprises a plurality of logs;

Clustering a plurality of logs in the log data according to data composition characteristics of the data in the logs to obtain clustered log sets and data composition characteristics corresponding to each log set, wherein each log set comprises at least one log;

determining at least one abnormality detection rule matched with the data composition characteristics corresponding to the log sets aiming at each log set;

and determining an abnormal log with an abnormality in the log set and abnormal data in the abnormal log by utilizing at least one abnormality detection rule corresponding to the log set for each log set.

In one possible case, the clustering the plurality of logs in the log data according to the data composition characteristics of the data in the logs includes:

And clustering a plurality of logs in the log data according to the function identifiers and character features contained in the logs, wherein the function identifiers in the logs are used for representing the types of the function modes to which the running state information recorded in the logs belongs.

In yet another possible case, the method further includes:

Determining at least one target data item to which abnormal data in each abnormal log in the log set belongs under the condition that the abnormal log exists in the log set, wherein the target data item belongs to at least one data item in each log of the log set;

Constructing an abnormal identification curve graph of each target data item corresponding to the log set, wherein the abnormal identification curve graph comprises a numerical value change curve graph corresponding to the numerical value of the target data item in each log of the log set, and information of abnormal data in the target data item is marked in the numerical value change curve graph;

Displaying an abnormal identification curve graph of each target data item in the log set.

In yet another possible case, the method further includes:

For each log set, determining a target log sample set matched with data composition characteristics corresponding to the log set from a plurality of log sample sets corresponding to stored log sample data, wherein the log sample data comprises a plurality of log samples without abnormal data, and the plurality of log sample sets are obtained by clustering the plurality of log samples of the log sample data based on the data composition characteristics of the data in the log sample;

For each log set, determining a first log occurrence frequency of the log set in the log data and a second log occurrence frequency of a target log sample set corresponding to the log set in the log sample data, and determining the log set as a risk log set with an abnormal risk if an increase in the number of logs in the log set is determined based on the first log occurrence frequency and the second log occurrence frequency, wherein the first log occurrence frequency is a ratio of a first number of logs contained in the log set to a first total number of logs contained in the log data, and the second log occurrence frequency is a ratio between a second number of logs contained in the target log sample set corresponding to the log set and a second total number of logs contained in the log sample data;

And outputting prompt information aiming at the risk log set.

In yet another possible case, before said determining, for each of said log sets, at least one anomaly detection rule matching a data composition characteristic corresponding to said log set, further comprises:

Determining a set detection mode, wherein the detection mode is one of a manual detection mode and an automatic detection mode;

for each log set, determining at least one anomaly detection rule matched with the data composition characteristic corresponding to the log set, including:

If the set detection mode is an automatic detection mode, determining at least one abnormality detection rule matched with the data composition characteristics corresponding to the log sets for each log set;

The method further comprises the steps of:

displaying a log display interface under the condition that the set detection mode is an artificial detection mode, wherein the log display interface displays a plurality of logs in the log data;

In response to the log presentation interface detecting a log selection operation, determining a target log selected by the log selection operation;

determining a target log set to which the target log belongs;

And displaying the data of each log in the target log set.

In still another aspect, the present application further provides an apparatus for determining an exception log, including:

A log acquisition unit for acquiring log data to be analyzed, the log data comprises a plurality of logs;

The system comprises a log clustering unit, a log processing unit and a log processing unit, wherein the log clustering unit is used for clustering a plurality of logs in log data according to data composition characteristics of the data in the logs to obtain a plurality of clustered log sets and data composition characteristics corresponding to each log set, and the log sets comprise at least one log;

A rule matching unit, configured to determine, for each log set, at least one anomaly detection rule matching a data composition feature corresponding to the log set;

An anomaly determination unit, configured to determine, for each log set, an anomaly log in which an anomaly exists in the log set and anomaly data in the anomaly log by using at least one anomaly detection rule corresponding to the log set.

In yet another aspect, the present application also provides a computer device comprising a processor and a memory;

The processor is used for calling and executing the program stored in the memory;

The memory is used for storing the program at least for implementing the method of determining an exception log according to any one of the claims.

In yet another aspect, the present application further provides a storage medium having stored therein computer-executable instructions that, when loaded and executed by a processor, implement the method for determining an exception log as described in any one of the above.

According to the method, the system and the device, the plurality of logs in the log data to be analyzed are clustered according to the data composition characteristics of the logs, the data composition characteristics of the data in the logs can reflect which function of a software product is used for generating the logs, so that the logs recording the running states of the same function can be clustered to the same log set based on the data composition characteristics, the functions aimed by the logs in the log set can be reflected through the data composition characteristics corresponding to the log set, and therefore the abnormality detection rules corresponding to the corresponding functions (namely the abnormality detection rules matched with the data composition characteristics of the log set) can be determined as the abnormality detection rules applicable to the logs in the log set, so that the abnormality detection rules applicable to the logs in the log set can be determined, the corresponding abnormality detection rules can be directly used for carrying out abnormality detection on the logs in the log set, the complexity caused by manually analyzing the abnormality of the logs one by one can be avoided, and the complexity of determining the abnormality logs is reduced.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only embodiments of the present application, and other drawings may be obtained according to the provided drawings without inventive effort for a person skilled in the art.

FIG. 1 is a schematic diagram of a composition architecture of a scenario to which a method of determining an exception log of the present application is applicable;

FIG. 2 is a flow chart of a method for determining an exception log according to the present application;

FIG. 3 is a flow diagram illustrating the output of an anomaly identification graph for a log collection of existing anomaly logs in accordance with the present application;

FIG. 4 is a schematic diagram of a graph of the numerical variation of data items in the present application;

FIG. 5 is a schematic diagram of a graph of anomaly identification for data items in the present application;

FIG. 6 is a schematic diagram of an anomaly display interface incorporating anomaly reporting and anomaly identification graphs in accordance with the present application;

FIG. 7 is a schematic flow chart of a method of determining an exception log according to the present application;

FIG. 8 is a schematic diagram of an exception summary report of the present application;

FIG. 9 is a schematic flow chart of a method of determining an exception log according to the present application;

FIG. 10 is a schematic diagram showing a constitution of an apparatus for determining an abnormality log according to the present application;

FIG. 11 is a schematic diagram of a component architecture of a computer device in accordance with the present application.

Detailed Description

The method and the device are suitable for determining the abnormal log from a large amount of log data, so that complexity of determining the abnormal log is reduced. The log mentioned in the application can be an operation log generated in the operation process of software products such as application, and the problems of poor performance, faults and the like in the software products can be found out in time by carrying out exception analysis on the log.

The scheme of the application can be applied to a personal computer, a server or a server system formed by a plurality of servers, and in order to improve the efficiency of determining the abnormal log, the scheme of the application can also be applied to a cloud platform or other computing systems.

For easy understanding, the scenario that the scheme of the application is applied to the cloud platform is exemplified. As shown in fig. 1, which shows a schematic view of the composition architecture of one scenario to which the present application is applicable.

As can be seen from fig. 1, the scenario includes a cloud platform 10, which may include a plurality of cloud servers 101.

The scenario may also include a plurality of application clients 20 in which applications to be monitored may be run. Accordingly, during the running process of the application, the application client may generate logs related to the running process of the application, such as the use condition of the memory, the processor and the like in the running process of the application, and some data response conditions of the application.

For example, assuming that it is necessary to analyze whether there is an anomaly in the game application based on the log of the game application, the application client may be a game client that generates and stores a relevant log during the running of the game.

The application client 20 may send a log generated during the application running to the cloud platform 10.

Correspondingly, the cloud platform can obtain log data corresponding to different application clients, obtain multiple log data, and sequentially perform exception analysis on each log data.

The cloud platform also becomes a cloud computing platform, and is a network platform constructed based on cloud technology. Cloud technology (Cloud technology) refers to a hosting technology that unifies serial resources such as hardware, software, networks and the like in a wide area network or a local area network to realize calculation, storage, processing and sharing of data.

Cloud technology (Cloud technology) is a generic term of network technology, information technology, integration technology, management platform technology, application technology and the like based on Cloud computing business model application, and can form a resource pool, so that the Cloud computing business model application system is flexible and convenient as required. Background services of technical network systems require a large amount of computing, storage resources, such as image storage and encoding, etc. Along with the high development and application of the internet industry, each article possibly has an own identification mark in the future, the identification mark needs to be transmitted to a background system for logic processing, data with different levels can be processed separately, and various industry data needs strong system rear shield support and can be realized only through cloud computing.

Among them, cloud computing (cloud computing) is a computing mode that distributes computing tasks over a resource pool formed by a large number of computers, enabling various application systems to acquire computing power, storage space, and information services as needed. The network that provides the resources is referred to as the "cloud". Resources in the cloud are infinitely expandable in the sense of users, and can be acquired at any time, used as needed, expanded at any time and paid for use as needed.

As a basic capability provider of cloud computing, a cloud computing resource pool (abbreviated as a cloud platform, generally referred to as IaaS (Infrastructure AS A SERVICE) platform) is established, in which multiple types of virtual resources are deployed for external clients to select for use.

In the scenario of fig. 1, the processing of collecting and analyzing log data, determining an abnormal log, and the like can be completed by the cloud server of the cloud platform.

It can be appreciated that, in fig. 1, taking an example that the cloud platform directly obtains log data generated in the application client from the application client, in practical application, the cloud platform may also obtain the log data through other devices or through other network paths, which is not limited.

In addition, in fig. 1, taking log data obtained from an application client as an example, in practical application, if analysis needs to be performed on a certain system or network platform, the cloud platform may also obtain log data related to operation of the system or network platform.

It should be noted that, in fig. 1, for convenience of understanding, the application of the present application to a cloud platform is taken as an example, and when log data is analyzed by using a single computer device, a server or a server cluster, only the cloud platform in the scenario shown in fig. 1 needs to be replaced by a corresponding device or a server cluster, and the scenario is similar, and will not be described herein.

The following describes the scheme of the present application with reference to the flow chart.

As shown in fig. 2, which is a schematic flow chart illustrating a method for determining an exception log according to the present application, the method of the present embodiment may be applied to the aforementioned computer device, server, cloud platform, or the like. The method of the embodiment can comprise the following steps:

s201, acquiring log data to be analyzed, wherein the log data comprises a plurality of logs.

For example, the log data to be analyzed may be running logs of the software product, and each log in the log data may record running state data in the running process of the software product. Such as a running log of an application in a certain client, a running log of a certain system or network platform, etc.

It will be appreciated that different logs may be generated at different times for an application or software system or the like, and thus each log data set actually contains a plurality of logs generated at different times, respectively. For example, the log data may be a log data text, and the log data text may include a plurality of rows of logs, where the plurality of rows of logs may be regarded as a log sequence generated at a plurality of moments.

Alternatively, to distinguish between the logs in the log data, each log may be associated with a log identification that may be used to uniquely identify one of the logs in the log data. For example, the log identification may be a log sequence number. For example, in the case where the log data is composed of a plurality of lines of logs, each line of log corresponds to a line number, which is an identification of the log.

In practical application, a plurality of pieces of log data from different equipment ends may be obtained at the same time, but the analysis process of each piece of log data is the same, and the determination of the abnormal log can be performed by adopting the subsequent steps in the application.

S202, clustering a plurality of logs in the log data according to the data composition characteristics of the data in the logs to obtain a plurality of clustered log sets and the data composition characteristics corresponding to each log set.

Wherein the data composition characteristics of the log represent characteristics possessed by the data content contained in the log. For example, the data composition characteristics of the log may characterize the number and type of data items contained in the log, the specific form of the value corresponding to each data item, and so forth.

It will be appreciated that since a software product will typically have a plurality of different functional modes, e.g. the software product comprises different functional modules, the different functional modules implement the different functional modes. For example, the game application may include a data codec function and an image rendering function, and the game application may obtain logs generated in each of the two function modes during the respective execution of the two function modes.

Accordingly, the log recorded by the software product may include logs of the running status of different functional modes, and the data content and the specific form included in the logs corresponding to the different functional modes may be greatly different. But the data content and the specific form in the logs for the same functional mode are the same or nearly the same (i.e., the functional mode data composition characteristics are the same or similar), it is possible to determine which logs in the log data are the logs generated for the same functional mode based on the data composition characteristics. Based on this, logs for the same functional mode can be clustered into one log set based on the data composition characteristics.

Wherein each log set may include at least one log therein. It will be appreciated that logs having the same data composition characteristics will be clustered into the same log set, and therefore, the data composition characteristics commonly applicable to each log in the log set are the data composition characteristics corresponding to the log set. Similarly, the data corresponding to the log set constitutes the functional mode for the log in the log set reflected by the characteristic.

It can be understood that, since the same log set contains logs with the same data composition characteristics, one log can only belong to one log set, and the logs in different log sets have no overlap.

It is understood that the data composition characteristics of the log may be reflected in a variety of dimensions. As one possible scenario, the data composition characteristics of the log may be the function identification and character characteristics contained in the log. The function identifier in the log is used for characterizing the type of the function mode to which the running state information recorded in the log belongs, for example, the function identifier may be the name of the function mode, etc. The character features may include one or more of the number of real numbers contained in the log, the number of characters other than the real numbers, and specific contents of the characters other than the real numbers. Correspondingly, a plurality of logs in the log data can be clustered according to the function identifiers and character features contained in the logs.

For example, taking log data of a game application as an example, a rendering function mode is involved in the game application, and a file name, a resolution and a transmission rate of the rendering data representing the rendering mode are recorded in a log of the rendering function mode. The corresponding log may be a. Render, a resolution of 1920 x 180, and a transmission rate of 25bps, then the log may contain a function identified as "render", and three real numbers, 1980, 180 and 25, respectively, with characters other than real numbers including at least "resolution" and "transmission rate". Accordingly, if other logs also have the function identifier ". Render", have two strings of "resolution" and "transmission rate", and have three real numbers, then these logs can be considered as logs generated for the rendering mode as well, thereby clustering these logs into a log set.

S203, determining at least one abnormality detection rule matched with the data composition characteristics corresponding to the log sets for each log set.

The abnormality detection rule matched with the data composition characteristic corresponding to the log set is actually an abnormality detection rule applicable to the functional mode corresponding to the log in the log set.

For example, at least one anomaly detection rule may be configured in advance for each data composition feature, and accordingly, only one anomaly detection rule suitable for anomaly detection of each log in the log set may be determined according to a correspondence between the data composition feature and the anomaly detection rule.

As another example, at least one abnormality detection rule applicable to each of the different functional modes may also be preconfigured. Correspondingly, for each log set, a functional mode corresponding to the log in the log set can be determined based on the data composition characteristic corresponding to the log set, and then at least one abnormality detection rule suitable for the functional mode is searched.

The anomaly detection rule may include conditions that need to be satisfied by different types of data items in the log. The data items in the log may also be referred to as parameter items, and the different data items represent a status indicator of the functional mode recorded in the log. For example, for the data items in the log, the CPU usage rate, the memory occupancy rate, and the like may be included, and the corresponding anomaly detection rule may include that the CPU usage rate is greater than the set CPU usage rate, the memory occupancy rate is greater than the set occupancy rate, and the like.

It will be appreciated that the anomaly detection rules may be set according to different software products and in combination with actual needs, without limitation.

S204, determining an abnormal log with an abnormality in the log set and abnormal data in the abnormal log by using at least one abnormality detection rule corresponding to the log set for each log set.

It can be appreciated that if the logs in the log set match the abnormal conditions in the abnormality detection rule, it is indicated that the logs belong to the abnormal log. The log belongs to an abnormal log, and the numerical value of at least one data item in the abnormal log is abnormal, so that abnormal data in the abnormal log can be positioned based on an abnormal detection rule. For example, the exception data may be a data item in which an exception exists in the exception log and a value of the data item in which the exception exists.

In one possible case, in order to facilitate the detection of the cached data and the anomaly of the data, for each log set, at least one data item in the log set may be extracted, each extracted data item is used as a field in the data table to be constructed, and the data table is constructed based on the respective value of the at least one data item in each log in the log set.

The data items of each log in the log set are the same, so that all the data items of any log in the log set can be extracted. If each log in the log set contains two data items, namely, memory occupancy rate and CPU utilization rate, the two data items are extracted as two fields of a data table.

Each record in the data table constructed based on the log set corresponds to one log in the log set, and different logs correspond to different records in the data table. If the logs in the log set have unique line numbers in the log data, the records corresponding to the logs can be identified in the data table by the line numbers corresponding to the logs.

For example, assuming that the log set includes two logs with log line numbers 11 and 13, and fields extracted from the logs of the log set are respectively a CPU utilization rate and a memory occupancy rate, a data table constructed based on the log set may have the CPU utilization rate and the memory occupancy rate, and one record in the data table is the log with the log line number 11, and a value corresponding to the CPU utilization rate and a value corresponding to the memory utilization rate are recorded in the record. Correspondingly, the record with the log line number of 13 is also recorded in the data table.

After the data of each log in the log set is cached through the data table, after at least one abnormality detection rule applicable to the log set is determined, abnormality detection can be performed on each record in the data table based on the at least one abnormality detection rule, so as to detect an abnormality record with abnormal value of a field. Correspondingly, the log corresponding to the abnormal record is an abnormal log, and the data of the field with the abnormality in the abnormal record is the abnormal data in the abnormal log.

Therefore, in the embodiment of the application, a plurality of logs in the log data to be analyzed can be clustered according to the data composition characteristics of the logs, and the data composition characteristics of the data in the logs can reflect that the logs are generated aiming at the running states of which function of the software product, so that the logs recording the running states of the same function can be clustered to the same log set based on the data composition characteristics; moreover, the function aimed by the log in the log set can be reflected through the data composition characteristic corresponding to the log set, so that the abnormality detection rule corresponding to the corresponding function (namely, the abnormality detection rule matched with the data composition characteristic of the log set) can be determined as the abnormality detection rule applicable to the log in the log set, thereby enabling the abnormality detection rule applicable to the log in the log set to be determined, further, the abnormality detection can be directly carried out on the log in the log set by utilizing the corresponding abnormality detection rule, the complexity caused by manually analyzing the log abnormality one by one is avoided, and the complexity of determining the abnormal log is reduced.

It can be understood that, after determining the abnormal log in the log set in step S204, in order to facilitate the user to intuitively understand which data items in which logs have an abnormality, the present application may further output, for the log set, a graph corresponding to the data items that can reflect that the abnormality exists in the abnormal log. For example, referring to fig. 3, which shows a schematic flow chart of outputting an anomaly identification graph for a log set with anomaly logs in the present application, the flow of the present embodiment may include:

s301, for each log set, determining at least one target data item to which the abnormal data in each abnormal log in the log set belongs when the abnormal log exists in the log set.

For any exception log, there may be at least one data item with an exception, and the data item with the exception and its value are the exception data, so each exception data corresponds to one data item. For convenience of distinction, a data item in which a data abnormality exists in an abnormality log is referred to as a target data item.

For example, the exception log in which an exception exists in the log set includes log 1 and log 2, where an exception exists in the data of the data item a1 in log 1, and an exception exists in the data of the data item a2 in log 2. Then for log 1 the target data item to which the exception data belongs is data item a1, while the target data item to which the exception data belongs in log 2 is data item a2.

Wherein, the abnormal logs in the log set can be one or more. When a plurality of abnormal logs exist in the log set, the types and the numbers of the target data items with data abnormality exist in different abnormal logs are different, and the application can count all the target data items with data abnormality, thereby obtaining at least one target data item. As in the above example, although the data item a1 in the log 1 has a data abnormality, the data item a1 belongs to a target data item having an abnormality, and although the data item a2 in the log 1 does not have a data abnormality, the data of the data item a2 in the log 2 has an abnormality, and therefore, the data item a2 also belongs to a target data item having an abnormality.

S302, constructing an abnormal identification curve graph of each target data item corresponding to the log set.

The anomaly identification graph of the target data item comprises a numerical value change graph of the numerical value of the target data item in each log of the log set, and information of the anomaly data in the target data item is marked in the numerical value change graph.

It will be appreciated that each target data item corresponds to a value change graph that is constructed based on the values of the target data item in the respective ones of the set of journals, in which the values of the target data item for each of the set of journals may be presented.

For example, a coordinate system in the numerical variation graph of the target data item may include two coordinate axes perpendicular to each other, each value in one coordinate axis identifies one log in the log set, and different values in the other coordinate axis represent the numerical value of the target data item, and correspondingly, the numerical values of the target data items of the respective logs in the log set may be marked in the coordinate system in turn, so as to obtain a numerical variation graph reflecting the numerical variation of the target data item of the respective log in the log set.

For ease of understanding, reference may be made to fig. 4, fig. 4 showing a schematic diagram of a graph of the numerical variation of a data item. In fig. 4, a case where the data items included in the respective logs of the log set are CPU usage is taken as an example, and a numerical variation graph of the CPU usage is shown in fig. 4.

The abscissa in fig. 4 represents the identification number of each log, such as the line number of each log contained in the log set in the log data. Thus, the abscissa corresponding to each point in the numerical variation curve of fig. 4 corresponds to the line number of one log. As shown in fig. 4, the log set includes at least logs with line numbers 12 and 20, and logs with line number 25, etc., and of course, the logs with other line numbers included in the log set cannot be listed one by one.

Meanwhile, the ordinate of fig. 4 identifies a specific value of the CPU usage. Correspondingly, according to the CPU utilization rate recorded in each log in the log set, a corresponding numerical value can be marked on the ordinate of the corresponding position of the log in the coordinate system. For example, if the CPU utilization rate in the log of line number 12 is 40%, the CPU utilization rate at the position corresponding to the horizontal axis 20 in the numerical variation graph is 60%, as shown in fig. 4, which is the coordinates (20, 60).

It can be understood that, in order to enable a user to intuitively see which log has an abnormality in the value of the target data item based on the value change curve of the target data item, the application can mark information of abnormal data having the abnormality in the target data item in the value change curve of the target data item, thereby obtaining an abnormality identification curve. For example, a coordinate point where an abnormality exists is marked in a numerical value change graph of a target data item, or an abscissa where a log of abnormal data exists for the target data item.

For example, referring to FIG. 5, a schematic diagram of an anomaly identification graph of the present application is shown.

Fig. 5 is an abnormality identification curve obtained on the basis of the numerical variation graph of the CPU usage shown in fig. 5. As can be seen by comparing fig. 4 and 5, in fig. 5, there is an abnormal log for the CPU usage, and the coordinate position 501 corresponding to the abnormal log is marked on the abscissa of the abnormality identification graph, as indicated by the white dot on the abscissa in fig. 5. In order to facilitate visual observation of the abscissa positions of the respective exception logs, only two logs with CPU utilization having exceptions are taken as an example in fig. 5, namely, the logs with line numbers 20 and 1001, respectively, and the abscissas of the two exception logs are 20 and 1001, respectively.

It can be understood that in the embodiment of the present application, after determining the abnormal logs in the log set, numerical value change graphs of the target data items may be respectively constructed for the target data items having the abnormalities corresponding to the abnormal logs in the log set. After the log set is obtained by clustering, a numerical value change curve graph is respectively constructed for each data item related to each log in the log set, and correspondingly, after the abnormal log in the log set is determined, the numerical value change curve graph of each target data item with abnormal data can be directly determined from the constructed numerical value change curve graphs of each data item.

For each data item of any log set, a specific way of constructing a numerical variation graph of the data item may have a plurality of possible ways:

For example, in one possible implementation, the value of the data item in each log in the log set may be extracted, and a numerical variation graph of the data item may be constructed based on the value of the data item in each log.

For another example, in another possible implementation manner, in the case that a data table corresponding to each log set is respectively constructed for each log set, a numerical variation graph of the data item may be directly constructed based on the value of the field corresponding to the data item in each record in the data table. In this case, the column corresponding to each field in the data table is the value of each log in the data item, so that the numerical variation graph of the data item can be directly generated based on the value of each record in the column.

S303, displaying an abnormal identification curve graph of each target data item in the log set.

In one possible implementation, for each log set, after completion of the exception log analysis for that log set, an exception identification graph for each target data item in that log set may be output. In this case, for each log set, when the abnormality identification graph of the target data item in the log of the log set is displayed, the data combination feature corresponding to the target log set may also be identified, so that the user intuitively knows which function mode of the software product is specific to the abnormality identification graph of the target data item, and knows which logs in the log corresponding to the function mode have abnormality based on the abnormality identification graph.

In another possible implementation manner, the method can uniformly output the abnormal identification curve graph of each target data item in each log set with the abnormal log after analyzing all log sets.

It can be understood that in practical application, the application can also aim at the exception report of the log set while outputting the exception identification curve graph of the target data item in the log set, and the exception report can indicate the exception log with the exception in the log set, the specific information of the exception in the exception log and the like.

For example, an anomaly display interface may be output after each log collection is analyzed. And displaying the exception report of each log set in an exception display interface, and presenting an exception identification curve graph of each target data item of each log set in the exception display interface.

FIG. 6 is a schematic diagram of an anomaly display interface according to the present application.

As can be seen from fig. 6, an abnormality report 601 for various specific abnormality reasons and other information existing in the abnormality logs in the log set can be displayed in the abnormality display interface, and at the same time, an abnormality identification curve 602 of each target data item having an abnormality in the log set can be displayed below the abnormality report, so that a user can intuitively understand the specific abnormality reason, and can determine the log having an abnormality based on the abnormality identification curve.

It will be appreciated that in practical applications, if the number of logs generated for a certain functional mode is normally small, then if log data is detected that the number of logs for that functional mode is large, this is an indication that an anomaly may exist in that functional mode. Therefore, in order to find out that an abnormality exists in a certain functional mode in time, the method can analyze whether the log set belongs to a risk log set with the abnormality based on whether the abnormality exists in the frequency of occurrence of the log in the log set, and further determine the functional mode with the abnormality risk based on the risk log set in time.

The operation of detecting whether the log set is the risk log set may be two ways of determining the abnormal log in parallel with the operation of performing the abnormal log detection on the log set in the foregoing embodiment, and of course, it may also be that whether the log set belongs to the risk log set is analyzed on the basis of performing the abnormal log detection on the log set in the foregoing.

For ease of understanding, a case will be described below as an example, referring to fig. 7, which shows still another implementation flow of a method of determining an exception log according to the present application. The flow of this embodiment may include:

s701, acquiring a piece of log data to be analyzed, wherein the log data comprises a plurality of logs.

S702, clustering a plurality of logs in the log data according to the data composition characteristics of the data in the logs to obtain a plurality of clustered log sets and the data composition characteristics corresponding to each log set.

S703, regarding each log set, taking at least one data item of each log in the log set as a field in a data table to be constructed, and constructing the data table based on the respective numerical value of the at least one data item in each log in the log set.

S704, determining at least one abnormality detection rule matched with the data composition characteristics corresponding to the log sets for each log set, and determining abnormal records with abnormality in a data table of the log sets and abnormal data in the abnormal records based on the at least one abnormality detection rule.

Because each record in the data table corresponding to the log set corresponds to one log in the log set, the log corresponding to the abnormal record is an abnormal log, and accordingly, the abnormal data in the abnormal record is determined to be the abnormal data in the abnormal log corresponding to the abnormal record.

The above steps S701 to S704 may be specifically referred to the related description of the previous embodiments, and are not repeated here.

It should be noted that, in the present application, after log sets are clustered, a data table is generated for each log set, and data of each log in each log set is cached through the data table. However, if the data table is not generated, the abnormality detection rule is used to directly detect the abnormality of each log in the log set, which is also applicable to the present embodiment.

S705, outputting an abnormal display interface.

The abnormality display interface can display abnormality reports corresponding to the log sets, and the abnormality reports can prompt reasons, abnormal conditions and the like of abnormal logs in the log sets, and meanwhile, the abnormality display interface can also display an abnormality identification curve of target data items with abnormalities in the log sets.

For example, according to the abnormal log with the abnormality indicated in the click abnormality report, the user may present, in the abnormality display interface, an abnormal curve identification chart corresponding to the target data item of the abnormal log in the log set where the abnormal log is located, or an abnormal identification chart of each target data item in the log set where the abnormal log is located.

Of course, a case of an abnormality display interface is taken as an example here, and other cases of the abnormality display interface mentioned above are also applicable to the present embodiment.

S706, for each log set, a target log sample set that matches the data composition feature corresponding to the log set is determined from among a plurality of log sample sets corresponding to the stored log sample data.

The log sample data are obtained under the condition that the software product has no abnormal operation. Accordingly, the log sample data includes a plurality of log samples without abnormal data, and the plurality of log samples in the log sample data are clustered into a plurality of log sample sets.

For example, a plurality of log samples in the log sample data may be clustered in advance based on the data composition characteristics of the data in the log sample, so as to obtain a plurality of log sample sets and the data composition characteristics corresponding to each log sample set. Based on the data composition characteristics corresponding to the clustered log sets, the log sample sets corresponding to the same data composition characteristics can be queried.

For convenience of distinction, a log sample set matching the data composition characteristics of the log set is referred to as a target log sample set.

S707, determining, for each log set, a first log occurrence frequency of the log set in log data and a second log occurrence frequency of a target log sample set corresponding to the log set in the log sample data, and if it is determined that the number of logs in the log set is increased and abnormal based on the first log occurrence frequency and the second log occurrence frequency, determining the log set as a risk log set having an abnormal risk.

The first log occurrence frequency is a ratio of a first number of logs contained in the log set to a first total number of logs contained in the log data. For example, the number of logs included in the log set is 50, and 1000 logs are included in the log data obtained in step S701, so that the frequency of occurrence of the first log corresponding to the log set is 1/20.

Accordingly, the second log occurrence frequency is a ratio between a second number of logs contained in the target log sample set corresponding to the log set and a second total number of logs contained in the log sample data.

The method for determining that the number of the logs in the log set has the abnormal increase number based on the first log occurrence frequency and the second log occurrence frequency can be multiple. For example, for one log set, if the ratio of the first log occurrence frequency to the second log occurrence frequency is greater than a set threshold, it is determined that the number of logs in the log set is abnormal. For another example, for one log set, if the difference between the first log occurrence frequency and the second log occurrence frequency is greater than a set value, it is determined that the number of the logs in the log set is abnormal due to an increase in the number.

It can be understood that, for a log set, the second log occurrence frequency of the target log sample set corresponding to the log set represents the occurrence frequency of the log that can be obtained under normal conditions by the functional mode corresponding to the log in the log set, so if the first log occurrence frequency corresponding to the log set is far greater than the second log occurrence frequency, it is indicated that more logs appear for the functional mode, which is different from the normal condition, so that the possibility of abnormality of the functional mode is increased, and therefore, the log set corresponding to the functional mode is determined as a risk log set, which is beneficial to the user to discover the possible risk of the functional mode in time.

S708, outputting prompt information for the risk log set.

The prompt information for the risk log set is used for prompting the user that the abnormal risk exists in the functional mode for which the log of the risk log set is aimed. If a plurality of risk log sets exist, corresponding prompt information can be output for the plurality of risk log sets.

For example, the prompt information may include a function mode corresponding to the risk log set and a prompt information that the function mode is at risk.

As another example, as an alternative, the prompt information may include a rule supplement prompt for the risk log set, where the rule supplement prompt is configured to prompt a user to add an anomaly detection rule for a data constituent feature corresponding to the risk log set. It can be understood that if the log set belongs to the risk log set, it is indicated that the functional mode corresponding to the log set has an abnormal risk, so that in order to discover such abnormal risk in time, the user can be prompted to deploy the abnormal detection rule of the functional mode more specifically and in detail, so as to discover the possible abnormality of the functional mode based on the log corresponding to the functional mode more timely and reliably.

Accordingly, after obtaining at least one abnormality detection rule set by the user for the data combination feature (or the functional mode) corresponding to the risk log set, the at least one abnormality detection rule corresponding to the data combination feature (or the functional mode) may be stored.

In this embodiment, after each log set is clustered and a corresponding data table is generated, the operations of performing abnormality detection on the log set based on the abnormality detection rule corresponding to the log set and detecting whether the log set is a risk log set are performed in parallel, that is, steps S704 to S705 are performed in parallel with steps S706 to S708. However, it will be understood that, in practical application, steps S706 to S708 may be performed after determining the abnormal logs in each log set based on the abnormality detection rule of each log set, or steps S706 to S708 may be performed again for the log set in which no abnormal log exists after determining the abnormal log in each log set, which is not limited.

It will be appreciated that in the embodiment of the present application, multiple pieces of log data may be obtained simultaneously, for example, log data may be obtained from different application clients, so as to obtain multiple pieces of log data from different application clients. In the case of obtaining a plurality of pieces of log data, each piece of log data may be sequentially processed in the manner of any one of the above embodiments of the present application.

Specifically, after a plurality of log data are obtained and the abnormal logs existing in each log set in each log are analyzed, an abnormal summary report may be generated for the plurality of log data. The abnormal summary report presents abnormal conditions existing in each log data, wherein the abnormal conditions existing in each log data are summary of abnormal conditions corresponding to abnormal logs in each log set clustered by the log data.

For example, referring to FIG. 8, a schematic diagram of an exception summary report of the present application is shown.

The various fields in the exception summary report of FIG. 8 represent various exception conditions that may exist. For example, the abnormal conditions include high CPU usage, low memory, high transmission/reception code rate, and the like.

In the abnormal summary report of fig. 8, each row represents an abnormal summary condition corresponding to one log data, where the log data may be identified by a corresponding file name, etc., and for convenience of description and intuitiveness, in fig. 8, log data 1, log data2, etc. are used to represent different logs.

If the abnormal condition represented by a certain field exists in the log data, the value of the field corresponding to the row of the log data is 1, otherwise, the value is 0. For example, if the log data 1 has a high CPU usage rate, but no low memory exception exists, the value of the field corresponding to the "high memory usage rate" of the log data 1 in the exception summary report is 1, and the value of the field corresponding to the "low memory" is 0.

It can be understood that the abnormal summary report can be directly displayed after the analysis of the log data and the generation of the abnormal summary report are completed, or the abnormal summary report can be displayed after the instruction of displaying the summary report is detected to be input by the user.

It can be understood that in the above embodiments, the detection of the abnormal log of the log data is automatically performed by a computer or a cloud platform or the like. However, it can be appreciated that in practical applications, by clustering each log in the log data, it is also possible to achieve more effective and efficient determination of the abnormal log by the auxiliary user.

Specifically, after the plurality of log sets are clustered, if it is determined that the manual detection mode needs to be entered, a log display interface may be displayed, where the log display interface displays a plurality of logs in log data. On the basis, responding to the log display interface to detect the log selection operation, determining the target log selected by the log selection operation, and determining a target log set to which the target log belongs, thereby displaying the data of each log in the target log set.

Because the logs belonging to the same function mode are clustered to the same log set through clustering, when a user needs to check a certain log, the data of the log belonging to the same function mode with the log are displayed, so that the user can check a plurality of logs in the function mode at the same time, and whether the function mode is abnormal or not can be analyzed more conveniently and efficiently.

The specific manner of displaying the data of each log in the target log set may be various, and a case will be described as an example. For example, referring to fig. 9, which is a schematic flow chart of a method for determining an exception log according to the present application, the method of this embodiment may include:

S901, acquiring a piece of log data to be analyzed, wherein the log data comprises a plurality of logs.

S902, clustering a plurality of logs in the log data according to the data composition characteristics of the data in the logs to obtain a plurality of clustered log sets and the data composition characteristics corresponding to each log set.

S903, regarding each log set, taking at least one data item of each log in the log set as a field in a data table to be constructed, and constructing the data table based on the respective numerical value of the at least one data item in each log in the log set.

As an alternative, after the data table of the log set is constructed, a numerical variation graph of the data item represented by each field in the data table may also be generated for the data item. The process of generating the numerical variation curve of the data item may be referred to the related description above, and will not be described herein. On the basis, when the numerical value change curve graph of a certain data item needs to be displayed later, the numerical value change curve graph of the data item can be directly displayed, or when the constant identification curve graph of the data item needs to be generated, the numerical value change curve graph of the data item can be directly constructed.

S904, determining whether the set detection mode is an automatic detection mode, if so, executing step S905, otherwise, executing step S909;

Wherein, in this step S904, a set detection mode may be determined, which is one of a manual detection mode and an automatic detection mode, and if the set detection mode is the automatic detection mode, the operations of steps S905 to S908 are performed, and otherwise, step S909 is performed to assist the manual detection.

It can be understood that the detection mode may be a detection mode that prompts the user to input or select after clustering each log set, or may be a detection mode that displays a mode display interface after obtaining log data, where the user may select or input a detection mode desired by the user in the display mode display interface, and correspondingly, determine a set detection mode in response to a mode selection or input operation in the display mode display interface.

S905, determining at least one abnormality detection rule matched with the data composition characteristics corresponding to the log sets for each log set, and determining an abnormality record with an abnormality in a data table of the log set and abnormal data in the abnormality record based on the at least one abnormality detection rule.

S906, for each log set, determining at least one target data item to which the abnormal data in each abnormal log in the log set belongs when the abnormal log exists in the log set.

S907, constructing an abnormal identification curve graph of the target data item aiming at each target data item corresponding to the log set.

The anomaly identification graph is a numerical value change graph of the numerical value of the target data item in each log of the log set, and information of anomaly data in the target data item is marked in the numerical value change graph.

For example, the numerical value change graph of the target data item may be constructed for the target data item of the log set, or the numerical value change graph of the target data item may be constructed after S903, and then the numerical value change graph of the target data item may be directly obtained. Then, an abnormality identification graph is constructed based on the numerical value change graph of the target data item.

S908, displaying an abnormal identification curve graph of each target data item in the log set.

S909, in the case of the set detection mode being the manual detection mode, displaying a log presentation interface.

The log display interface displays a plurality of logs in log data.

The log display interface can display the log mark in setting display mode, which includes font thickening or brightness enhancement to highlight the log mark, so that the user can know the log to view the log relevant information.

Optionally, the value of each data item in the log can be marked in the log displayed in the log display interface, so that the user can intuitively see the data of each data item in the log. For example, data of a data item in the log may be displayed in a different color or with a different brightness from other data in the log, and so on.

S910, in response to the log presentation interface detecting the log selection operation, determining a target log selected by the log selection operation.

The log selection operation may be that the log in the log presentation interface is clicked, or other operations for triggering and selecting the log, etc.

For convenience of distinction, the log triggered by the log selection operation is referred to as a target log.

For example, when the log display interface displays the log identifier in a specific display manner, the clicked log corresponding to the clicked log identifier may be determined as the target log in response to the clicked operation of the log identifier.

S911, determining a target log set to which the target log belongs, and respectively determining respective numerical change curves of various data items corresponding to the target log set.

The log set to which the target log is clustered is the target log set to which the target log belongs.

The data items corresponding to the target log set are data items contained in the logs of the target log set, and the numerical change curve of each data item is a numerical change curve constructed based on the numerical values of the data items in each log of the target log set.

It may be appreciated that, if after the log set is clustered, only the data table of the log set is generated, and the numerical change curves are generated for the data items related to the log set respectively, which are not based on the data table, the step S911 may construct the numerical change curves corresponding to the data items related to the target log set respectively for the data table corresponding to the target log set. The process of constructing a numerical variation curve of a data item can be seen from the previous relevant description.

If the numerical value change curve corresponding to each data item in the target log set is generated before the manual detection mode is entered, the corresponding numerical value change curve can be directly inquired and obtained.

S912, displaying a graph display interface, wherein the graph display interface displays numerical change curves of various data items corresponding to the target log set.

The numerical change curve of each data item displayed by the graph display interface can be shown in fig. 4, and will not be described herein.

In this embodiment, if the set detection mode is an artificial detection mode, the present application may display a log display interface, and on this basis, if a user clicks a log in the log display interface, a log set to which the log is clustered is determined, and a numerical change curve corresponding to a data item related to each log in the log set is displayed, so that the user may intuitively see a value change condition of each data item recorded in a different log in a functional mode corresponding to the log, thereby being beneficial to the user to more efficiently find abnormal data, and thus efficiently locating the abnormal log.

Corresponding to a method of determining an exception log of the present application, the application also provides a device for determining the abnormal log. Referring to fig. 10, which is a schematic diagram illustrating a composition structure of an embodiment of an apparatus for determining an exception log according to the present application, the apparatus of this embodiment may include:

A log obtaining unit 1001, configured to obtain log data to be analyzed, where the log data includes a plurality of logs;

A log clustering unit 1002, configured to cluster a plurality of logs in the log data according to data composition characteristics of the data in the log, to obtain a plurality of clustered log sets and data composition characteristics corresponding to each log set, where the log set includes at least one log;

a rule matching unit 1003 configured to determine, for each log set, at least one abnormality detection rule that matches a data composition feature corresponding to the log set;

The anomaly determination unit 1004 is configured to determine, for each log set, an anomaly log in which an anomaly exists in the log set and anomaly data in the anomaly log, using at least one anomaly detection rule corresponding to the log set.

In one possible implementation, the log clustering unit includes:

and the log clustering subunit is used for clustering a plurality of logs in the log data according to the function identifiers and character features contained in the logs, wherein the function identifiers in the logs are used for representing the types of the function modes to which the running state information recorded in the logs belongs.

In yet another possible implementation manner, the apparatus may further include:

a data item determining unit, configured to determine, when an abnormal log exists in the log set, at least one target data item to which abnormal data in each abnormal log in the log set belongs, where the target data item belongs to at least one data item in each log in the log set;

The system comprises an abnormal curve construction unit, a data processing unit and a data processing unit, wherein the abnormal curve construction unit is used for constructing an abnormal identification curve graph of each target data item corresponding to the log set, the abnormal identification curve graph comprises a numerical value change curve graph corresponding to the numerical value of the target data item in each log of the log set, and the numerical value change curve graph is marked with the information of abnormal data in the target data item;

And the curve display unit is used for displaying an abnormal identification curve graph of each target data item in the log set.

In yet another possible implementation manner, the apparatus of the embodiment of the present application may further include:

The sample set matching unit is used for determining a target log sample set matched with the data composition characteristics corresponding to the log set from a plurality of log sample sets corresponding to the stored log sample data for each log set, wherein the log sample data comprises a plurality of log samples without abnormal data, and the plurality of log sample sets are obtained by clustering the plurality of log samples of the log sample data based on the data composition characteristics of the data in the log sample;

A risk log set determining unit, configured to determine, for each log set, a first log occurrence frequency of the log set in the log data and a second log occurrence frequency of a target log sample set corresponding to the log set in the log sample data, and determine the log set as a risk log set having an abnormal risk if an increase in the number of the logs in the log set is determined based on the first log occurrence frequency and the second log occurrence frequency, where the first log occurrence frequency is a ratio of a first number of the logs included in the log set to a first total number of the logs included in the log data, and the second log occurrence frequency is a ratio between a second number of the logs included in the target log sample set corresponding to the log set and a second total number of the logs included in the log sample data;

and the risk prompting unit is used for outputting prompting information aiming at the risk log set.

Optionally, the risk prompting unit includes:

The rule supplementing prompting unit is used for outputting rule supplementing prompts aiming at the risk log set, and the rule supplementing prompts are used for prompting a user to add an abnormality detection rule aiming at the data composition characteristics corresponding to the risk log set.

A pattern determining unit for determining a set detection pattern, which is one of a manual detection pattern and an automatic detection pattern, before the rule matching unit determines at least one abnormality detection rule matching the data composition characteristics corresponding to the log set;

The rule matching unit is specifically configured to determine, for each log set, at least one anomaly detection rule matching a data composition feature corresponding to the log set, where the set detection mode is an automatic detection mode;

The apparatus may further include:

The log display unit is used for displaying a log display interface under the condition that the set detection mode is the manual detection mode, wherein the log display interface displays a plurality of logs in the log data;

The log selection unit is used for responding to the log display interface to detect a log selection operation and determining a target log selected by the log selection operation;

The set determining unit is used for determining a target log set to which the target log belongs;

And the data display unit is used for displaying the data of each log in the target log set.

Optionally, the apparatus may further include:

The curve determining unit is used for respectively determining the numerical value change curves of various data items corresponding to the target log set before the data display unit displays the data of each log in the target log set, wherein the data items corresponding to the target log set are the data items contained in the logs of the target log set, and the numerical value change curve of each data item is a numerical value change curve constructed based on the numerical values of the data items in each log of the target log set;

The data display unit is specifically configured to display a graph display interface, where the graph display interface displays numerical variation curves of various data items corresponding to the target log set.

In yet another aspect, the present application further provides a computer device, which may be a personal computer, a server, or a node in a cloud platform, or the like. Fig. 11 is a schematic diagram showing a composition architecture of a computer device according to the present application. In fig. 11, the computer device 1100 may include a processor 1101 and a memory 1102.

Optionally, the computer device may also include a communication interface 1103, an input unit 1104 and a display 1105 and communication bus 1106.

Wherein the processor 1101, the memory 1102, the communication interface 1103, the input unit 1104 and the display 1105 all perform communication with each other via a communication bus 1106.

In the embodiment of the present application, the processor 1101 may be a central processing unit, an application specific integrated circuit, or the like.

The processor may call a program stored in the memory 1102, and in particular, the processor may perform the operations performed on the cloud computer device side in the above embodiments.

The memory 1102 is used to store one or more programs, and the programs may include program code including computer operation instructions, and in an embodiment of the present application, at least a program for implementing the method for determining an exception log in any of the above embodiments is stored in the memory.

In one possible implementation, the memory 1102 may include a storage program area that may store an operating system, the above-mentioned programs, application programs required for functions such as image playback, and the like, and a storage data area that may store data created according to the use of the computer device.

The communication interface 1103 may be an interface of a communication module.

The present application may also include an input unit 1104, which may include a touch sensing unit, a keyboard, and the like.

The display 1105 includes a display panel such as a touch display panel or the like.

Of course, the computer device structure shown in fig. 11 does not limit the computer device in the embodiment of the present application, and the computer device may include more or less components than those shown in fig. 11, or may combine some components in practical applications.

In another aspect, the present application further provides a storage medium having stored therein computer-executable instructions that, when loaded and executed by a processor, implement a method for determining an exception log as in any of the above embodiments.

It should be noted that, in the present specification, each embodiment is described in a progressive manner, and each embodiment is mainly described as different from other embodiments, and identical and similar parts between the embodiments are referred to each other, and different embodiments may be combined with each other. For the apparatus class embodiments, the description is relatively simple as it is substantially similar to the method embodiments, and reference is made to the description of the method embodiments for relevant points.

Finally, it is further noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions.

The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

The foregoing is merely a preferred embodiment of the present invention and it should be noted that modifications and adaptations to those skilled in the art may be made without departing from the principles of the present invention, which are intended to be comprehended within the scope of the present invention.

Claims

1. A method for determining abnormal logs, characterized by comprising:

Obtaining log data to be analyzed, wherein the log data includes multiple logs;

Clustering multiple logs in the log data according to the function identifier and character features contained in the log to obtain multiple clustered log sets and data composition features corresponding to each log set, wherein the log set includes at least one log, the function identifier in the log is used to characterize the type of functional mode to which the running status information recorded in the log belongs, the data composition features characterize the number and type of data items contained in the log, and the value corresponding to each data item, and the character features include one or more of the number of real numbers contained in the log, the number of characters other than real numbers, and the specific content of the characters other than real numbers;

Determine a set detection mode, where the detection mode is one of a manual detection mode and an automatic detection mode;

In the case where the set detection mode is the automatic detection mode, for each of the log sets, based on the data composition features corresponding to the log set, the functional mode corresponding to the logs in the log set is determined, and at least one anomaly detection rule matching the functional mode is searched according to the correspondence between the data composition features and the anomaly detection rules, wherein the anomaly detection rules include conditions that different types of data items in the log need to satisfy;

For each of the log sets, using at least one anomaly detection rule corresponding to the log set, determine an abnormal log in the log set and abnormal data in the abnormal log;

When the set detection mode is the manual detection mode, a log display interface is displayed, wherein the log display interface displays a plurality of logs in the log data;

Determine the target log set to which the target log belongs;

Display the data of each log in the target log set.

2. The method according to claim 1, further comprising:

In the case where there are abnormal logs in the log set, determining at least one target data item to which the abnormal data in each abnormal log in the log set belongs, wherein the target data item belongs to at least one data item in each log in the log set;

For each target data item corresponding to the log set, construct an abnormal identification curve graph of the target data item, wherein the abnormal identification curve graph includes a value change curve graph corresponding to the value of the target data item in each log of the log set, and information of abnormal data in the target data item is marked in the value change curve graph;

An abnormal identification curve graph of each target data item in the log set is displayed.

3. The method according to claim 1, further comprising:

For each log set, determine a target log sample set that matches the data composition feature corresponding to the log set from multiple log sample sets corresponding to the stored log sample data, wherein the log sample data includes multiple log samples without abnormal data, and the multiple log sample sets are obtained by clustering multiple log samples of the log sample data based on the data composition feature of the data in the log samples;

For each log set, determine a first log occurrence frequency of the log set in the log data and a second log occurrence frequency of a target log sample set corresponding to the log set in the log sample data, and in a case where it is determined based on the first log occurrence frequency and the second log occurrence frequency that the number of logs in the log set has an abnormal increase in number, determine the log set as a risk log set with an abnormal risk, wherein the first log occurrence frequency is a ratio of a first number of logs included in the log set to a first total number of logs included in the log data, and the second log occurrence frequency is a ratio of a second number of logs included in the target log sample set corresponding to the log set to a second total number of logs included in the log sample data;

Output prompt information for the risk log set.

4. The method according to claim 3, characterized in that the outputting of prompt information for the risk log set comprises:

A rule supplement prompt for the risk log set is output, where the rule supplement prompt is used to prompt a user to add an anomaly detection rule for a data composition feature corresponding to the risk log set.

5. The method according to claim 1, characterized in that before displaying the data of each log in the target log set, it also includes:

Determine the respective value change curves of various data items corresponding to the target log set, the data items corresponding to the target log set are the data items contained in the logs of the target log set, and the value change curve of each data item is a value change curve constructed based on the value of the data item in each log of the target log set;

The displaying of data of each log in the target log set includes:

A curve chart display interface is displayed, wherein the curve chart display interface displays value change curves of various data items corresponding to the target log set.

6. A device for determining abnormal logs, characterized by comprising:

A log acquisition unit, used to acquire log data to be analyzed, wherein the log data includes a plurality of logs;

A log clustering unit, configured to cluster multiple logs in the log data according to data composition features of the data in the logs, to obtain multiple clustered log sets and data composition features corresponding to each of the log sets, wherein the log set includes at least one log;

A mode determination unit, used to determine a set detection mode, wherein the detection mode is one of a manual detection mode and an automatic detection mode;

A rule matching unit, configured to determine, for each of the log sets, a functional mode corresponding to the logs in the log set based on the data composition features corresponding to the log set, and search for at least one anomaly detection rule matching the functional mode according to the correspondence between the data composition features and the anomaly detection rules, wherein the anomaly detection rules include conditions that different types of data items in the log need to satisfy;

an abnormality determination unit, configured to determine, for each of the log sets, an abnormal log having an abnormality in the log set and abnormal data in the abnormal log by using at least one abnormality detection rule corresponding to the log set;

The log clustering unit comprises:

The log clustering subunit is used to cluster multiple logs in the log data according to the function identifier and character features contained in the log, wherein the function identifier in the log is used to characterize the type of the function mode to which the running status information recorded in the log belongs, the data composition feature of the log characterizes the number and type of data items contained in the log, and the value corresponding to each data item, and the character feature includes one or more of the number of real numbers contained in the log, the number of characters other than real numbers, and the specific content of characters other than real numbers;

A log display unit, used for displaying a log display interface when the set detection mode is a manual detection mode, wherein the log display interface displays a plurality of logs in the log data;

A log selection unit, configured to determine a target log selected by the log selection operation in response to the log presentation interface detecting the log selection operation;

A set determining unit, used to determine the target log set to which the target log belongs;

The data display unit is used to display the data of each log in the target log set.

7. A computer device, characterized in that the computer device comprises: a processor and a memory;

The processor is used to call and execute the program stored in the memory;

The memory is used to store the program, and the program is at least used to implement the method for determining an abnormal log as described in any one of claims 1 to 5 above.

8. A storage medium, characterized in that the storage medium stores computer executable instructions, and when the computer executable instructions are loaded and executed by a processor, the method for determining an abnormal log as described in any one of claims 1 to 5 above is implemented.