CN101453378A - Method and system for log damp and audit - Google Patents
Method and system for log damp and audit Download PDFInfo
- Publication number
- CN101453378A CN101453378A CNA2008102475508A CN200810247550A CN101453378A CN 101453378 A CN101453378 A CN 101453378A CN A2008102475508 A CNA2008102475508 A CN A2008102475508A CN 200810247550 A CN200810247550 A CN 200810247550A CN 101453378 A CN101453378 A CN 101453378A
- Authority
- CN
- China
- Prior art keywords
- log
- file
- dumped
- type
- dumping
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 43
- 238000012550 audit Methods 0.000 title claims description 65
- 238000013507 mapping Methods 0.000 claims description 20
- 238000007906 compression Methods 0.000 claims description 15
- 230000006835 compression Effects 0.000 claims description 14
- 238000012545 processing Methods 0.000 claims description 5
- 230000001960 triggered effect Effects 0.000 claims description 4
- 238000012856 packing Methods 0.000 claims description 3
- 238000012163 sequencing technique Methods 0.000 claims description 3
- 238000012217 deletion Methods 0.000 claims description 2
- 230000037430 deletion Effects 0.000 claims description 2
- 230000008569 process Effects 0.000 description 9
- 238000010586 diagram Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 7
- 230000007547 defect Effects 0.000 description 6
- 230000008676 import Effects 0.000 description 6
- 230000006978 adaptation Effects 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 230000003044 adaptive effect Effects 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 201000009032 substance abuse Diseases 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Landscapes
- Debugging And Monitoring (AREA)
Abstract
The invention discloses a method and a system for dumping and auditing logs with strong universality and expansibility. The method comprises: establishing a profile, recording names of log types to be dumped, field to be dumped corresponding to each log type and display name of each field to be dumped in the profile; during dumping the logs, according to record of the profile, looking up a log list matched with the log type to be dumped, and storing the field to be dumped in the found log list into a log file; and during auditing the logs, looking up the log file matched with auditing conditions, and parsing and displaying the found log file according to the record of the profile.
Description
Technical Field
The invention relates to a network log processing technology, in particular to a method and a system for dumping and auditing logs.
Background
At present, enterprises usually provide network connection for employees, and in order to ensure that the enterprises reasonably utilize Internet resources and prevent enterprise information from leaking, Internet surfing behaviors of the employees are usually recorded in a network Log (Log). By auditing the network log, the employee can know whether the employee abuses the enterprise network resources, what the employee uses the network to do, and the access condition of the employee to the network and the public server.
The weblog is stored in a log database. The capacity of the log database is limited, but the weblogs are massive, so that the weblogs need to be periodically dumped to protect internet records. The log dump is to export the log from the log database and store the log in a log file. Existing dump operations typically rely on the export functionality of the log database.
After the log is dumped, an auditing method for the dump log needs to be provided, so that the network manager can check the content of the network log. Wherein, the log audit is to display the content of the log file. The existing auditing operation has two modes: firstly, importing a dumped log file into a log database through an importing function of the log database, and displaying a log through an original software interface of the log database; secondly, a special file auditing tool is adopted to directly analyze and display the content of the log file.
However, the existing log dump and audit scheme has the following defects:
first, different types of logs are stored in different log tables, and if it is desired to export A, B, C three types of logs, an export function needs to be set for the three types of logs. The export setting function is realized through system programming, when the programming is finished, the programming code needs to be recompiled, and after the compilation is successful, the log database supports A, B, C export of three types of logs. When a new log type is needed, a corresponding export function must be added for the newly added log type, and then compiling is performed again, and after compiling is successful, the log database can support export of the new log type. Similarly, when the exported log file is audited by using the log database, an import function needs to be set for each type of log. If a special file auditing tool is used for auditing log files, an import function also needs to be set for each type of log, and the field names in the log database also need to be analyzed to be display names, for example, the field name 'start _ time' in the log database is analyzed to be 'start time', then each field is displayed by the display name, otherwise, auditors cannot understand the displayed log files.
It can be seen that no matter the log database is adopted to realize dump and audit or the file audit tool is adopted to realize audit, if a new log type is added, the functions of the export part and the audit part must be developed aiming at the newly added log type to adapt to the new log type, so that the universality and expansibility of the existing log dump and audit scheme are not ideal.
Secondly, each type of log is usually dumped into a log file, so that the log file is huge. If the log database or the file auditing tool is adopted to directly import the large files into the system, the operation speed is low, the import time is long, the system resources are greatly consumed, and the error probability is high during the import.
Disclosure of Invention
In view of the above, the present invention provides a log dumping and auditing method, which can solve the problem of log adaptation of dumping and auditing, and solve the defect of poor generality and expansibility of the log dumping and auditing scheme in the prior art.
The method comprises the following steps:
establishing a configuration file, and recording the name of the log type to be dumped, the field to be dumped corresponding to each log type and the display name of each field to be dumped in the configuration file;
when dumping the log, according to the type and the field of the log to be dumped recorded by the configuration file, searching a log table matched with the type of the log to be dumped, and storing the field to be dumped in the searched log table into a log file;
and when the log is audited, searching the log file matched with the auditing conditions, and analyzing and displaying the searched log file according to the display name recorded by the configuration file.
Preferably, the method further comprises: when the log types are added, adding the names of the newly added log types, the corresponding fields to be dumped and the display names of the fields to be dumped into the configuration file;
and when the log type is deleted, deleting the name of the log type to be deleted, the corresponding field to be dumped and the display name of the field to be dumped from the configuration file.
Preferably, the method further comprises: and registering the configuration file, and performing dump and audit operations by adopting the registered configuration file content.
The log table is an hour log table, and each hour log table stores logs in a preset time length;
the searching the log table matched with the log type to be dumped according to the log type to be dumped and the field to be dumped recorded by the configuration file comprises: processing the log types to be dumped in the configuration file one by one; and when the current to-be-dumped log type is processed, searching the hourly log table matched with the current to-be-dumped log type.
Wherein, the searching of the hour log table matched with the current to-be-dumped log type is as follows: setting a time range of the log to be dumped, and searching an hour log table matched with the type of the current log to be dumped in the hour log table in the time range; or looking up an hour log table matched with the type of the current log to be dumped in the hour log table of the previous day.
Preferably, the capacity limit of the log file is preset;
the step of saving the found fields to be dumped in the log table into the log file comprises the following steps:
according to the capacity limit of the log file, the log in a log table is dumped into one or more log files, and a first association relation between the name of the log file and the log type and the log time of the log stored in the log file is established.
Wherein the capacity limit of the preset log file comprises: setting the dump time length and/or the maximum log number of the log file;
when the logs in the log table are dumped into the log files according to the dumping time length, each log file allows to store at most one log in a preset dumping time length;
when the logs in the log table are dumped into the log files according to the maximum number of the log pieces, the number of the log pieces allowed to be stored in each log file is less than or equal to the maximum number of the log pieces;
when the logs in the log table are dumped into the log files according to the dumping time length and the maximum number of the logs, each log file allows to store the logs in a preset dumping time length at most, and when the number of the logs in the preset dumping time length is larger than the maximum log capacity, the logs in the dumping time length are dumped into a plurality of log files according to the maximum log capacity.
Preferably, the establishing a first association between the name of the log file and the log type and the log time of the log stored in the log file is as follows:
setting the name of a log file to comprise the log type and the time range of the log stored in the log file, and simultaneously recording the mapping relation between the name of the log file and the starting time of the first log in the log file.
When the log is audited, the log file matched with the auditing condition is searched, and the searched log file is analyzed and displayed according to the display name recorded by the configuration file, and the method comprises the following steps:
c1, according to the first incidence relation, searching a log file matched with the type of the log to be audited and the time range to be audited in the auditing condition;
c2, obtaining logs from the searched log files;
c3, obtaining a display name corresponding to the type of the log to be audited, dynamically generating an auditing frame containing the obtained display name, and displaying the log obtained in the step c2 in the generated auditing frame.
Preferably, the audit condition further comprises a maximum number of query pieces;
the step c2 includes: acquiring logs from searched log files according to the maximum number of inquired logs in the auditing conditions, and stopping log acquisition operation and recording the position of the last log to be acquired when the number of the acquired logs reaches the maximum number of inquired logs;
after the step c3, the method further comprises: when a continue audit command is received, returning to execute the step c2, and starting acquisition from the position of the last acquired log when executing step c 2.
Preferably, the step of saving the found to-be-dumped field in the log table into the log file further includes:
after the logs in the log table are stored in the log file, the log file in a unit time is packaged into a compressed packet by taking a preset time length as a unit;
for each compressed packet, acquiring a display name corresponding to the log type of the log stored in the compressed packet, outputting the acquired display name and a first association relation corresponding to each log file in the compressed packet to an index file, and adding the index file into the compressed packet; establishing a second association relation between the name of the compressed packet and the log type and the log time of the log stored in the compressed packet;
the step c1 includes:
searching a compressed packet matched with the type of the log to be audited and the time range to be audited in the auditing condition according to the second incidence relation;
and searching a log file matched with the time range to be audited in the auditing condition in the compressed packet according to the first incidence relation recorded by the index file in the compressed packet.
After the step c1 and before the step c2, the method further comprises: and sequencing the searched log files according to time.
Preferably, the operation of dumping the log is triggered periodically.
The invention also provides a log dumping and auditing system, which can solve the problem of log adaptation of dumping and auditing and solve the defects of poor universality and expansibility of log dumping and auditing schemes in the prior art.
The system comprises a configuration file storage unit, a dump unit, an audit unit and a log file storage unit;
the configuration file storage unit is used for reading and storing configuration files from the outside; the configuration file records the name of the log type to be dumped, the field to be dumped corresponding to each log type and the display name of each field to be dumped;
the dumping unit is used for reading the configuration file from the configuration file storage unit when dumping the log, searching a log table matched with the type of the log to be dumped according to the type and the field of the log to be dumped recorded by the read configuration file, storing the field to be dumped in the searched log table into the log file, and storing the log file into the log file storage unit;
the auditing unit is used for searching the log file matched with the auditing conditions in the log file storage unit when the log is audited, and analyzing and displaying the searched log file according to the display name recorded by the configuration file;
the log file storage unit is used for storing log files.
Preferably, the configuration file storage unit is further configured to, when receiving a registration command of a newly added log type, read and store a name of the externally added log type, a corresponding field to be dumped, and a display name of the field to be dumped;
when a deletion command of an existing log type is received, the name of the log type to be deleted, the corresponding field to be dumped and the display name of the field to be dumped are deleted from the unit.
The dump unit comprises a log table lookup module, a log dump module and an association module;
the log table searching module is used for reading the configuration file from the configuration file storage unit when the log is dumped, and searching the log table matched with the type of the log to be dumped according to the type and the field of the log to be dumped recorded by the read configuration file;
the log dumping module is used for dumping the log table searched by the log table searching module; during dumping, according to the capacity limit of the log files, dumping the logs in a log table into one or more log files, and storing the log files obtained by dumping into the log file storage unit;
the association module is used for establishing a first association relationship between the name of the log file obtained by dumping and the log type and the log time of the log stored in the log file after the log dumping module dumps the log into the log file, and storing the first association relationship into the log file storage unit.
Wherein, the capacity limit of the log file is realized by setting the dump time length and/or the maximum number of the log files;
the log dumping module is further used for storing logs in a preset dumping time length at most in each log file when dumping is carried out according to the dumping time length; when dumping is carried out according to the maximum number of the log files, the number of the log files allowed to be stored is smaller than or equal to the maximum number of the log files; when dumping is carried out according to the dumping time length and the maximum number of the logs, each log file allows to store at most one log in a preset dumping time length, and when the number of the logs in the preset dumping time length is larger than the maximum log capacity, the logs in the dumping time length are dumped into a plurality of log files according to the maximum log capacity.
The correlation module names the log file obtained by dumping, the name of the log file comprises the log type and the time range of the log stored in the log file, and meanwhile, the mapping relation between the name of the log file and the starting time of the first log in the log file is recorded.
The auditing unit comprises a log searching module and an auditing display module;
the log searching module is used for searching log files matched with the types of the logs to be audited and the time range to be audited in the auditing conditions in the log file storage unit according to the first incidence relation, and acquiring logs from the searched log files;
the audit display module is used for acquiring a display name corresponding to the type of the log to be audited, dynamically generating an audit frame containing the acquired display name, and displaying the log acquired by the log searching module in the generated audit frame.
Preferably, the dump unit further comprises a compression module, and the compression module is connected with the log dump module, the association module and the configuration file storage unit;
the compression module is used for receiving the log files dumped by the log dump module and packing the log files in unit time into compression packets by taking the preset time length as a unit; for each compressed packet, acquiring a first association relation corresponding to each log file in the compressed packet from the association module, acquiring a display name corresponding to the log type of the log stored in the compressed packet from the configuration file storage unit, outputting the acquired first association relation and the display name to an index file, and adding the index file to the compressed packet; establishing a second association relation between the name of the compressed packet and the log type and the log time of the log stored in the compressed packet;
the log searching module is further used for searching a compressed packet matched with the type of the log to be audited and the time range to be audited in the auditing condition in the log file storage unit according to the second incidence relation; and searching a log file matched with the time range to be audited in the auditing condition in the compressed packet according to the first incidence relation recorded by the index file in the compressed packet.
According to the technical scheme, the defects of poor universality and expansibility in the prior art can be overcome by applying the method and the device. Specifically, the following beneficial effects are achieved:
firstly, the invention adopts the configuration file to record the name of the log type to be dumped, the field to be dumped corresponding to each log type and the display name of each field to be dumped, and the system only needs to read the configuration file and carry out the log dumping operation according to the configuration file. When a new log type needs to be added, dumping and auditing of the new log can be realized only by incorporating the new log type into the configuration file. Therefore, the invention completely decouples the log dump and audit and the log type, does not need to adapt and develop a log database or a special file audit tool like the prior art, and solves the defects of poor universality and expansibility of the log dump and audit scheme in the prior art.
Secondly, when the log is dumped into the log file, the log in one log table is dumped into one or more log files according to the capacity limit of the log file, thereby avoiding the generation of overlarge log files and being convenient for reading the log file during log audit.
In addition, the name of each log file is associated with the log type and the log time of the log file, for example, a mapping relation is established, so that when the log files matched with the auditing conditions are searched from numerous log files, the log files can be quickly and pertinently searched according to the association relation, the searching time is shortened, the searching efficiency is improved, and the auditing efficiency is improved.
In addition, when the log file is stored, the log file is compressed and stored, so that the storage space is saved, and the disorder of the log file is avoided. And in the compression process, the display names corresponding to the association relation and the log types of the compressed log files are also stored in the compressed packets, so that the auditing unit can generate an auditing interface at any place as long as the compressed packets exist, association with the existing system is not needed, and the flexibility of log auditing is greatly improved.
Drawings
FIG. 1 is a flow chart illustrating the process of a log dump according to an embodiment of the present invention.
Fig. 2 is a flowchart of log audit processing according to an embodiment of the present invention.
FIG. 3 is a schematic diagram of an audit interface in an embodiment of the invention.
FIG. 4 is a schematic diagram of a log dump and audit system according to an embodiment of the present invention.
Fig. 5 is a schematic structural diagram of the transfer unit in fig. 4.
Fig. 6 is a schematic structural diagram of the auditing unit in fig. 4.
Detailed Description
The invention relates to a scheme for dumping and auditing logs, which has the basic ideas that: establishing a configuration file, and recording the name of the log type to be dumped, the field to be dumped corresponding to each log type and the display name of each field to be dumped in the configuration file; the field to be dumped is a field name used in the log database, such as start _ time, and the display name is a field name displayed in the audit interface, such as "start time" corresponding to the start _ time.
When the log is dumped, according to the type and the field of the log to be dumped recorded by the configuration file, a log table matched with the type of the log to be dumped is searched, and the field to be dumped in the searched log table is stored in a log file.
And when the log is audited, searching the log file matched with the auditing conditions, and analyzing and displaying the searched log file according to the display name recorded by the configuration file. Wherein the parsing operation is to parse a dump field in the log file into a display name.
As can be seen, the present invention completely decouples log dumps and audits from log types. When a new log type needs to be added, dumping and auditing of the new log type can be realized only by incorporating the new log type into the configuration file, adaptive development on a log database or a special file auditing tool is not needed, and the defect of low universality and expansibility in the prior art is overcome.
The invention is described in detail below by way of example with reference to the accompanying drawings.
First, a configuration file for recording the type of the log to be dumped, the field to be dumped and the display name is described in detail. In this embodiment, the configuration file is referred to as a log type specification table, which is referred to as a specification table for short.
The specification table introduced by the invention records the related information of the log dump and the audit in a universal format, including the log type, the storage field name of the log field in the log database, the display name of the log field in the audit interface and the like. Wherein, the storage field names and the display names are in one-to-one correspondence.
The log type specification table format is defined as follows:
[LogType1]
[LogTypeName]
[DbFieldNames]
[FieldName1]
[FieldName2]
……
[FieldNameN]
[GuiFieldNames]
[FieldName1]
[FieldName2]
……
[FieldNameN]
……
[LogTypeN]
……
wherein, LogType1 represents the first log type, and [ LogTypeN ] represents the Nth log type; the LogTypeName represents the name of the log type of each log type, and the name of the log type is the unique identifier of each log type; DbFieldNames represents a field to be dumped, the field name to be dumped is a storage field name of a log field in a database, GuiFieldNames represents a display name of the field to be dumped, the display name is a display name of the log field in an audit interface, and the display name is an interface element name finally displayed by the log audit interface; fieldname represents the specific contents of the storage field name and the display name.
When a new log type is added, the name of the new log type, the corresponding field to be dumped and the display name of the field to be dumped are added into the standard table according to the format of the standard table. For example, when adding the log of the network traffic NetStreamV5, it is only necessary to determine the fields to be dumped and the corresponding display names under the NetStreamV5 log, and then configure the fields according to the specification table.
Assume that the NetStreamV5 log has log fields start _ time, end _ time, src _ ip, dest _ ip, src _ port, dest _ port, prot, tos, app _ name, and dev _ ip in the log database. Where the field tos has no meaning for auditing and does not need to be dumped, it is discarded when the specification table is configured. Only fields of start _ time, end _ time, src _ ip, dest _ ip, src _ port, dest _ port, prot, app _ name, and dev _ ip may be configured as fields to be dumped. Meanwhile, the field names of the database cannot be directly displayed by the auditing result, but the actual meanings of the field names should be displayed, so that the display names corresponding to the fields to be dumped are configured in the content of the specification table, and the display names corresponding to the fields to be dumped are respectively: start time, end time, source IP, destination IP, source port, destination port, protocol, application, and device IP.
According to the format of the specification table, the log is added with the following configuration for NetStreamV 5:
[LogType]
[NetStreamV5]
[DbFieldNames]
[start_time]
[end_time]
[src_ip]
… …
[dev_ip]
[GuiFieldNames]
[ Start time ]
[ end time ]
[ Source IP ]
… …
[ Equipment IP ]
By adopting the configuration specification table, a network administrator only needs to configure interested fields, but not all fields in the log database, so that the flexibility of dumping and auditing is improved. In practice, the specific configuration of the specification table can be in various forms, such as using an XML configuration file as a carrier of the specification table, and the following shows an example of the specification table being presented in the form of an XML configuration file:
<LogType>
<LogTypeName>NetStreamV5</LogTypeName>
<DbFieldNames>
<Item>start_time</Item>
<Item>end_time</Item>
<Item>src_ip</Item>
…
<Item>dev_ip</Item>
</DbFieldNames>
<GuiFieldNames>
< Item > Start time </Item >
< Item > end time </Item >
< Item > Source IP </Item >
…
< Item > device IP </Item >
</GuiFieldNames>
</LogType>
After the configuration of the specification table is completed, the specification table needs to be registered in a system for executing the log dump and audit scheme of the invention, which is referred to as a log dump and audit system, so that the log dump and audit system executes dump and audit operations according to the content of the registered configuration file. The registration process is to import and store the contents in the specification table into the system. When the specification table is registered, the configuration content of the specification table must be in accordance with a preset format, otherwise, the registration process fails.
When a registered log type is logged out, the name of the log type to be deleted, namely LogTypeName, is used as an index, the name of the log type to be deleted is searched in a registered specification table, and the name of the log type to be deleted, the corresponding field to be dumped and the display name of the field to be dumped are deleted.
After the specification table is registered to the system, the system may perform dump and audit operations according to the registered specification table. Dumping and auditing are described separately below.
FIG. 1 is a flow chart illustrating the process of a log dump according to an embodiment of the present invention. As shown in fig. 1, the process includes the following steps:
step 101: when the dump operation is triggered, the registered specification table is read, and the log types in the specification table are processed one by one.
Step 102: a configuration of an unprocessed log type is read from the specification table. The read content comprises information such as a log type name LogTypeName, a field name DbFieldNames to be dumped, a display name GuiFieldNames and the like.
The dump operation may be triggered by a timed task or manually.
Step 103: and searching a log table matched with the log type name LogTypeName in a log database according to the read log type name LogTypeName. The name of the log table is usually bound with the log type, and the log type in the log table can be directly known from the name of the log table.
Usually, the log dump is performed once a day, and the logs of the previous day are dumped, so when the matched log table is searched, the log table matched with the read log type name on the previous day is searched; in practice, a time range in which the log to be dumped is located may also be preset, and in the hour log table in the time range, a log table matching the read log type name is searched.
In this embodiment, when the log table is generated, to avoid forming an oversized log table, the log is stored by hours to generate an hour log table, and meanwhile, the name of the hour log table is bound with the log type and the log time, and the log type and the log time of the stored log are reflected on the name of the hour log table. For example, the name of the hour log table is netstreamv 5-08092218, which indicates that the hour log table stores logs within 2008-09-2218:00: 00-18: 59: 59.
In this step, suppose that 2008-09-23 morning dumps the log of the previous day, and first processes the log type NetStreamV5, and searches the hour log table with the name matched with NetStreamV5 in the log database according to the log type name NetStreamV5, and then searches the 24-hour log table.
Step 104: and acquiring the field name to be dumped corresponding to the read log type name, and saving the field content matched with the field name to be dumped in the log table into a log file aiming at each log table searched in the step 103. In order to distinguish different log files dumped from the same log table, the name of the dumped log file is associated with the log type and the log time of the log stored in the log file.
Assuming that the NetStreamV5 log is stored in this step, only the fields of start _ time, end _ time, src _ ip, dest _ ip, src _ port, dest _ port, prot, app _ name, and dev _ ip need to be dumped according to the specification table.
Step 105: judging whether all log types in the specification table are processed or not; if yes, ending the process; otherwise, return to execute step 102.
This flow ends by this point.
In general, weblogs are massive, often reaching tens of millions of logs in 1 hour. In order to avoid that an oversized log file is formed by dumping, which affects the dumping performance and the query performance during auditing, in this embodiment, when the log is dumped to the log file in step 104, the log in one log table is dumped to a plurality of log files according to the capacity limit of the log file, so as to avoid the formation of the oversized log file. Wherein, the capacity limit of the log file can be realized by setting the dump time length and/or the maximum number of the log files. In particular, the present invention relates to a method for producing,
when dumping is carried out according to the dumping time length, each log file allows the storage of logs within a preset dumping time length at most. For example, if the dump time length is set to 10 minutes, one log file holds logs within 10 minutes at most. The value of the dump time length is predetermined according to the network traffic.
When dumping is carried out according to the maximum log number, the number of log pieces allowed to be stored in each log file is less than or equal to the maximum log number. For example, if the maximum number of log files is set to 50 ten thousand, a log file can accommodate 50 ten thousand logs at the maximum.
When dumping is carried out according to the dumping time length and the maximum log number, each log file allows to store logs in a preset dumping time length at most, and when the number of logs in the preset dumping time length is larger than the maximum log capacity, the number of required log files is determined according to the number of logs in the dumping time length and the maximum log capacity, and the logs in the dumping time length are dumped into the plurality of determined log files. More specifically, this can be achieved by: firstly, determining a current time period to be dumped, such as 2008-09-2218: 10: 00-2008-09-2218: 19:59, in a log table (netstreamv 5-08092218) corresponding to the current time period to be dumped, inquiring the total number of logs in the current time period to be dumped according to a start time field, determining the number of required log files according to the inquired total number of logs and the maximum log capacity, and dumping the searched logs into each log file according to a time sequence.
For example, the dump time length is set to be 10 minutes, the maximum log capacity is set to be 50 ten thousand, and the logs in the time period of 2008-09-2218: 10: 00-2008-09-2218: 19:59 are dumped currently. Then, at this time, in the log table named netstreamv 5-08092218, logs in the time period of 2008-09-2218: 10: 00-2008-09-2218: 19:59 need to be searched according to the start time (start time) field, and if 70 ten thousand matching logs are found, more than 50 ten thousand matching logs are supposed to be found, so that it is determined that 2 log files are needed to store the logs, then the first 50 ten thousand logs are exported to the first log file according to the time sequence by adopting an export tool provided by the database, and the last 20 ten thousand logs are exported to the second log file. Both log files correspond to the time period 2008-09-2218: 10: 00-2008-09-2218: 19: 59.
After the log is exported to the log file, the log file name is associated with the log type and the log time. The association means is not limited. For example, the log type and the log time are embodied in the log file name, or a mapping relation between the log file name and the log type and the log time is established and recorded; or the log type is embodied in the log file name, and a mapping relation between the log file name and the log time is established. The log time may be a time range of each log in the log file, or may be a start time field content of a first log in the log file.
Or, the name of the log file is made to include the log type and the time range of the log stored in the log file, and the mapping relationship between the name of the log file and the start time of the first log in the log file is recorded. This association is illustrated by way of example below.
Suppose that 70 ten thousand NetstreamV5 logs in the time period of 2008-09-2218: 10: 00-2008-09-2218: 19:59 are saved into two log files, and the two log files are named NetstreamV5_0809221810_1 and NetstreamV5_0809221810_ 2. NetStreamV 5-0809221810-1 shows that the log type of the log stored in the file is NetStreavv 5, the log time is within 2008-09-2218: 10: 00-18: 19:59, and the log time is the first file corresponding to the time period of 2008-09-2218: 10: 00-18: 19: 59. And simultaneously establishing a mapping relation between the name of the log File and the starting Time of the first log in the log File, and recording the mapping relation in a mapping table Time-File. Suppose the start time fields of the first record in the two log files netstreamv5_0809221810_1 and netstreamv5_0809221810_2 are: 2008-09-2218: 10:00, 2008-09-2218: 17:53, then the mapping table Time-File record is formed as follows:
netstreamv5_0809221810_1：2008-09-22 18:10:00
netstreamv5_0809221810_2：2008-09-22 18:17:53
in practice, another association manner described above may also be adopted to name two log files corresponding to the same time period, and the start time of the first log in the log files is used as a part of the file name during the naming. Such as netstreamv5_080922181000, netstreamv5_ 080922181753.
Therefore, when the log is dumped into the log file, the generation of an overlarge log file can be avoided by limiting the capacity of the log file, and the log file can be conveniently read during log audit. The name of each log file is associated with the log type and the log time of the log file, for example, a mapping relation is established, so that when log files matched with the auditing conditions are searched from numerous log files in the follow-up process, the log files can be searched quickly and pertinently according to the association relation, the searching time is shortened, the searching efficiency is improved, and the auditing efficiency can be improved.
After the log table is dumped into the log file, the log file is scattered and occupies a huge space, so that the management and the search are inconvenient. For convenience of management, the present embodiment packs the log files in a unit time into a compressed packet and stores the compressed packet in a unit of a preset time length (e.g., 1 hour). And for each compressed packet, outputting the mapping relation in the packing time range in the mapping table to an Index file Index, and adding the Index file Index into the compressed packet. Meanwhile, in order to provide a display name for the audit interface, a display name corresponding to the log type of the log stored in the compressed package needs to be acquired, and the acquired display name is also output to the index file and is input into the compressed package. The name of the compressed packet is associated with the log type and log time of the stored log. For example, a compressed package named NetStreamV5_08092218.zip, indicates that the compressed package stores NetStreamV5 logs and index files index within a 2008-09-2218:00: 00-18: 59:59 time range.
The log audit provided by the invention adopts a file audit mode, provides a general log file audit frame, and is decoupled from a specific log type. FIG. 2 is a flow chart of log auditing in an embodiment of the present invention. As shown in fig. 2, the process includes the following steps:
step 201: and setting auditing conditions, wherein the auditing conditions can comprise the type of the log to be audited and the scope of the time to be audited, and can also comprise fields needing to be audited and the maximum number of inquired pieces, and the like.
Step 202: and searching a log file matched with the type of the log to be audited and the time range to be audited in the auditing condition. In order to display the logs in a time sequence and facilitate reading of an administrator, after the log files are obtained, the log files are sorted according to time information contained in the log file names.
If the log file is compressed and stored during dumping, the method specifically comprises the following steps:
and searching the compressed packets matched with the pending log type and the pending time range according to the name of the compressed packet, wherein the name comprises the log type and the log time, and sequencing the obtained compressed packets according to the time.
Each compressed packet is processed one by one. And when one of the compression packets is processed, opening the index file index in the compression packet, acquiring the mapping relation in the index file, and acquiring the log file with the log time within the pending time range according to the mapping relation. Until all compressed packets are processed.
For example, the audit conditions are: the type of the log to be audited is NetStreamV5, the range of the time to be audited is 2008-09-2218: 18-19: 57, and at most 5000 queries are carried out.
Then, in this step, it is first determined that the required compression package is netstreamv5_08092218.zip and netstreamv5_08092219.zip according to the type of the pending log and the time range. Then, opening an index file in a first compression package netstreamv 5-08092218. zip, acquiring a file name-time mapping relation in the index file, and acquiring a matched log file as netstreamv 5-0809221810-2 according to the mapping relation; then, the index file in the second compressed package netstreamv5_08092219.zip is opened, and the matched log file is obtained as netstreamv5_0809221820_1 according to the mapping relation in the index file. Preferably, the acquired log files are also sorted by log time at this time.
Step 203: and acquiring logs from the searched log files.
If the audit condition sets the maximum number of query pieces, the step acquires logs from all searched log files according to the maximum number of query pieces, and stops the acquisition operation and records the position of the last acquired log when the number of the acquired logs reaches the maximum number of query pieces. In this case, step 204 will only display a portion of the log, the number of which is the maximum number of query pieces.
When a continuous audit command is subsequently received, the log is continuously acquired from the position of the last acquired log of the record.
Step 204: acquiring a display name corresponding to the log type to be audited, and dynamically generating an auditing frame containing the acquired display name; and resolving the storage field in the log obtained in the step 203 into a corresponding display name according to the obtained display name, and then displaying the corresponding display name in the generated audit frame. A dynamically generated audit framework is shown in fig. 3.
The audit frame is universal, but the display content in the audit frame is generated dynamically according to the content of the log file. In the embodiment, the display name corresponding to the type of the log to be audited is obtained from the compressed packet, so that the audit interface can be generated at any place as long as the compressed packet exists, association with the existing system is not needed, and the flexibility of log audit is greatly improved. In practice, the display name corresponding to the pending log type may be obtained from the specification table.
A system for performing the log dump and audit method of the present invention is described below. FIG. 4 is a schematic diagram of a log dump and audit system according to an embodiment of the present invention. As shown in fig. 4, the system includes a configuration file storage unit 41, a dump unit 42, an audit unit 43, and a log file storage unit 44. Wherein,
and a configuration file storage unit 41 for reading and storing the specification table from the outside. As mentioned above, the specification table records the name of the log type to be dumped, the field to be dumped corresponding to each log type, and the display name of each field to be dumped.
The configuration file storage unit 41 further reads and stores the name of the externally added log type, the corresponding field to be dumped and the display name of the field to be dumped when receiving the registration command of the new log type; when a deleting command of the existing log type is received, the name of the log type to be deleted is used as an index, the name of the log type to be deleted is searched in the unit, and the name of the log type to be deleted, the corresponding name of the field to be dumped and the display name of the field to be dumped are deleted.
And a dumping unit 42, configured to, when dumping the log, read the specification table from the configuration file storage unit 41, search, according to the log type to be dumped and the field to be dumped recorded in the read specification table, a log table matching the log type to be dumped from a log database connected to the system, store the field to be dumped in the searched log table in a log file, and store the log file in the log file storage unit 44.
And the auditing unit 43 is configured to, when auditing the log, search for a log file matching the auditing conditions in the log file storage unit 44, and analyze and display the found log file according to the display name recorded by the specification table.
A log file storage unit 44 for storing a log file.
Fig. 5 is a schematic structural diagram of the memory cell 42 in fig. 4. As shown in fig. 5, the dump unit 42 specifically includes a log table lookup module 421, a log dump module 422, and an association module 423. Wherein,
the log table lookup module 421 is configured to, when dumping the log, read the specification table from the configuration file storage unit 41, and lookup a log table matching the log type to be dumped from the log database according to the log type to be dumped and the field to be dumped recorded in the read specification table. When the log table is searched for a certain log type, the hour log table matched with the name of the current log type in the time range can be searched for according to the set time range, or the hour log table matched with the name of the current log type in the previous day can be searched for. The trigger for the dump log operation may be automatic or manual at regular intervals.
A log dumping module 422, configured to perform dumping processing on the log table found by the log table searching module 421; during dumping, according to the preset capacity limit of the log file, the log in one log table is dumped into one or more log files, the dumped log file is stored in the log file storage unit 44, and the association module 423 is notified. The log dump module 422 limits the capacity of the log file in the same manner as the log file capacity in the previous embodiment of the method, which is not described herein again.
The association module 423 is configured to, when receiving the notification, establish a first association relationship between the name of the log file obtained by dumping and the log type and the log time of the log stored in the log file, and store the first association relationship in the log file storage unit 44. The association operation performed by the association module 423 is the same as the association method described in the foregoing method embodiment, and is not described herein again.
Fig. 6 is a schematic structural diagram of the auditing unit 43 in fig. 4. As shown in fig. 6, the auditing unit 43 specifically includes a log lookup module 431 and an audit display module 432. Wherein,
a log searching module 431, configured to search, in the log file storage unit 44, a log file that matches the pending log type and the pending time range in the auditing condition according to the first association relationship; preferably, the matched log files can be sorted by log time; then, the log is obtained from the found log files. And when the logs are acquired, if the number of the acquired logs reaches the maximum number of the inquired logs in the auditing condition, stopping the acquisition operation and recording the current position. And subsequently, if an audit continuation command is received, acquiring the log from the current recorded position.
And the audit display module 432 is configured to dynamically generate an audit frame including a display name according to the display name corresponding to the type of the log to be audited, and display the log acquired by the log search module 431 in the generated audit frame.
In this case, the log searching module 431 is further configured to search, according to the second association relationship, a compressed packet matching the pending log type and the pending time range in the audit condition in the log file storage unit 44; and searching a log file matched with the time range to be audited in the auditing condition in the compressed packet according to the first incidence relation recorded by the index file in the compressed packet.
In summary, the above description is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (20)
1. A method for log dumping and auditing, the method comprising:
establishing a configuration file, and recording the name of the log type to be dumped, the field to be dumped corresponding to each log type and the display name of each field to be dumped in the configuration file;
when dumping the log, according to the type and the field of the log to be dumped recorded by the configuration file, searching a log table matched with the type of the log to be dumped, and storing the field to be dumped in the searched log table into a log file;
and when the log is audited, searching the log file matched with the auditing conditions, and analyzing and displaying the searched log file according to the display name recorded by the configuration file.
2. The method of claim 1, further comprising: when the log types are added, adding the names of the newly added log types, the corresponding fields to be dumped and the display names of the fields to be dumped into the configuration file;
and when the log type is deleted, deleting the name of the log type to be deleted, the corresponding field to be dumped and the display name of the field to be dumped from the configuration file.
3. The method of claim 1, further comprising: and registering the configuration file, and performing dump and audit operations by adopting the registered configuration file content.
4. The method of claim 1, wherein the log table is an hour log table, each hour log table storing logs for a preset length of time;
the searching the log table matched with the log type to be dumped according to the log type to be dumped and the field to be dumped recorded by the configuration file comprises: processing the log types to be dumped in the configuration file one by one; and when the current to-be-dumped log type is processed, searching the hourly log table matched with the current to-be-dumped log type.
5. The method of claim 4, wherein the looking up the hourly log tables matching the current pending log type is: setting a time range of the log to be dumped, and searching an hour log table matched with the type of the current log to be dumped in the hour log table in the time range; or looking up an hour log table matched with the type of the current log to be dumped in the hour log table of the previous day.
6. The method of claim 1, wherein a capacity limit of the log file is preset;
the step of saving the found fields to be dumped in the log table into the log file comprises the following steps:
according to the capacity limit of the log file, the log in a log table is dumped into one or more log files, and a first association relation between the name of the log file and the log type and the log time of the log stored in the log file is established.
7. The method of claim 6, wherein presetting the capacity limit of the log file comprises: setting the dump time length and/or the maximum log number of the log file;
when the logs in the log table are dumped into the log files according to the dumping time length, each log file allows to store at most one log in a preset dumping time length;
when the logs in the log table are dumped into the log files according to the maximum number of the log pieces, the number of the log pieces allowed to be stored in each log file is less than or equal to the maximum number of the log pieces;
when the logs in the log table are dumped into the log files according to the dumping time length and the maximum number of the logs, each log file allows to store the logs in a preset dumping time length at most, and when the number of the logs in the preset dumping time length is larger than the maximum log capacity, the logs in the dumping time length are dumped into a plurality of log files according to the maximum log capacity.
8. The method as claimed in claim 6, wherein the first association between the name of the log file and the log type and the log time of the log stored in the log file is established as follows:
setting the name of a log file to comprise the log type and the time range of the log stored in the log file, and simultaneously recording the mapping relation between the name of the log file and the starting time of the first log in the log file.
9. The method as claimed in claim 6, 7 or 8, wherein, when auditing logs, searching log files matched with auditing conditions, and analyzing and displaying the searched log files according to the display names recorded by the configuration files comprises:
c1, according to the first incidence relation, searching a log file matched with the type of the log to be audited and the time range to be audited in the auditing condition;
c2, obtaining logs from the searched log files;
c3, obtaining a display name corresponding to the type of the log to be audited, dynamically generating an auditing frame containing the obtained display name, and displaying the log obtained in the step c2 in the generated auditing frame.
10. The method of claim 9, wherein the audit condition further comprises a maximum number of query bars;
the step c2 includes: acquiring logs from searched log files according to the maximum number of inquired logs in the auditing conditions, and stopping log acquisition operation and recording the position of the last log to be acquired when the number of the acquired logs reaches the maximum number of inquired logs;
after the step c3, the method further comprises: when a continue audit command is received, returning to execute the step c2, and starting acquisition from the position of the last acquired log when executing step c 2.
11. The method of claim 9, wherein the step of saving the to-be-dumped field in the searched log table into the log file further comprises:
after the logs in the log table are stored in the log file, the log file in a unit time is packaged into a compressed packet by taking a preset time length as a unit;
for each compressed packet, acquiring a display name corresponding to the log type of the log stored in the compressed packet, outputting the acquired display name and a first association relation corresponding to each log file in the compressed packet to an index file, and adding the index file into the compressed packet; establishing a second association relation between the name of the compressed packet and the log type and the log time of the log stored in the compressed packet;
the step c1 includes:
searching a compressed packet matched with the type of the log to be audited and the time range to be audited in the auditing condition according to the second incidence relation;
and searching a log file matched with the time range to be audited in the auditing condition in the compressed packet according to the first incidence relation recorded by the index file in the compressed packet.
12. The method of claim 9, wherein after step c1 and before step c2, further comprising: and sequencing the searched log files according to time.
13. The method of claim 1, wherein the operation of dumping logs is triggered periodically.
14. A log dumping and auditing system is characterized by comprising a configuration file storage unit, a dumping unit, an auditing unit and a log file storage unit;
the configuration file storage unit is used for reading and storing configuration files from the outside; the configuration file records the name of the log type to be dumped, the field to be dumped corresponding to each log type and the display name of each field to be dumped;
the dumping unit is used for reading the configuration file from the configuration file storage unit when dumping the log, searching a log table matched with the type of the log to be dumped according to the type and the field of the log to be dumped recorded by the read configuration file, storing the field to be dumped in the searched log table into the log file, and storing the log file into the log file storage unit;
the auditing unit is used for searching the log file matched with the auditing conditions in the log file storage unit when the log is audited, and analyzing and displaying the searched log file according to the display name recorded by the configuration file;
the log file storage unit is used for storing log files.
15. The system of claim 14, wherein the configuration file storage unit is further configured to, upon receiving a registration command of the new journal type, read and store a name of the new journal type from outside, a corresponding field to be dumped, and a display name of the field to be dumped;
when a deletion command of an existing log type is received, the name of the log type to be deleted, the corresponding field to be dumped and the display name of the field to be dumped are deleted from the unit.
16. The system of claim 14, wherein the dump unit comprises a log table lookup module, a log dump module, and an association module;
the log table searching module is used for reading the configuration file from the configuration file storage unit when the log is dumped, and searching the log table matched with the type of the log to be dumped according to the type and the field of the log to be dumped recorded by the read configuration file;
the log dumping module is used for dumping the log table searched by the log table searching module; during dumping, according to the capacity limit of the log files, dumping the logs in a log table into one or more log files, and storing the log files obtained by dumping into the log file storage unit;
the association module is used for establishing a first association relationship between the name of the log file obtained by dumping and the log type and the log time of the log stored in the log file after the log dumping module dumps the log into the log file, and storing the first association relationship into the log file storage unit.
17. The system of claim 16, wherein the capacity limitation of the log file is achieved by setting a dump time length and/or a maximum number of log pieces of the log file;
the log dumping module is further used for storing logs in a preset dumping time length at most in each log file when dumping is carried out according to the dumping time length; when dumping is carried out according to the maximum number of the log files, the number of the log files allowed to be stored is smaller than or equal to the maximum number of the log files; when dumping is carried out according to the dumping time length and the maximum number of the logs, each log file allows to store at most one log in a preset dumping time length, and when the number of the logs in the preset dumping time length is larger than the maximum log capacity, the logs in the dumping time length are dumped into a plurality of log files according to the maximum log capacity.
18. The system of claim 16, wherein the association module names the log file obtained by dumping, the name of the log file includes the log type and the time range of the log stored in the log file, and records the mapping relationship between the name of the log file and the start time of the first log in the log file.
19. The system of claim 16 or 17 or 18, wherein the audit unit comprises a log lookup module and an audit display module;
the log searching module is used for searching log files matched with the types of the logs to be audited and the time range to be audited in the auditing conditions in the log file storage unit according to the first incidence relation, and acquiring logs from the searched log files;
the audit display module is used for acquiring a display name corresponding to the type of the log to be audited, dynamically generating an audit frame containing the acquired display name, and displaying the log acquired by the log searching module in the generated audit frame.
20. The system of claim 19, wherein the dump unit further comprises a compression module coupled to the log dump module, the association module, and the configuration file storage unit;
the compression module is used for receiving the log files dumped by the log dump module and packing the log files in unit time into compression packets by taking the preset time length as a unit; for each compressed packet, acquiring a first association relation corresponding to each log file in the compressed packet from the association module, acquiring a display name corresponding to the log type of the log stored in the compressed packet from the configuration file storage unit, outputting the acquired first association relation and the display name to an index file, and adding the index file to the compressed packet; establishing a second association relation between the name of the compressed packet and the log type and the log time of the log stored in the compressed packet;
the log searching module is further used for searching a compressed packet matched with the type of the log to be audited and the time range to be audited in the auditing condition in the log file storage unit according to the second incidence relation; and searching a log file matched with the time range to be audited in the auditing condition in the compressed packet according to the first incidence relation recorded by the index file in the compressed packet.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2008102475508A CN101453378B (en) | 2008-12-30 | 2008-12-30 | Method and system for log dump and audit |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2008102475508A CN101453378B (en) | 2008-12-30 | 2008-12-30 | Method and system for log dump and audit |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101453378A true CN101453378A (en) | 2009-06-10 |
| CN101453378B CN101453378B (en) | 2011-01-12 |
Family
ID=40735409
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2008102475508A Active CN101453378B (en) | 2008-12-30 | 2008-12-30 | Method and system for log dump and audit |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101453378B (en) |
Cited By (23)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101931562A (en) * | 2010-09-29 | 2010-12-29 | 杭州华三通信技术有限公司 | Web log processing method and device |
| CN101944115A (en) * | 2010-09-14 | 2011-01-12 | 杭州海康威视数字技术股份有限公司 | Method and system for searching logs |
| CN102541850A (en) * | 2010-12-09 | 2012-07-04 | 北京北方微电子基地设备工艺研究中心有限责任公司 | Methods and devices and systems for obtaining and uploading log records |
| CN103793479A (en) * | 2014-01-14 | 2014-05-14 | 上海上讯信息技术股份有限公司 | Log management method and log management system |
| CN104067587A (en) * | 2011-09-08 | 2014-09-24 | 英特尔公司 | Methods and arrangements for device profiles in wireless networks |
| CN104506390A (en) * | 2014-12-31 | 2015-04-08 | 上海大唐移动通信设备有限公司 | Log storage method and device of road test system |
| CN105824837A (en) * | 2015-01-06 | 2016-08-03 | 中国移动通信集团广东有限公司 | A log processing method and device |
| CN105975602A (en) * | 2016-05-11 | 2016-09-28 | 广州御银自动柜员机科技有限公司 | Log sorting system |
| CN106202305A (en) * | 2016-06-30 | 2016-12-07 | 北京北信源软件股份有限公司 | A kind of log processing method, device and Database Systems |
| CN106570163A (en) * | 2016-11-07 | 2017-04-19 | 深圳市任子行科技开发有限公司 | Unreliable environment-oriented audit log read-write managing method and system |
| CN106776942A (en) * | 2016-11-30 | 2017-05-31 | 任子行网络技术股份有限公司 | A kind of transmission of network audit daily record preserves system and method |
| CN107145427A (en) * | 2017-05-11 | 2017-09-08 | 暴风体育（北京）有限责任公司 | A kind of method and system of automatic classification monitoring application service daily record |
| CN107305521A (en) * | 2016-04-20 | 2017-10-31 | 百度在线网络技术（北京）有限公司 | Log recording method and device |
| CN107342888A (en) * | 2016-12-02 | 2017-11-10 | 杭州迪普科技股份有限公司 | The storage method and device of daily record message |
| CN107368404A (en) * | 2017-08-02 | 2017-11-21 | 山东浪潮通软信息科技有限公司 | A kind of method of auditing administration and system |
| CN107590056A (en) * | 2017-09-27 | 2018-01-16 | 郑州云海信息技术有限公司 | The dump method and device of audit log in a kind of storage system |
| CN107688624A (en) * | 2017-08-18 | 2018-02-13 | 杭州迪普科技股份有限公司 | A kind of daily record index structuring method and device |
| CN109033813A (en) * | 2018-07-09 | 2018-12-18 | 携程旅游信息技术（上海）有限公司 | The auditing system and method for Linux operation log |
| CN110110516A (en) * | 2019-01-04 | 2019-08-09 | 北京车和家信息技术有限公司 | Log recording method, apparatus and system |
| CN110109809A (en) * | 2019-04-08 | 2019-08-09 | 武汉思普崚技术有限公司 | According to the method and apparatus of syslog test log audit function |
| CN110661650A (en) * | 2019-09-05 | 2020-01-07 | 苏州浪潮智能科技有限公司 | Log management method and device, electronic equipment and storage medium |
| CN110995836A (en) * | 2019-11-29 | 2020-04-10 | 安徽江淮汽车集团股份有限公司 | Log management method, device, storage medium and device based on Internet of vehicles platform |
| CN114338352A (en) * | 2021-12-31 | 2022-04-12 | 南通机敏软件科技有限公司 | Audit log configuration and analysis method, storage medium and processor |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104408136A (en) * | 2014-11-26 | 2015-03-11 | 合肥晶奇电子科技有限公司 | Log treatment method for public medical system |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN100359495C (en) * | 2003-09-04 | 2008-01-02 | 上海格尔软件股份有限公司 | Information system auditing method based on data storehouse |
| CN1917445B (en) * | 2006-09-07 | 2010-09-29 | 上海交通大学 | Firewall log event audit method and teaching experiment system |
| CN101075256A (en) * | 2007-06-08 | 2007-11-21 | 北京神舟航天软件技术有限公司 | System and method for real-time auditing and analyzing database |
-
2008
- 2008-12-30 CN CN2008102475508A patent/CN101453378B/en active Active
Cited By (33)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101944115A (en) * | 2010-09-14 | 2011-01-12 | 杭州海康威视数字技术股份有限公司 | Method and system for searching logs |
| CN101944115B (en) * | 2010-09-14 | 2012-07-25 | 杭州海康威视数字技术股份有限公司 | Method and system for searching logs |
| CN101931562B (en) * | 2010-09-29 | 2013-08-28 | 杭州华三通信技术有限公司 | Web log processing method and device |
| CN101931562A (en) * | 2010-09-29 | 2010-12-29 | 杭州华三通信技术有限公司 | Web log processing method and device |
| CN102541850A (en) * | 2010-12-09 | 2012-07-04 | 北京北方微电子基地设备工艺研究中心有限责任公司 | Methods and devices and systems for obtaining and uploading log records |
| CN104067587B (en) * | 2011-09-08 | 2017-10-20 | 英特尔公司 | Method and apparatus for device configuration file in wireless network |
| CN104067587A (en) * | 2011-09-08 | 2014-09-24 | 英特尔公司 | Methods and arrangements for device profiles in wireless networks |
| US9870380B2 (en) | 2011-09-08 | 2018-01-16 | Intel Corporation | Methods and arrangements for device profiles in wireless networks |
| US10122817B2 (en) | 2011-09-08 | 2018-11-06 | Intel Corporation | Methods and arrangements for device profiles in wireless networks |
| CN103793479A (en) * | 2014-01-14 | 2014-05-14 | 上海上讯信息技术股份有限公司 | Log management method and log management system |
| CN104506390A (en) * | 2014-12-31 | 2015-04-08 | 上海大唐移动通信设备有限公司 | Log storage method and device of road test system |
| CN105824837A (en) * | 2015-01-06 | 2016-08-03 | 中国移动通信集团广东有限公司 | A log processing method and device |
| CN105824837B (en) * | 2015-01-06 | 2019-04-02 | 中国移动通信集团广东有限公司 | A kind of log processing method and device |
| CN107305521A (en) * | 2016-04-20 | 2017-10-31 | 百度在线网络技术（北京）有限公司 | Log recording method and device |
| CN105975602A (en) * | 2016-05-11 | 2016-09-28 | 广州御银自动柜员机科技有限公司 | Log sorting system |
| CN106202305A (en) * | 2016-06-30 | 2016-12-07 | 北京北信源软件股份有限公司 | A kind of log processing method, device and Database Systems |
| CN106570163A (en) * | 2016-11-07 | 2017-04-19 | 深圳市任子行科技开发有限公司 | Unreliable environment-oriented audit log read-write managing method and system |
| CN106776942A (en) * | 2016-11-30 | 2017-05-31 | 任子行网络技术股份有限公司 | A kind of transmission of network audit daily record preserves system and method |
| CN106776942B (en) * | 2016-11-30 | 2019-10-15 | 任子行网络技术股份有限公司 | A kind of transmission preservation system and method for network audit log |
| CN107342888A (en) * | 2016-12-02 | 2017-11-10 | 杭州迪普科技股份有限公司 | The storage method and device of daily record message |
| CN107145427A (en) * | 2017-05-11 | 2017-09-08 | 暴风体育（北京）有限责任公司 | A kind of method and system of automatic classification monitoring application service daily record |
| CN107368404A (en) * | 2017-08-02 | 2017-11-21 | 山东浪潮通软信息科技有限公司 | A kind of method of auditing administration and system |
| CN107688624A (en) * | 2017-08-18 | 2018-02-13 | 杭州迪普科技股份有限公司 | A kind of daily record index structuring method and device |
| CN107688624B (en) * | 2017-08-18 | 2020-12-29 | 杭州迪普科技股份有限公司 | Log index construction method and device |
| CN107590056A (en) * | 2017-09-27 | 2018-01-16 | 郑州云海信息技术有限公司 | The dump method and device of audit log in a kind of storage system |
| CN109033813A (en) * | 2018-07-09 | 2018-12-18 | 携程旅游信息技术（上海）有限公司 | The auditing system and method for Linux operation log |
| CN110110516A (en) * | 2019-01-04 | 2019-08-09 | 北京车和家信息技术有限公司 | Log recording method, apparatus and system |
| CN110109809A (en) * | 2019-04-08 | 2019-08-09 | 武汉思普崚技术有限公司 | According to the method and apparatus of syslog test log audit function |
| CN110109809B (en) * | 2019-04-08 | 2020-04-10 | 武汉思普崚技术有限公司 | Method and equipment for testing log auditing function according to syslog |
| CN110661650A (en) * | 2019-09-05 | 2020-01-07 | 苏州浪潮智能科技有限公司 | Log management method and device, electronic equipment and storage medium |
| CN110661650B (en) * | 2019-09-05 | 2022-06-07 | 苏州浪潮智能科技有限公司 | Log management method and device, electronic equipment and storage medium |
| CN110995836A (en) * | 2019-11-29 | 2020-04-10 | 安徽江淮汽车集团股份有限公司 | Log management method, device, storage medium and device based on Internet of vehicles platform |
| CN114338352A (en) * | 2021-12-31 | 2022-04-12 | 南通机敏软件科技有限公司 | Audit log configuration and analysis method, storage medium and processor |
Also Published As
| Publication number | Publication date |
|---|---|
| CN101453378B (en) | 2011-01-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101453378B (en) | Method and system for log dump and audit | |
| CN109460349B (en) | Test case generation method and device based on log | |
| US8688659B2 (en) | Method for indexed-field based difference detection and correction | |
| CN103793479A (en) | Log management method and log management system | |
| CN103733195A (en) | Managing storage of data for range-based searching | |
| CN107924408A (en) | The system and method that the isomery of search metadata and label indexes in file system | |
| US20030187957A1 (en) | Automatic data download system and method | |
| CN102096626A (en) | Mobile terminal and processing method of test log thereof | |
| CN106021566A (en) | Method, device and system for improving concurrent processing capacity of single database | |
| CN106326280A (en) | Data processing method, apparatus and system | |
| CN117389968A (en) | Vehicle log storage and transmission method and device, vehicle, electronic equipment and medium | |
| CN112965912B (en) | Interface test case generation method and device and electronic equipment | |
| CN112328568B (en) | Service system log recording method and system based on SDK package | |
| US20230122194A1 (en) | Data loading and processing system, and method therefor | |
| CN115994124A (en) | Document scanning method, device, electronic device and computer-readable storage medium | |
| CN111507066B (en) | Method and system for uploading and loading files and analyzing and warehousing | |
| CN110515955A (en) | Storage, querying method, system, electronic equipment and the storage medium of data | |
| CN114817300A (en) | Log query method based on SQL (structured query language) statements and application thereof | |
| CN115550231A (en) | Multi-protocol testing platform based on configuration file and testing method thereof | |
| CN109996089B (en) | Method and system for processing operation log and streaming media server | |
| CN113709059A (en) | Link flow recording method and node | |
| CN113204379B (en) | Method, system, equipment and medium for dynamically loading test resources | |
| CN113835963B (en) | Kubernetes-based automated deployment tool integration method | |
| CN116501579A (en) | Log recording method and system based on ELK automatic classification | |
| CN118014286A (en) | A kind of audit auxiliary system and method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CP03 | Change of name, title or address |
Address after: 310052 Binjiang District Changhe Road, Zhejiang, China, No. 466, No. Patentee after: Xinhua three Technology Co., Ltd. Address before: 310053 Hangzhou hi tech Industrial Development Zone, Zhejiang province science and Technology Industrial Park, No. 310 and No. six road, HUAWEI, Hangzhou production base Patentee before: Huasan Communication Technology Co., Ltd. |
|
| CP03 | Change of name, title or address |