CN112307077A - Data archiving method, device, server and system - Google Patents
Data archiving method, device, server and system Download PDFInfo
- Publication number
- CN112307077A CN112307077A CN201911267638.0A CN201911267638A CN112307077A CN 112307077 A CN112307077 A CN 112307077A CN 201911267638 A CN201911267638 A CN 201911267638A CN 112307077 A CN112307077 A CN 112307077A
- Authority
- CN
- China
- Prior art keywords
- data
- abnormal
- server
- normal
- database
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 48
- 230000002159 abnormal effect Effects 0.000 claims abstract description 116
- 238000004458 analytical method Methods 0.000 claims description 9
- 230000002547 anomalous effect Effects 0.000 claims description 2
- 238000013523 data management Methods 0.000 abstract description 8
- 238000010295 mobile communication Methods 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 230000009286 beneficial effect Effects 0.000 description 4
- 238000004590 computer program Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 3
- 238000007726 management method Methods 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 230000002035 prolonged effect Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2465—Query processing support for facilitating data mining operations in structured databases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2474—Sequence data queries, e.g. querying versioned data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
- G06F16/285—Clustering or classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/243—Classification techniques relating to the number of classes
- G06F18/2433—Single-class perspective, e.g. one-against-all classification; Novelty detection; Outlier detection
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Fuzzy Systems (AREA)
- Probability & Statistics with Applications (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Evolutionary Computation (AREA)
- Evolutionary Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to the technical field of big data, in particular to a data archiving method, a data archiving device, a server and a data archiving system. Applied to a server, the method comprises the following steps: the data management method comprises the steps of obtaining data, analyzing whether the data are abnormal or not, if the data are abnormal, prolonging a data filing period and temporarily storing the data to a real-time database, if the data are normal, storing the data to a historical database, and analyzing whether the data are abnormal or not, so that the data are stored to different databases, and therefore the data management efficiency can be improved.
Description
Technical Field
The invention relates to the technical field of big data, in particular to a data archiving method, a data archiving device, a server and a data archiving system.
Background
Along with the continuous development of information technology, the urban informatization application level is continuously improved, and the construction of smart cities is in due course. By applying the intelligent computing technology, the smart city can enable key infrastructure components and services of city composition such as city management, education, medical treatment, real estate, transportation, public utilities, public safety and the like to be more interconnected, efficient and intelligent.
The data category of the smart city is wide, and the data amount is huge. After the data is accumulated more, the historical data is typically archived. The traditional data filing mode is to file according to the time of data generation, the filing process is long, the efficiency is low, and the process of reusing after filing is complex, so that the data is not beneficial to viewing.
Disclosure of Invention
In view of the foregoing, it is desirable to provide a data archiving method, device, server and system, which can improve data management efficiency.
In a first aspect, an embodiment of the present invention provides a data archiving method, which is applied to a server, and the method includes:
acquiring data;
analyzing whether the data is abnormal;
if the data is abnormal, prolonging the data filing period and temporarily storing the data to a real-time database;
and if the data are normal, storing the data in a historical database.
In some embodiments, the server comprises a preset context database, wherein the preset context database comprises an abnormal context data set and a basic data set;
after analyzing whether the data is abnormal, the method further comprises:
if the data is abnormal, classifying the data into the abnormal situation data set;
if the data is normal, classifying the data into the basic data set.
In some embodiments, said analyzing whether said data is anomalous comprises:
comparing the data with the data in the abnormal situation data set;
if the similarity between the data and the data in the abnormal situation data set is greater than or equal to a preset threshold value, determining that the data is abnormal;
and if the similarity between the data and the data in the abnormal scene data set is smaller than a preset threshold value, determining that the data is normal.
In some embodiments, the storing the data to a history database if the data is normal includes:
if the data are normal, the data are filed to a historical database according to a preset period.
In a second aspect, an embodiment of the present invention further provides a data archiving apparatus, applied to a server, where the apparatus includes:
the acquisition module is used for acquiring data;
the analysis module is used for analyzing whether the data are abnormal or not;
the first storage module is used for prolonging the data archiving period and temporarily storing the data to a real-time database if the data is abnormal;
and the second storage module is used for storing the data to a historical database if the data are normal.
In some embodiments, the analysis module is specifically configured to:
comparing the data with the data in the abnormal situation data set;
if the similarity between the data and the data in the abnormal situation data set is greater than or equal to a preset threshold value, determining that the data is abnormal;
and if the similarity between the data and the data in the abnormal scene data set is smaller than a preset threshold value, determining that the data is normal.
In some embodiments, the second storage module is specifically configured to:
if the data are normal, the data are filed to a historical database according to a preset period.
In a third aspect, an embodiment of the present invention further provides a server, including:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the above-described data archiving method.
In a fourth aspect, an embodiment of the present invention further provides a data archiving system, where the system includes at least one sensor and a server, and the server is connected to the sensor;
the sensor is used for collecting data and sending the data to the server, and the server is used for analyzing and storing the data.
In a fifth aspect, the present invention also provides a non-volatile computer-readable storage medium, which stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, the processor is caused to execute the above data archiving method.
Compared with the prior art, the invention has the beneficial effects that: different from the situation of the prior art, in the data archiving method in the embodiment of the invention, the server acquires data acquired by the sensor and analyzes whether the data is abnormal, if the data is abnormal, the archiving period of the data is prolonged and the data is temporarily stored in the real-time database, so that the data can be conveniently checked at any time, if the data is normal, the data is stored in the historical database, and the data is stored in different databases by analyzing whether the data is abnormal, so that the data management efficiency can be improved.
Drawings
One or more embodiments are illustrated by way of example in the accompanying drawings, which correspond to the figures in which like reference numerals refer to similar elements and which are not to scale unless otherwise specified.
FIG. 1 is a schematic diagram of an application scenario of a data archiving method according to an embodiment of the present invention;
FIG. 2 is a schematic flow chart of a data archiving method according to an embodiment of the present invention;
FIG. 3 is a schematic flow chart of analyzing data for anomalies according to an embodiment of the data archiving method of the present invention;
FIG. 4 is a schematic structural diagram of an embodiment of a data archive device of the present invention;
fig. 5 is a schematic diagram of a hardware structure of a server in an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that, if not conflicted, the various features of the embodiments of the invention may be combined with each other within the scope of protection of the invention. Additionally, while functional block divisions are performed in apparatus schematics, with logical sequences shown in flowcharts, in some cases, steps shown or described may be performed in sequences other than block divisions in apparatus or flowcharts. The terms "first", "second", "third", and the like used in the present invention do not limit data and execution order, but distinguish the same items or similar items having substantially the same function and action.
The data archiving method provided by the embodiment of the invention is applicable to the application scenario shown in fig. 1, and in the embodiment of the invention, the application scenario is a data archiving system and comprises at least one sensor and a server, and the server is in communication connection with the at least one sensor. Fig. 1 illustrates a server 10, a sensor 1, a sensor 2, a sensor N, and may include more servers and sensors in a real network environment. The sensors are connected to the server through network communication, for example, the sensors are connected to the server 10 through local area network, wide area network, wireless network, Global System for Mobile communication (GSM), third generation Mobile communication network, fourth generation Mobile communication network, fifth generation Mobile communication network, and the like. The sensor is used for collecting data in different fields and sending the collected data to the server, and the server is used for analyzing the data collected by the sensor and storing the analyzed data.
The server may be a server, such as a rack server, a blade server, a tower server, or a cabinet server, or may be a server cluster composed of a plurality of servers, or a cloud computing service center.
It should be noted that the method in the embodiment of the present invention may be further extended to other suitable application environments, and is not limited to the application environment shown in fig. 1. In practical applications, the application environment may also include more or fewer sensors and servers.
As shown in fig. 2, an embodiment of the present invention provides a data archiving method, which is applied to a server, and the method includes:
at step 202, data is acquired.
In the embodiment of the present invention, the data may be data of various fields of a smart city, for example, data of a traffic field, an education field, a medical field, a logistics field, and the like. Wherein the type of the acquired data is not limited to audio, video, or files. Specifically, the server acquires data of each field of the smart city acquired by the sensor.
In the embodiment of the invention, the reasons for causing the data exception are many. For example, data acquired when a vehicle drives out of a track, a vehicle collides with the track, a person collides with the vehicle, or the like may be referred to as abnormal data. And after the server acquires the data, analyzing whether the data is abnormal or not.
And step 206, if the data is abnormal, prolonging the data archiving period and temporarily storing the data in a real-time database.
And step 208, if the data is normal, storing the data in a historical database.
In the embodiment of the invention, the real-time database is used for temporarily storing abnormal data, and the historical database is used for storing normal data. It may be determined whether the data can currently be archived based on the analysis of step 204. Specifically, when the server analyzes that the data is abnormal, the filing period of the data is prolonged, and the data is temporarily stored in the real-time database, so that the abnormal data is kept in the working system for a long time and is convenient to check at any time. And when the server analyzes that the data is normal, classifying the data into the historical database so as to finish data archiving.
In the embodiment of the invention, the server acquires the data acquired by the sensor, analyzes whether the data is abnormal or not, prolongs the filing period of the data and temporarily stores the data to the real-time database if the data is abnormal, so that the data can be conveniently checked at any time, stores the data to the historical database if the data is normal, and stores the data to different databases by analyzing whether the data is abnormal or not, so that the data management efficiency can be improved.
In some embodiments, the server comprises a preset context database, wherein the preset context database comprises an abnormal context dataset and a basic dataset,
after analyzing whether the data is abnormal, the method further comprises:
if the data is abnormal, classifying the data into the abnormal situation data set; if the data is normal, classifying the data into the basic data set.
The preset context database stores a plurality of context data information, and the context data information may be, for example, data information in different fields. The preset scene database takes all large behavior recognition scene data sets as basic data sets, abnormal scene data sets as expansion data sets, the abnormal scene data sets are used for storing abnormal data, and the basic data sets are used for storing normal data.
Specifically, it can be determined whether the data can be currently archived based on the analysis result of step 204. When the server analyzes that the data are abnormal, the server classifies the abnormal data into an abnormal situation data set, and when the server analyzes that the data are normal, the server classifies the data into a basic data set. By classifying the normal data into the basic data set and classifying the abnormal data into the abnormal situation data set, the data set is expanded, and updating iteration of the situation database is realized. It is understood that, in some other embodiments, the abnormal data and the abnormal reason may also be stored in the abnormal situation data set together, so as to facilitate the later search of the abnormal reason.
In some embodiments, as shown in fig. 3, the analyzing whether the data is abnormal includes:
In the embodiment of the present invention, since the abnormal situation data set stores a plurality of abnormal data, that is, abnormal situation samples, the abnormal data is composed of a plurality of abnormal segments. For example, if the acquired data is video data, whether the data is abnormal or not is analyzed, and whether the data is abnormal or not can be known through comparing the acquired data with the video segments and intelligently analyzing.
In the embodiment of the invention, abnormal scene samples of data in different fields are stored in the abnormal scene data set. For example, when the acquired data belongs to the traffic field, specifically, a car is driven out of a track, a vehicle collides with the track, a person collides with the car, or the like, the acquired data and abnormal data in a corresponding field in an abnormal situation data set, that is, abnormal situation samples are compared and analyzed, and if the similarity between the acquired data and the data in the abnormal situation data set is 70% and the preset threshold value is 60%, it is known that the similarity between the acquired data and the data in the abnormal situation data set is greater than the preset threshold value, it is determined that the data is abnormal. On one hand, the abnormal data are classified into the abnormal situation data set to enrich the data in the abnormal situation data set, on the other hand, the filing period is prolonged, and the abnormal data are temporarily stored in the real-time database, so that the abnormal data are kept in the working system for a long time and are convenient to check at any time, and the data management efficiency is improved.
If the similarity between the acquired data and the data in the abnormal situation data set is 50% and the preset threshold value is 60%, it is known that the similarity between the acquired data and the data in the abnormal situation data set is smaller than the preset threshold value, and the data is determined to be normal. Normal data is classified into a basic data set on one hand, and normal data is stored into a historical database on the other hand.
In some embodiments, the storing the data to a history database if the data is normal includes: if the data are normal, the data are filed to a historical database according to a preset period. Specifically, the preset period may be, for example, one week, or one month. If the data are normal, the data are filed to a historical database according to a preset period. The preset period can be set according to the service requirement, and the limitation in the embodiment is not required.
In order to facilitate understanding of the present invention, the following description will be made by taking one embodiment as an example:
firstly, a server acquires data of each field of a smart city acquired by a sensor, compares the data with data in an abnormal scene data set in a preset scene information base, specifically, when the acquired data belongs to the traffic field, specifically, a car is driven out of a track, a vehicle collides with the track, a person collides with the car, and the like, compares the acquired data with abnormal data in a corresponding field in the abnormal scene data set, namely, an abnormal scene sample, and determines that the data is abnormal if the acquired data has a similarity of 70% with the data in the abnormal scene data set and a preset threshold value of 60%, and if the acquired data has a similarity of more than the preset threshold value with the data in the abnormal scene data set. If the similarity between the acquired data and the data in the abnormal situation data set is 50% and the preset threshold value is 60%, it is known that the similarity between the acquired data and the data in the abnormal situation data set is smaller than the preset threshold value, and the data is determined to be normal.
And then, classifying the abnormal data into an abnormal situation data set so as to continuously update the data in the abnormal situation data set, prolonging the filing period, and temporarily storing the abnormal data into a real-time database, so that the abnormal data is kept in a working system for a long time and is convenient to check at any time, and the data management efficiency is improved. And on the other hand, normal data are classified into a basic data set, and the normal data are stored into a historical database.
Correspondingly, an embodiment of the present invention further provides a data archiving apparatus 400, where the apparatus is applied to a server, and as shown in fig. 4, the apparatus includes:
an obtaining module 402, configured to obtain data;
an analysis module 404, configured to analyze whether the data is abnormal;
the first storage module 406 is configured to, if the data is abnormal, extend the data archiving period and temporarily store the data in a real-time database;
the second storage module 408 is configured to store the data in the history database if the data is normal.
Optionally, in another embodiment of the apparatus, referring to fig. 4, the apparatus 400 further includes:
a classification template 410, configured to classify the data into the abnormal situation data set if the data is abnormal; if the data is normal, classifying the data into the basic data set.
Optionally, in some embodiments, the analysis module 404 is specifically configured to:
comparing the data with data in an abnormal situation data set;
if the similarity between the data and the data in the abnormal situation data set is greater than or equal to a preset threshold value, determining that the data is abnormal;
and if the similarity between the data and the data in the abnormal scene data set is smaller than a preset threshold value, determining that the data is normal.
Optionally, in some embodiments, the second storage module 408 is specifically configured to:
if the data are normal, the data are filed to a historical database according to a preset period.
According to the data archiving device provided by the embodiment of the invention, the data is acquired through the acquisition module, then the data is analyzed through the analysis module, whether the data is abnormal or not is judged, if the data is abnormal, the first storage module prolongs the data archiving period and temporarily stores the data into the real-time database for convenient checking, if the data is normal, the second storage module stores the data into the historical database, and if the data is abnormal or not, the data is stored into different databases, so that the data management efficiency can be improved.
It should be noted that the data archiving device can execute the data archiving method provided by the embodiment of the present invention, and has corresponding functional modules and beneficial effects of the execution method, and reference may be made to the data archiving method provided by the embodiment of the present invention without detailed technical details in the embodiment of the data archiving device.
Fig. 5 is a schematic diagram of a hardware structure of a server provided in the present invention, and as shown in fig. 5, the server 500 includes:
one or more processors 502 and a memory 504, with one processor 502 being an example in FIG. 5.
The processor 502 and the memory 504 may be connected by a bus or other means, such as by a bus in FIG. 5.
The memory 504, which is a non-volatile computer-readable storage medium, may be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as instructions/modules corresponding to the data archiving method in the embodiment of the present invention (for example, the obtaining module 402, the analyzing module 404, the first storage module 406, the second storage module 408, and the classification template 410 shown in fig. 4). The processor 502 executes various functional applications of the server and data processing, i.e., implementing the data archiving method of the above-described method embodiment, by executing the nonvolatile software program, instructions and modules stored in the memory 504.
The memory 504 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created according to use of the data archive device, and the like. Further, the memory 504 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some embodiments, memory 504 may optionally include memory located remotely from processor 502, which may be connected to a data archive over a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The one or more modules stored in the memory 504, when executed by the one or more servers, perform the data archiving method in any of the above-described method embodiments, e.g., performing the above-described method steps 202-208 of fig. 2, 302-306 of fig. 3; the function of step 410 in block 402 in figure 4 is implemented.
The server 500 in embodiments of the present invention exists in a variety of forms, including but not limited to:
(1) tower server
The general tower server chassis is almost as large as the commonly used PC chassis, while the large tower chassis is much larger, and the overall dimension is not a fixed standard.
(2) Rack-mounted server
Rack-mounted servers are a type of server that has a standard width of 19 inch racks, with a height of from 1U to several U, due to the dense deployment of the enterprise. Placing servers on racks not only facilitates routine maintenance and management, but also may avoid unexpected failures. First, placing the server does not take up too much space. The rack servers are arranged in the rack in order, and no space is wasted. Secondly, the connecting wires and the like can be neatly stored in the rack. The power line, the LAN line and the like can be distributed in the cabinet, so that the connection lines accumulated on the ground can be reduced, and the accidents such as the electric wire kicking off by feet can be prevented. The specified dimensions are the width (48.26cm ═ 19 inches) and height (multiples of 4.445 cm) of the server. Because of its 19 inch width, a rack that meets this specification is sometimes referred to as a "19 inch rack".
(3) Blade server
A blade server is a HAHD (High Availability High Density) low cost server platform designed specifically for the application specific industry and High Density computer environment, where each "blade" is actually a system motherboard, similar to an individual server. In this mode, each motherboard runs its own system, serving a designated group of different users, without any relationship to each other. Although system software may be used to group these motherboards into a server cluster. In the cluster mode, all motherboards can be connected to provide a high-speed network environment, and resources can be shared to serve the same user group.
(4) Cloud server
The cloud server (ECS) is a computing Service with simplicity, high efficiency, safety, reliability, and flexible processing capability. The management mode is simpler and more efficient than that of a physical server, and a user can quickly create or release any plurality of cloud servers without purchasing hardware in advance. The distributed storage of the cloud server is used for integrating a large number of servers into a super computer, and a large number of data storage and processing services are provided. The distributed file system and the distributed database allow access to common storage resources, and IO sharing of application data files is achieved. The virtual machine can break through the limitation of a single physical machine, dynamically adjust and allocate resources to eliminate single-point faults of the server and the storage equipment, and realize high availability.
Embodiments of the present invention also provide a computer program product, including a computer program stored on a non-volatile computer-readable storage medium, the computer program including program instructions that, when executed by a computer, cause the computer to perform: method steps 202 through 208 in fig. 2.
The product can execute the method provided by the embodiment of the invention, and has corresponding functional modules and beneficial effects of the execution method. For technical details that are not described in detail in this embodiment, reference may be made to the method provided by the embodiment of the present invention.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a general hardware platform, and certainly can also be implemented by hardware. It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware related to instructions of a computer program, which can be stored in a computer readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; within the idea of the invention, also technical features in the above embodiments or in different embodiments may be combined, steps may be implemented in any order, and there are many other variations of the different aspects of the invention as described above, which are not provided in detail for the sake of brevity; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present invention.
The above description is only an embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes performed by the present specification and drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (10)
1. A data archiving method is applied to a server, and is characterized by comprising the following steps:
acquiring data;
analyzing whether the data is abnormal;
if the data is abnormal, prolonging the data filing period and temporarily storing the data to a real-time database;
and if the data are normal, storing the data in a historical database.
2. The method of claim 1, wherein the server comprises a preset context database, wherein the preset context database comprises an abnormal context data set and a basic data set;
after analyzing whether the data is abnormal, the method further comprises:
if the data is abnormal, classifying the data into the abnormal situation data set;
if the data is normal, classifying the data into the basic data set.
3. The method of claim 2, wherein said analyzing whether said data is anomalous comprises:
comparing the data with the data in the abnormal situation data set;
if the similarity between the data and the data in the abnormal situation data set is greater than or equal to a preset threshold value, determining that the data is abnormal;
and if the similarity between the data and the data in the abnormal scene data set is smaller than a preset threshold value, determining that the data is normal.
4. The method of claim 3, wherein storing the data to a historical database if the data is normal comprises:
if the data are normal, the data are filed to a historical database according to a preset period.
5. A data archiving apparatus, applied to a server, the apparatus comprising:
the acquisition module is used for acquiring data;
the analysis module is used for analyzing whether the data are abnormal or not;
the first storage module is used for prolonging the data archiving period and temporarily storing the data to a real-time database if the data is abnormal;
and the second storage module is used for storing the data to a historical database if the data are normal.
6. The apparatus of claim 5, wherein the analysis module is specifically configured to:
comparing the data with data in an abnormal situation data set;
if the similarity between the data and the data in the abnormal situation data set is greater than or equal to a preset threshold value, determining that the data is abnormal;
and if the similarity between the data and the data in the abnormal scene data set is smaller than a preset threshold value, determining that the data is normal.
7. The apparatus of claim 6, wherein the second storage module is specifically configured to:
if the data are normal, the data are filed to a historical database according to a preset period.
8. A server, comprising:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-4.
9. A data archiving system, characterized in that said system comprises at least one sensor and a server according to claim 8, said server being connected to said sensor;
the sensor is used for collecting data and sending the data to the server, and the server is used for analyzing and storing the data.
10. A non-transitory computer-readable storage medium storing computer-executable instructions that, when executed by a processor, cause the processor to perform the method of any one of claims 1-4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911267638.0A CN112307077A (en) | 2019-12-11 | 2019-12-11 | Data archiving method, device, server and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911267638.0A CN112307077A (en) | 2019-12-11 | 2019-12-11 | Data archiving method, device, server and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112307077A true CN112307077A (en) | 2021-02-02 |
Family
ID=74336223
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911267638.0A Pending CN112307077A (en) | 2019-12-11 | 2019-12-11 | Data archiving method, device, server and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112307077A (en) |
Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2009274588A (en) * | 2008-05-14 | 2009-11-26 | Mitsubishi Heavy Ind Ltd | Aircraft soundness diagnostic device, method, and program |
| CN103678493A (en) * | 2013-11-13 | 2014-03-26 | 许继集团有限公司 | Data storage system and method for monitoring operation of electric automobiles |
| CN104986347A (en) * | 2015-06-03 | 2015-10-21 | 中国民航大学 | Real-time detection method for civil aircraft airline pilot operation errors |
| US20150347493A1 (en) * | 2014-05-29 | 2015-12-03 | Samsung Sds Co., Ltd. | System and method for processing data |
| CN106851229A (en) * | 2017-04-01 | 2017-06-13 | 山东瀚岳智能科技股份有限公司 | A kind of method and system of the security protection intelligent decision based on image recognition |
| KR20170067519A (en) * | 2015-12-08 | 2017-06-16 | 두산중공업 주식회사 | Integrated Diagnostic System and Database based on rules and cases |
| CN108387794A (en) * | 2018-01-09 | 2018-08-10 | 国网山东省电力公司滨州供电公司 | A kind of electrical energy monitoring system |
| CN108520569A (en) * | 2018-03-30 | 2018-09-11 | 深圳市华宝电子科技有限公司 | Storage method, device, system and the storage medium of video data |
| CN109327345A (en) * | 2017-08-01 | 2019-02-12 | 中国移动通信集团湖北有限公司 | Method and device for detecting abnormal network traffic, and computer-readable storage medium |
| CN109542868A (en) * | 2018-09-28 | 2019-03-29 | 中国平安人寿保险股份有限公司 | Position method, apparatus, electronic equipment and the storage medium of abnormal SQL statement |
| CN109634942A (en) * | 2018-11-16 | 2019-04-16 | 许继集团有限公司 | A kind of energy data exception judgment method and device |
| US20190302713A1 (en) * | 2018-03-27 | 2019-10-03 | Terminus (Beijing) Technology Co., Ltd. | Method and device for automatically diagnosing and controlling apparatus in intelligent building |
| CN110457194A (en) * | 2019-08-02 | 2019-11-15 | 广东小天才科技有限公司 | Electronic equipment stability early warning method, system, device, equipment and storage medium |
-
2019
- 2019-12-11 CN CN201911267638.0A patent/CN112307077A/en active Pending
Patent Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2009274588A (en) * | 2008-05-14 | 2009-11-26 | Mitsubishi Heavy Ind Ltd | Aircraft soundness diagnostic device, method, and program |
| CN103678493A (en) * | 2013-11-13 | 2014-03-26 | 许继集团有限公司 | Data storage system and method for monitoring operation of electric automobiles |
| US20150347493A1 (en) * | 2014-05-29 | 2015-12-03 | Samsung Sds Co., Ltd. | System and method for processing data |
| CN104986347A (en) * | 2015-06-03 | 2015-10-21 | 中国民航大学 | Real-time detection method for civil aircraft airline pilot operation errors |
| KR20170067519A (en) * | 2015-12-08 | 2017-06-16 | 두산중공업 주식회사 | Integrated Diagnostic System and Database based on rules and cases |
| CN106851229A (en) * | 2017-04-01 | 2017-06-13 | 山东瀚岳智能科技股份有限公司 | A kind of method and system of the security protection intelligent decision based on image recognition |
| CN109327345A (en) * | 2017-08-01 | 2019-02-12 | 中国移动通信集团湖北有限公司 | Method and device for detecting abnormal network traffic, and computer-readable storage medium |
| CN108387794A (en) * | 2018-01-09 | 2018-08-10 | 国网山东省电力公司滨州供电公司 | A kind of electrical energy monitoring system |
| US20190302713A1 (en) * | 2018-03-27 | 2019-10-03 | Terminus (Beijing) Technology Co., Ltd. | Method and device for automatically diagnosing and controlling apparatus in intelligent building |
| CN108520569A (en) * | 2018-03-30 | 2018-09-11 | 深圳市华宝电子科技有限公司 | Storage method, device, system and the storage medium of video data |
| CN109542868A (en) * | 2018-09-28 | 2019-03-29 | 中国平安人寿保险股份有限公司 | Position method, apparatus, electronic equipment and the storage medium of abnormal SQL statement |
| CN109634942A (en) * | 2018-11-16 | 2019-04-16 | 许继集团有限公司 | A kind of energy data exception judgment method and device |
| CN110457194A (en) * | 2019-08-02 | 2019-11-15 | 广东小天才科技有限公司 | Electronic equipment stability early warning method, system, device, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8949677B1 (en) | Detecting anomalies in time series data | |
| CN109740004B (en) | Filing method and device | |
| US10834183B2 (en) | Managing idle and active servers in cloud data centers | |
| CN105898205B (en) | Method and apparatus for monitoring target object by multiple cameras | |
| US10025671B2 (en) | Smart virtual machine snapshotting | |
| CN106815125A (en) | A kind of log audit method and platform | |
| US10834017B2 (en) | Cloud-driven hybrid data flow and collection | |
| CN111131304A (en) | Method and system for detecting fine-grained abnormal behavior of large-scale virtual machines on cloud platforms | |
| CN111523527B (en) | Special transport vehicle monitoring method and device, medium and electronic equipment | |
| CN112799785B (en) | Virtual machine cluster migration method, device, equipment and medium | |
| US12242995B2 (en) | Environment change management and risk analysis | |
| CN112784025B (en) | Method and device for determining target event | |
| CN113505048A (en) | Unified monitoring platform based on application system portrait and implementation method | |
| CN112364176A (en) | Method, equipment and system for constructing personnel action track | |
| CN110807050B (en) | Performance analysis method, device, computer equipment and storage medium | |
| CN106326082B (en) | Method and device for recording log in network system | |
| CN103716384A (en) | Method and device for realizing cloud storage data synchronization in cross-data-center manner | |
| CN108347698A (en) | A kind of on-line off-line event trace analysis method, apparatus and system | |
| US9613271B2 (en) | Determining severity of a geomagnetic disturbance on a power grid using similarity measures | |
| CN110322688A (en) | A kind of method of data processing, the method for data query and relevant device | |
| CN111970151A (en) | Flow fault positioning method and system for virtual and container network | |
| US9912545B2 (en) | High performance topology resolution for non-instrumented nodes | |
| CN112307077A (en) | Data archiving method, device, server and system | |
| CN104077422A (en) | Repeated APK removing method and device in APK downloading | |
| CN116469039B (en) | Hot video event determination method and system, storage medium and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |