CN106844091A - One kind compression, restoring method and terminal - Google Patents
One kind compression, restoring method and terminal Download PDFInfo
- Publication number
- CN106844091A CN106844091A CN201611125510.7A CN201611125510A CN106844091A CN 106844091 A CN106844091 A CN 106844091A CN 201611125510 A CN201611125510 A CN 201611125510A CN 106844091 A CN106844091 A CN 106844091A
- Authority
- CN
- China
- Prior art keywords
- message digest
- character string
- target character
- digest value
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
- 238000000034 method Methods 0.000 title claims abstract description 50
- 230000006835 compression Effects 0.000 title claims abstract description 21
- 238000007906 compression Methods 0.000 title claims abstract description 21
- 230000009467 reduction Effects 0.000 claims abstract description 5
- 238000001514 detection method Methods 0.000 claims abstract description 3
- 238000010586 diagram Methods 0.000 description 10
- 238000004891 communication Methods 0.000 description 8
- 230000008569 process Effects 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000012795 verification Methods 0.000 description 3
- 230000001413 cellular effect Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000003203 everyday effect Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1458—Management of the backup or restore process
- G06F11/1469—Backup restoration techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1471—Saving, restoring, recovering or retrying involving logging of persistent data for recovery
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/174—Redundancy elimination performed by the file system
- G06F16/1744—Redundancy elimination performed by the file system using compression, e.g. sparse files
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Quality & Reliability (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the invention discloses one kind compression, restoring method and terminal.A kind of compression method includes：The target string in file destination is obtained, wherein, the size of the target string is more than predetermined threshold value；Whether there is first message digest value corresponding with the target string in detection backup database；If do not had, then the target string is compressed so as to obtain the second message digest value by Message Digest 5, and the target string is stored in the backup database with the corresponding relation of second message digest value, wherein, length of the length of second message digest value less than the target string；The target string is substituted so as to obtain the compressed file of the file destination with second message digest value.Methods described can be compressed storage to big character string, save memory space, and the corresponding relation of big character string and message digest value is backed up into storage, the convenient reduction compressed file.
Description
Technical Field
The invention relates to the technical field of electronics, in particular to a compression and reduction method and a terminal.
Background
With the rapid development of the internet, a large amount of data is generated every day, and a large amount of repeated data with large bytes is generated, so that the storage space is wasted during storage, and the query performance is also influenced. In order to save storage space, data is generally compressed and stored by using a compression algorithm.
The existing compression method compresses data through algorithms such as Quicklz, Snappy, zlib and the like, and decompresses the data when the data is needed, so as to achieve the purpose of saving storage space.
Disclosure of Invention
The embodiment of the invention provides a compression and recovery method and a terminal, which can replace a large character string with an abstract value for storage and save storage space.
In a first aspect, an embodiment of the present invention provides a compression method, including:
acquiring a target character string in a target file, wherein the size of the target character string is larger than a preset threshold value;
detecting whether a first message digest value corresponding to the target character string exists in a backup database;
if not, compressing the target character string through a message digest algorithm to obtain a second message digest value, and storing the corresponding relation between the target character string and the second message digest value in the backup database, wherein the length of the second message digest value is smaller than that of the target character string;
and replacing the target character string with the second message digest value to obtain a compressed file of the target file.
In a second aspect, an embodiment of the present invention provides a reduction method, including:
acquiring a message digest value in a compressed file, wherein the message digest value is obtained by compressing a target character string through a message digest algorithm, and the length of the message digest value is smaller than that of the target character string;
acquiring a target character string corresponding to the message digest value from a backup database, wherein the corresponding relation between the target character string and the message digest value is stored in the backup database;
and replacing the message digest value with the corresponding target character string to obtain an original file of the compressed file.
In a third aspect, an embodiment of the present invention provides a terminal, including:
the device comprises a first acquisition unit, a second acquisition unit and a third acquisition unit, wherein the first acquisition unit is used for acquiring a target character string in a target file, and the size of the target character string is larger than a preset threshold value;
the first detection unit is used for detecting whether a first message digest value corresponding to the target character string exists in a backup database or not;
the compressing unit is used for compressing the target character string through a message digest algorithm to obtain a second message digest value, wherein the length of the second message digest value is smaller than that of the target character string;
the storage unit is used for storing the corresponding relation between the target character string and the second message abstract value into the backup database;
and the replacing unit is used for replacing the target character string with the second message digest value so as to obtain a compressed file of the target file.
In a fourth aspect, an embodiment of the present invention provides a terminal, including:
the device comprises a first acquisition unit, a second acquisition unit and a third acquisition unit, wherein the first acquisition unit is used for acquiring a message digest value in a compressed file, the message digest value is obtained by compressing a target character string through a message digest algorithm, and the length of the message digest value is smaller than that of the target character string;
a second obtaining unit, configured to obtain a target character string corresponding to the message digest value from a backup database, where a correspondence between the target character string and the message digest value is stored in the backup database;
and the replacing unit is used for replacing the message digest value with the corresponding target character string so as to obtain the original file of the compressed file.
In the embodiment of the invention, the target character string in the target file is acquired, whether the first message digest value corresponding to the target character string exists in the backup database is detected, if not, the target character string is compressed through a message digest algorithm to obtain the second message digest value, the corresponding relation between the target character string and the second message digest value is stored in the backup database, and the target character string is replaced by the second message digest value to obtain the compressed file of the target file.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
FIG. 1 is a schematic diagram of a prior art logging;
FIG. 2 is a flow chart of a compression method provided by an embodiment of the invention;
FIG. 3 is a schematic diagram of a compressed log record provided by an embodiment of the invention;
FIG. 4 is a flow chart of another compression method provided by an embodiment of the present invention;
FIG. 5 is a flow chart of a reduction method provided by an embodiment of the present invention;
fig. 6 is a schematic structural diagram of a terminal according to an embodiment of the present invention;
fig. 7 is a schematic structural diagram of another terminal according to an embodiment of the present invention;
fig. 8 is a schematic structural diagram of another terminal provided in an embodiment of the present invention;
fig. 9 is a schematic structural diagram of another terminal according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It is to be understood that the terminology used in the embodiments of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in the examples of the present invention and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.
With the rapid development of the internet, a large amount of data is generated every day. In the server, log records interacting with the terminal are stored in a log database, each log record comprises a user agent field for storing basic information (such as browser identification, operating system identification, browser language, version information and the like) of the terminal, the user agent field has a size of more than 200 bytes, and other fields in one log record are added to be less than 80 bytes. For the same terminal, a plurality of log records are stored in the log database, and the Useragent field in each log record is the same, which is equivalent to that the Useragent field is repeatedly stored for many times, thus wasting storage space. As shown in fig. 1, fig. 1 is a schematic diagram illustrating byte sizes of fields in a log record.
In order to solve the above problem, an embodiment of the present invention discloses a compression method. Referring to fig. 2, fig. 2 is a flowchart of a compression method according to an embodiment of the present invention, the method includes the following steps:
s201: the method comprises the steps of obtaining a target character string in a target file, wherein the size of the target character string is larger than a preset threshold value.
In this embodiment, the target file is a file that needs to be compressed, for example, a log database in a server. The target character string is a character string of which the size is larger than a preset threshold value in the target file, wherein the preset threshold value is set or defaulted by a user according to the specific situation of the target file.
In a specific embodiment, in the log database of the server, assuming that the default preset threshold is 200 bytes, and the size of the user agent field in one log record exceeds the preset threshold of 200 bytes, the user agent field is the target character string.
In this embodiment, the target file may be a stored file, and the acquiring a target character string in the target file specifically includes: periodically detecting whether a character string with the size larger than a preset threshold value exists in the target file, if so, taking the character string as a target character string, and acquiring the target character string in the target file; in other embodiments, the target file may be a file that is performing a storage operation, and the obtaining a target character string in the target file specifically includes: when the target file is stored, whether a character string with the size larger than a preset threshold value exists in the target file is detected, if yes, the character string is the target character string, and the target character string in the target file is obtained.
S202: and detecting whether a first message digest value corresponding to the target character string exists in a backup database.
Specifically, the backup database is configured to store a corresponding relationship between a character string and a message digest value, where the message digest value is obtained by compressing the character string through a message digest algorithm.
In this embodiment, whether the backup database has the first message digest value corresponding to the target character string is detected, and if not, the steps S203-S204 are executed to ensure that the correspondence between the target character string and the message digest value for the same target character string in the backup database is only stored once, so that the storage capacity is reduced, and the search is facilitated.
In another embodiment, if the backup database has a first message digest value corresponding to the target character string, the following steps are executed: and acquiring the first message digest value corresponding to the target character string from the backup database, and replacing the target character string with the corresponding first message digest value to obtain a compressed file of the target file. Specifically, if a first message digest value corresponding to the target character string exists in the backup database, it indicates that the target character string has appeared in a previously stored file, and the target character string is stored in the backup database after a compression operation.
S203: and if not, compressing the target character string through a message digest algorithm to obtain a second message digest value, and storing the corresponding relation between the target character string and the second message digest value in the backup database, wherein the length of the second message digest value is smaller than that of the target character string.
In this embodiment, the Message Digest Algorithm may be any one of MD5(Message Digest Algorithm fifth version 5), sha-1(Secure Hash Algorithm), ripem (RACE original integrity verification Message Digest), and a Hash Algorithm, and may also be other compression algorithms capable of effectively compressing large strings.
In this embodiment, the message digest algorithm compresses a target character string of any size into a second message digest value of a fixed length, and two different target character strings do not generate the same message digest value, thereby ensuring a unique correspondence between the target character string and the message digest value.
S204: and replacing the target character string with the second message digest value to obtain a compressed file of the target file.
Specifically, the target character string in the target file is replaced by the second message digest value, and the size of the target file is reduced, so that the purpose of compressing the target file is achieved.
In an embodiment, as shown in fig. 3, fig. 3 is a schematic diagram of replacing a target string with a message digest value in a log record, a plurality of log records in a log database of a server each include a same 200-byte user agent field, the user agent field is compressed into a fixed 16-byte message digest value by using an MD5 algorithm, the message digest value is represented by UAMd5 in fig. 3, and the 16-byte message digest value is used to replace the user agent field in the plurality of log records under the terminal for storage. If 1000 log records exist in the log database for the terminal, the total storage capacity is reduced by about 200000 bytes after the 16-byte message digest value is used for replacing the user agent field for storage.
As can be seen from the above, in the embodiment of the present invention, by obtaining a target character string in a target file, detecting whether a first message digest value corresponding to the target character string exists in a backup database, and if not, compressing the target character string by using a message digest algorithm to obtain a second message digest value, storing a correspondence between the target character string and the second message digest value in the backup database, and replacing the target character string with the second message digest value to obtain a compressed file of the target file, a storage space can be saved, a correspondence between a large character string and a message digest value is backed up and stored, so that the compressed file is conveniently restored.
Referring to fig. 4, fig. 4 is a flowchart of another compression method according to an embodiment of the present invention, the method includes the following steps:
s401: the method comprises the steps of obtaining a target character string in a target file, wherein the size of the target character string is larger than a preset threshold value.
In this embodiment, the target file is a file already stored, for example, a log database in a server. The target character string is a character string of which the size is larger than a preset threshold value in the target file, wherein the preset threshold value is set or defaulted by a user according to the specific situation of the target file.
In a specific embodiment, in the log database of the server, the default preset threshold is 200 bytes, and the size of the user agent field in one log record exceeds the preset threshold of 200 bytes, so that the user agent field is the target character string.
S402: and detecting the repeated times of the target character string in the target file.
S403: and detecting whether a first message digest value corresponding to the target character string exists in a backup database.
In this embodiment, whether the backup database has the first message digest value corresponding to the target character string is detected, and if not, the steps S404 to S406 are executed to ensure that the correspondence between the target character string and the message digest value for the same target character string in the backup database is only stored once, so that the storage capacity is reduced, and the search is facilitated.
S404: and under the condition that the repetition times of the target character string are greater than a quantity threshold value and a first message digest value corresponding to the target character string does not exist in the data backup library, compressing the target character string through a message digest algorithm to obtain a second message digest value, wherein the length of the second message digest value is smaller than that of the target character string.
Specifically, the number of times of repetition of the target character string in the target file is detected, that is, the number of times of repeated storage of the target character string in the target file is detected, and when the number of times of repetition of the target character string is greater than a number threshold, it indicates that the number of times of repeated storage of the target character string reaches a certain number, and the storage space wasted by repeated storage also reaches a certain degree, and in this case, the target character string is compressed, so that more storage space can be saved. Wherein the number threshold is set or defaulted by a user according to the specific situation of the target file.
In this embodiment, the Message Digest Algorithm may be any one of MD5(Message Digest Algorithm fifth version 5), sha-1(Secure Hash Algorithm), ripem (RACE original integrity verification Message Digest), and a Hash Algorithm, and may also be other compression algorithms capable of effectively compressing large strings. In the embodiment, the message digest algorithm compresses the target character string with any size into the message digest value with fixed length, and two different target character strings cannot generate the same message digest value, so that the unique corresponding relation between the target character string and the message digest value is ensured.
S405: and storing the corresponding relation between the target character string and the second message digest value in the backup database.
S406: and replacing the target character string with the second message digest value to obtain a compressed file of the target file.
Specifically, the target character string in the target file is replaced by the message digest value, and the size of the target file is reduced, so that the purpose of compressing the target file is achieved.
In a specific embodiment, in a log database of a server, there are 1000 log records for a terminal, each log record includes the same 200-byte user agent field, the repeatedly stored user agent fields exceed a quantity threshold, the user agent fields are compressed into a fixed 16-byte message digest value by using an MD5 algorithm, and the user agent fields in the 1000 log records under the terminal are all stored by replacing the 16-byte message digest values. If 1000 log records exist in the log database for the terminal, the total storage capacity is reduced by about 200000 bytes after the 16-byte message digest value is used for replacing the user agent field for storage.
In the above description, the compression method includes obtaining a target character string in a target file, compressing the target character string by using a message digest algorithm under the condition that the repeated position of the target character string in the target file is greater than a quantity threshold value, so as to obtain a message digest value, storing the corresponding relationship between the target character string and the message digest value in the backup database, and replacing the target character string with the message digest value so as to obtain a compressed file of the target file, so that the storage space can be saved, the corresponding relationship between a large character string and a message digest value can be backed up and stored, and the compressed file can be conveniently restored.
Referring to fig. 5, fig. 5 is a restoring method according to an embodiment of the present invention, which restores a compressed file obtained by the compression method shown in fig. 2 or fig. 4. The method comprises the following steps:
s501: and acquiring a message digest value in the compressed file, wherein the message digest value is obtained by compressing a target character string through a message digest algorithm, and the length of the message digest value is smaller than that of the target character string.
In this embodiment, when data needs to be checked, a compressed file needs to be restored, and a message digest value in the compressed file is first obtained, where the message digest value is obtained by compressing a target character string through a message digest algorithm in the method embodiment shown in fig. 2 or fig. 4.
In this embodiment, the Message Digest Algorithm may be any one of MD5(Message Digest Algorithm gmmd 5, fifth version of Message Digest Algorithm), sha-1(Secure Hash Algorithm), ripem (RACE raw integrity verification Message Digest), and a Hash Algorithm. In the embodiment, the message digest algorithm compresses the target character string with any size into the message digest value with fixed length, and two different target character strings cannot generate the same message digest value, so that the unique corresponding relation between the target character string and the message digest value is ensured.
S502: and acquiring a target character string corresponding to the message digest value from a backup database, wherein the corresponding relation between the target character string and the message digest value is stored in the backup database.
S503: and replacing the message digest value with the corresponding target character string to obtain an original file of the compressed file.
Specifically, the message digest value in the compressed file is replaced with the corresponding target character string to obtain an original file of the compressed file, so that the purpose of restoring the compressed file is achieved.
In a specific embodiment, a plurality of log records are stored in a log database of a server, each log record includes a same 200-byte user agent field, the plurality of log records are stored in a manner of replacing the user agent field with a message digest value by using the method shown in fig. 2 or fig. 4, and at this time, the log record is shown in fig. 3, and if analysis or other operations need to be performed on the log record of the terminal in the log database, the compressed log record needs to be restored. Firstly, obtaining a message digest value in the log record, finding a UserAgent field corresponding to the message digest value in a backup database, and then replacing the message digest value in the log record with the UserAgent field to obtain an original file of the log record, wherein the log record is shown in FIG. 1.
In the foregoing, the restoring method obtains the message digest value in the compressed file, obtains the target character string corresponding to the message digest value from the backup database, and replaces the message digest value with the corresponding target character string, thereby obtaining the original file of the compressed file.
While the method of the embodiment of the present invention has been described in detail, in order to better implement the above-described aspects of the embodiment of the present invention, the following also provides a terminal for implementing the above-described aspects.
Referring to fig. 6, fig. 6 is a terminal according to an embodiment of the present invention, where the terminal includes the following functional units: a first obtaining unit 610, configured to obtain a target character string in a target file, where a size of the target character string is greater than a preset threshold;
a first detecting unit 620, configured to detect whether a first message digest value corresponding to the target character string exists in a backup database;
a compressing unit 630, configured to compress the target character string through a message digest algorithm to obtain a second message digest value, where a length of the second message digest value is smaller than a length of the target character string;
a storage unit 640, which stores the corresponding relationship between the target character string and the second message digest value in the backup database;
a replacing unit 650, configured to replace the target character string with the second message digest value, so as to obtain a compressed file of the target file.
Optionally, the method further comprises: a second detecting unit 660, configured to detect a number of times that the target character string is repeated in the target file; the compressing unit is specifically configured to, when the repetition number of the target character string is greater than a number threshold, compress the target character string by using a message digest algorithm to obtain the second message digest value.
Optionally, the method further comprises: a second obtaining unit 670, configured to obtain the first message digest value corresponding to the target character string from the backup database; the replacing unit is specifically configured to replace the target character string with the corresponding first message digest value to obtain a compressed file of the target file.
Optionally, the message digest algorithm includes: any one of MD5, sha-1, RIPEMD and Haval.
It can be understood that the functions of the functional units in the terminal 600 in the embodiment of the present invention may be specifically implemented according to the method in the foregoing method embodiment, and the specific implementation process may refer to the related description in the foregoing method embodiment of fig. 2 or fig. 4, which is not described herein again.
Referring to fig. 7, fig. 7 is a terminal according to an embodiment of the present invention, where the terminal restores a compressed file after compression processing by the terminal shown in fig. 6, and the terminal 700 includes the following functional units:
a first obtaining unit 710, configured to obtain a message digest value in a compressed file, where the message digest value is obtained by compressing a target character string through a message digest algorithm, and a length of the message digest value is smaller than a length of the target character string;
a second obtaining unit 720, configured to obtain a target character string corresponding to the message digest value from a backup database, where a correspondence relationship between the target character string and the message digest value is stored in the backup database;
a replacing unit 730, configured to replace the message digest value with the corresponding target character string, so as to obtain an original file of the compressed file.
It can be understood that the functions of the functional units in the terminal 700 in the embodiment of the present invention may be specifically implemented according to the method in the foregoing method embodiment, and the specific implementation process may refer to the related description in the foregoing method embodiment of fig. 5, which is not described herein again.
Please refer to fig. 8, fig. 8 is a schematic structural diagram of a terminal according to an embodiment of the present invention. The terminal of the embodiment includes: at least one processor 801, a communication interface 802, a user interface 803, and a memory 804, wherein the processor 801, the communication interface 802, the user interface 803, and the memory 804 may be connected by a bus or by other means, and the embodiment of the present invention is exemplified by being connected by the bus 805. Wherein,
processor 801 may be a general-purpose processor, such as a Central Processing Unit (CPU).
The communication interface 802 may be a wired interface (e.g., an ethernet interface) or a wireless interface (e.g., a cellular network interface or using a wireless local area network interface) for communicating with other terminals or websites.
The user interface 803 may be specifically a touch panel, including a touch screen and a touch screen, for detecting an operation instruction on the touch panel, and the user interface 803 may also be a physical button or a mouse. The user interface 803 may also be a display screen for outputting, displaying images or data.
The Memory 804 may include Volatile Memory (Volatile Memory), such as Random Access Memory (RAM); the Memory may also include a Non-volatile Memory (Non-volatile Memory), such as a Read-Only Memory (ROM), a Flash Memory (Flash Memory), a Hard Disk (Hard Disk Drive, HDD), or a Solid-State Drive (SSD); the memory 804 may also comprise a combination of the above-described types of memory. The memory 804 is used for storing a set of program codes, and the processor 801 is used for calling the program codes stored in the memory 804 and executing the following operations:
acquiring a target character string in a target file, wherein the size of the target character string is larger than a preset threshold value;
detecting whether a first message digest value corresponding to the target character string exists in a backup database;
if not, compressing the target character string through a message digest algorithm to obtain a second message digest value, and storing the corresponding relation between the target character string and the second message digest value in the backup database, wherein the length of the second message digest value is smaller than that of the target character string;
and replacing the target character string with the second message digest value to obtain a compressed file of the target file.
Optionally, after obtaining the target character string in the target file, before compressing the target character string by using a message digest algorithm to obtain a second message digest value, the method further includes:
detecting the repeated times of the target character string in the target file;
compressing the target character string by using a message digest algorithm to obtain the second message digest value specifically comprises: and under the condition that the repetition times of the target character string are greater than the number threshold, compressing the target character string through a message digest algorithm to obtain the second message digest value.
Optionally, if there is the first message digest value, the first message digest value corresponding to the target character string is obtained from the backup database, and the target character string is replaced with the corresponding first message digest value, so as to obtain a compressed file of the target file.
Optionally, the message digest algorithm includes: any one of MD5, sha-1, RIPEMD and Haval.
Referring to fig. 9, fig. 9 is a schematic structural diagram of a terminal according to an embodiment of the present invention. The terminal of the embodiment includes: at least one processor 901, a communication interface 902, a user interface 903 and a memory 904, wherein the processor 901, the communication interface 902, the user interface 903 and the memory 904 may be connected through a bus or in other ways, and the embodiment of the present invention is exemplified by being connected through the bus 905. Wherein,
processor 901 may be a general-purpose processor, such as a Central Processing Unit (CPU).
The communication interface 902 may be a wired interface (e.g., an ethernet interface) or a wireless interface (e.g., a cellular network interface or using a wireless local area network interface) for communicating with other terminals or websites.
The user interface 903 may be specifically a touch panel, including a touch screen and a touch screen, for detecting an operation instruction on the touch panel, and the user interface 903 may also be a physical button or a mouse. The user interface 903 may also be a display screen for outputting, displaying images or data.
Memory 904 may include Volatile Memory (Volatile Memory), such as Random Access Memory (RAM); the Memory may also include a Non-volatile Memory (Non-volatile Memory), such as a Read-Only Memory (ROM), a Flash Memory (Flash Memory), a Hard Disk (Hard Disk Drive, HDD), or a Solid-State Drive (SSD); the memory 904 may also comprise a combination of the above-described types of memory. The memory 904 is used for storing a set of program codes, and the processor 901 is used for calling the program codes stored in the memory 904 and executing the following operations:
acquiring a message digest value in a compressed file, wherein the message digest value is obtained by compressing a target character string through a message digest algorithm, and the length of the message digest value is smaller than that of the target character string;
acquiring a target character string corresponding to the message digest value from a backup database, wherein the corresponding relation between the target character string and the message digest value is stored in the backup database;
and replacing the message digest value with the corresponding target character string to obtain an original file of the compressed file.
Those of ordinary skill in the art will appreciate that the elements and algorithm steps of the examples described in connection with the embodiments disclosed herein may be embodied in electronic hardware, computer software, or combinations of both, and that the components and steps of the examples have been described in a functional general in the foregoing description for the purpose of illustrating clearly the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the terminal and the unit described above may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the several embodiments provided in the present application, it should be understood that the disclosed terminal and method can be implemented in other manners. For example, the above-described terminal embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may also be an electric, mechanical or other form of connection.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment of the present invention.
The steps in the method of the embodiment of the invention can be sequentially adjusted, combined and deleted according to actual needs.
The units in the terminal of the embodiment of the invention can be merged, divided and deleted according to actual needs.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention essentially or partially contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
While the invention has been described with reference to specific embodiments, the invention is not limited thereto, and various equivalent modifications and substitutions can be easily made by those skilled in the art within the technical scope of the invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (10)
1. A method of compression, comprising:
acquiring a target character string in a target file, wherein the size of the target character string is larger than a preset threshold value;
detecting whether a first message digest value corresponding to the target character string exists in a backup database;
if not, compressing the target character string through a message digest algorithm to obtain a second message digest value, and storing the corresponding relation between the target character string and the second message digest value in the backup database, wherein the length of the second message digest value is smaller than that of the target character string;
and replacing the target character string with the second message digest value to obtain a compressed file of the target file.
2. The method of claim 1, after obtaining the target string in the target file, before compressing the target string by a message digest algorithm to obtain a second message digest value, further comprising:
detecting the repeated times of the target character string in the target file;
compressing the target character string by using a message digest algorithm to obtain the second message digest value specifically comprises: and under the condition that the repetition times of the target character string are greater than the number threshold, compressing the target character string through a message digest algorithm to obtain the second message digest value.
3. The method of claim 1, wherein if the first message digest value is present, retrieving the first message digest value corresponding to the target string from the backup database, and replacing the target string with the corresponding first message digest value to obtain the compressed file of the target file.
4. The method of claim 1, wherein the message digest algorithm comprises: any one of MD5, sha-1, RIPEMD and Haval.
5. A reduction method, comprising:
acquiring a message digest value in a compressed file, wherein the message digest value is obtained by compressing a target character string through a message digest algorithm, and the length of the message digest value is smaller than that of the target character string;
acquiring a target character string corresponding to the message digest value from a backup database, wherein the corresponding relation between the target character string and the message digest value is stored in the backup database;
and replacing the message digest value with the corresponding target character string to obtain an original file of the compressed file.
6. A terminal, comprising:
the device comprises a first acquisition unit, a second acquisition unit and a third acquisition unit, wherein the first acquisition unit is used for acquiring a target character string in a target file, and the size of the target character string is larger than a preset threshold value;
the first detection unit is used for detecting whether a first message digest value corresponding to the target character string exists in a backup database or not;
the compressing unit is used for compressing the target character string through a message digest algorithm to obtain a second message digest value, wherein the length of the second message digest value is smaller than that of the target character string;
the storage unit is used for storing the corresponding relation between the target character string and the second message abstract value into the backup database;
and the replacing unit is used for replacing the target character string with the second message digest value so as to obtain a compressed file of the target file.
7. The terminal of claim 5, further comprising:
a second detecting unit, configured to detect a number of times of repetition of the target character string in the target file;
the compressing unit is specifically configured to, when the repetition number of the target character string is greater than a number threshold, compress the target character string by using a message digest algorithm to obtain the second message digest value.
8. The terminal of claim 5, further comprising:
a second obtaining unit, configured to obtain the first message digest value corresponding to the target character string from the backup database;
the replacing unit is specifically configured to replace the target character string with the corresponding first message digest value to obtain a compressed file of the target file.
9. The terminal of claim 6, wherein the message digest algorithm comprises: any one of MD5, sha-1, RIPEMD and Haval.
10. A terminal, comprising:
the device comprises a first acquisition unit, a second acquisition unit and a third acquisition unit, wherein the first acquisition unit is used for acquiring a message digest value in a compressed file, the message digest value is obtained by compressing a target character string through a message digest algorithm, and the length of the message digest value is smaller than that of the target character string;
a second obtaining unit, configured to obtain a target character string corresponding to the message digest value from a backup database, where a correspondence between the target character string and the message digest value is stored in the backup database;
and the replacing unit is used for replacing the message digest value with the corresponding target character string so as to obtain the original file of the compressed file.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611125510.7A CN106844091A (en) | 2016-12-08 | 2016-12-08 | One kind compression, restoring method and terminal |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611125510.7A CN106844091A (en) | 2016-12-08 | 2016-12-08 | One kind compression, restoring method and terminal |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN106844091A true CN106844091A (en) | 2017-06-13 |
Family
ID=59139168
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611125510.7A Withdrawn CN106844091A (en) | 2016-12-08 | 2016-12-08 | One kind compression, restoring method and terminal |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106844091A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111683036A (en) * | 2020-02-29 | 2020-09-18 | 新华三信息安全技术有限公司 | Data storage method and device and message identification method and device |
| CN113220651A (en) * | 2021-04-25 | 2021-08-06 | 暨南大学 | Operation data compression method and device, terminal equipment and storage medium |
| CN113688108A (en) * | 2021-07-16 | 2021-11-23 | 深圳智慧林网络科技有限公司 | Data processing method and related equipment |
| CN114579571A (en) * | 2022-03-01 | 2022-06-03 | 珠海金山数字网络科技有限公司 | Data processing method and device |
| CN118660085A (en) * | 2024-08-19 | 2024-09-17 | 杭州浩联智能科技有限公司 | Communication parameter transmission method, device, equipment and storage medium |
-
2016
- 2016-12-08 CN CN201611125510.7A patent/CN106844091A/en not_active Withdrawn
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111683036A (en) * | 2020-02-29 | 2020-09-18 | 新华三信息安全技术有限公司 | Data storage method and device and message identification method and device |
| CN111683036B (en) * | 2020-02-29 | 2022-05-27 | 新华三信息安全技术有限公司 | Data storage method and device and message identification method and device |
| CN113220651A (en) * | 2021-04-25 | 2021-08-06 | 暨南大学 | Operation data compression method and device, terminal equipment and storage medium |
| CN113220651B (en) * | 2021-04-25 | 2024-02-09 | 暨南大学 | Method, device, terminal equipment and storage medium for compressing operation data |
| CN113688108A (en) * | 2021-07-16 | 2021-11-23 | 深圳智慧林网络科技有限公司 | Data processing method and related equipment |
| CN114579571A (en) * | 2022-03-01 | 2022-06-03 | 珠海金山数字网络科技有限公司 | Data processing method and device |
| CN118660085A (en) * | 2024-08-19 | 2024-09-17 | 杭州浩联智能科技有限公司 | Communication parameter transmission method, device, equipment and storage medium |
| CN118660085B (en) * | 2024-08-19 | 2024-11-15 | 杭州浩联智能科技有限公司 | Communication parameter transmission method, device, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106844091A (en) | One kind compression, restoring method and terminal | |
| US9935973B2 (en) | Systems and methods for automatic detection of malicious activity via common files | |
| US10083299B2 (en) | Systems and methods for automatic snapshotting of backups based on malicious modification detection | |
| CN107229420B (en) | Data storage method, reading method, deleting method and data operating system | |
| KR101456757B1 (en) | Method and Apparatus for recovering deleted data for SQLite database | |
| CN111008034B (en) | Patch generation method and device | |
| CN110399546B (en) | Link duplicate removal method, device, equipment and storage medium based on web crawler | |
| JP2020047107A (en) | Data deduplication apparatus, data deduplication method, and data deduplication program | |
| CN106503165A (en) | Compression, decompressing method, device and equipment | |
| WO2014094479A1 (en) | Method and device for deleting duplicate data | |
| US9679007B1 (en) | Techniques for managing references to containers | |
| US20150007328A1 (en) | Method and System for Quickly Scanning Files | |
| JP6170900B2 (en) | File processing method and apparatus | |
| CN105095367A (en) | Method and device for acquiring client data | |
| Li et al. | Database management strategy and recovery methods of Android | |
| CN112181769A (en) | Application abnormal data collection method, device, computer equipment and storage medium | |
| CN113468118B (en) | File increment storage method, device and storage medium based on blockchain | |
| CN111061693B (en) | Method, device and medium for decompressing, separating and compressing file during downloading | |
| CN109582501B (en) | File recovery method and device, computer equipment and storage medium | |
| CN114020522A (en) | Data backup method and device, electronic equipment and system | |
| US20170103123A1 (en) | Index creating device, index creating method, search device, search method, and computer-readable recording medium | |
| CN107436836A (en) | The processing method and relevant device of a kind of journal file | |
| Satrya et al. | A novel Android memory forensics for discovering remnant data | |
| KR101111400B1 (en) | Apparatus and method for recovering the data of embeded system | |
| CN109614370B (en) | Disk file scanning method and device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20170613 |