CN112600938B - Enterprise communication data compression system - Google Patents
Enterprise communication data compression system Download PDFInfo
- Publication number
- CN112600938B CN112600938B CN202011608497.7A CN202011608497A CN112600938B CN 112600938 B CN112600938 B CN 112600938B CN 202011608497 A CN202011608497 A CN 202011608497A CN 112600938 B CN112600938 B CN 112600938B
- Authority
- CN
- China
- Prior art keywords
- module
- compression
- storage
- communication data
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000004891 communication Methods 0.000 title claims abstract description 149
- 238000013144 data compression Methods 0.000 title claims abstract description 33
- 238000007906 compression Methods 0.000 claims abstract description 175
- 230000006835 compression Effects 0.000 claims abstract description 173
- 238000013500 data storage Methods 0.000 claims abstract description 82
- 230000010365 information processing Effects 0.000 claims abstract description 71
- 230000007774 longterm Effects 0.000 claims abstract description 28
- 238000004364 calculation method Methods 0.000 claims description 37
- 238000000034 method Methods 0.000 claims description 14
- 238000009825 accumulation Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/565—Conversion or adaptation of application format or content
- H04L67/5651—Reducing the amount or size of exchanged application data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/174—Redundancy elimination performed by the file system
- G06F16/1744—Redundancy elimination performed by the file system using compression, e.g. sparse files
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Telephonic Communication Services (AREA)
- Information Transfer Between Computers (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention discloses an enterprise communication data compression system, relates to a data compression system, and belongs to the technical field of data compression; the system comprises an information acquisition module, an information processing module, a compression distribution module, a storage distribution module, a history recording module, a server, a data storage module and a data sharing module; the server is an enterprise internal communication server, the information acquisition module is directly connected with the server and used for acquiring communication data information in an enterprise, the information processing module is used for processing the communication data information acquired by the information acquisition module, and the information processing module outputs the processed result to the compression distribution module and the storage distribution module; the storage allocation module is used for performing storage allocation according to the data processing result; the data storage module is divided into a temporary storage unit and a long-term storage unit, and the data calling speed of the temporary storage unit is greater than that of the long-term storage unit.
Description
Technical Field
The invention relates to a data compression system, in particular to an enterprise communication data compression system, and belongs to the technical field of data compression.
Background
Data compression is a technical method for reducing the data volume to reduce the storage space and improve the transmission, storage and processing efficiency of the data on the premise of not losing useful information, or for reorganizing the data according to a certain algorithm and reducing the redundancy and storage space of the data. Data compression includes lossy compression and lossless compression. In computer science and information theory, data compression or source coding is the process of representing information with fewer bits of data than uncoded according to a particular coding scheme.
Most of the existing data compression is autonomous data compression, and whether compression is performed or not and the compression mode is not selected according to the stored data.
In order to solve the above technical problems, the present invention provides the following technical solutions.
Disclosure of Invention
The invention aims to provide an enterprise communication data compression system, which is used for solving the problem that whether compression is carried out or not and the compression mode is not selected according to the judgment of the stored data per se in the prior art. According to the invention, through the cooperation of the information acquisition module, the information processing module, the compression distribution module, the storage distribution module, the history recording module, the server, the data storage module and the data sharing module, the communication data file is judged according to the compression mode and the storage position, so that the problem of large occupied space for the accumulation and storage of the communication data file is avoided.
The purpose of the invention can be realized by the following technical scheme: an enterprise communication data compression system comprises an information acquisition module, an information processing module, a compression distribution module, a storage distribution module, a history recording module, a server, a data storage module and a data sharing module; the server is an enterprise internal communication server, the information acquisition module is directly connected with the server and used for acquiring communication data information in an enterprise, the information processing module is used for processing the communication data information acquired by the information acquisition module, the information processing module outputs the processed result to the compression distribution module and the storage distribution module, the compression distribution module is used for adopting different data compression modes according to the data processing result, and the compression modes comprise lossy compression and lossless compression; the storage allocation module is used for performing storage allocation according to the data processing result; the data storage module is divided into a temporary storage unit and a long-term storage unit, and the data calling speed of the temporary storage unit is greater than that of the long-term storage unit;
the history recording module is used for recording compression and storage records of communication data information and sharing the communication data information in the server through the data sharing module;
the information processing module is used for receiving and processing the data information acquired by the information acquisition module and outputting the processed result to the compression distribution module and the storage distribution module, and the specific information processing process comprises the following steps:
the method comprises the following steps: the communication data information is acquired through the information acquisition module, and the acquired communication data information comprises data storage time, data storage size, calling times after data storage and data importance; sending the acquired communication data information to an information processing module; and obtaining the current time T;
step two: after receiving the communication data information, the information processing module marks the communication data information as Ti, Di, Ci and Zi respectively; wherein i represents a communication data file number, i is 1, …, n;
step three: the information processing module calls a calculation formula for calculating the compression coefficient Kyi from the data storage module, wherein the calculation formula is
Wherein alpha, beta and gamma are preset proportionality coefficient values;
step four: the information processing module sets a compression coefficient threshold, when the compression coefficient Kyi is greater than or equal to the compression coefficient threshold, the communication data file i is compressed, and a compression signal is sent to the compression distribution module; if the compression coefficient Kyi is smaller than the compression coefficient threshold value, it indicates that the communication data file i is not compressed, and step five is executed;
step five: the information processing module calls a calculation formula for calculating the storage coefficient Cci from the data storage module, wherein the calculation formula is
Wherein a1, a2 and a3 are fixed values of preset proportionality coefficients;
step six: setting a storage coefficient threshold, if the storage coefficient Cci is larger than or equal to the storage coefficient threshold, sending a temporary storage signal to a data storage module, and after receiving the signal, the data storage module stores a communication data file i in a temporary storage unit; and if the storage coefficient Cci is smaller than the storage coefficient threshold value, sending a long-term storage signal to the data storage module, and storing the communication data file i in a long-term storage unit after the data storage module receives the signal.
Preferably, the obtaining mode of the importance Zi in the step one is the following process:
the method comprises the steps of obtaining keywords appearing in enterprise communication, storing the keywords in a data storage module, obtaining the number of different keywords appearing in a communication data file i, recording the keywords as one time when the keywords are the same, and marking the number of times of the keywords appearing as importance Zi.
Preferably, the compression distribution module is configured to perform data compression on the communication data file, and the compression distribution module includes a lossy compression unit and a lossless compression unit, and the specific distribution manner includes the following:
when the compression distribution module receives the compression signal sent by the information processing module, the compression distribution module sends a data acquisition signal to the information acquisition module;
the information acquisition module acquires a keyword j appearing in the communication data file i and the occurrence frequency Kj of the keyword j; and when the K is less than the keyword threshold, the compression distribution module performs lossy compression on the communication data file i and distributes the communication data file i to a lossy compression unit for lossy compression.
Preferably, the history recording module is configured to record whether the communication data file i is compressed or not and a storage location, record the compression time, the compressed size, and the compressed storage location of the communication data file i in a history recording table, and log enterprise employees in the enterprise communication data compression system to perform history recording query.
Preferably, the data storage module is divided into a temporary storage unit and a long-term storage unit according to the storage duration, and the data calling speed of the temporary storage unit is greater than that of the long-term storage unit; the data storage module is also used for storing the communication data information acquired by the information acquisition module and the data processed by the information processing module, storing a calculation formula for calculating the storage coefficient Cci and a calculation formula for calculating the compression coefficient Kyi, and after receiving the call of the information processing module, using the calculation formulas
And
and sending the information to an information processing module.
Preferably, the compression process of the enterprise communication data compression system comprises the following steps:
step S1: acquiring data storage time, data storage size, calling times after data storage and data importance by an information acquisition module; sending the acquired communication data information to an information processing module; and obtaining the current time T; after receiving the communication data information, the information processing module marks the communication data information as Ti, Di, Ci and Zi respectively; wherein i represents a communication data file number, i is 1, …, n;
step S2: the information processing module calls a calculation formula for calculating the compression coefficient Kyi from the data storage module, wherein the calculation formula is
The information processing module sets a compression coefficient threshold, when the compression coefficient Kyi is greater than or equal to the compression coefficient threshold, the communication data file i is compressed, and a compression signal is sent to the compression distribution module; if the compression coefficient Kyi is smaller than the compression coefficient threshold, the communication data file i is not compressed, and the next step is executed;
step S21: when the compression distribution module receives the compression signal sent by the information processing module, the compression distribution module sends a data acquisition signal to the information acquisition module;
step S22: the information acquisition module acquires a keyword j appearing in the communication data file i and the occurrence frequency Kj of the keyword j; calculating the total occurrence frequency K of the summation keywords;
step S23: setting a keyword threshold, when K is greater than or equal to the keyword threshold, enabling the compression distribution module to perform lossless compression on the communication data file i, distributing the communication data file i to a lossless compression unit for performing lossless compression, when K is less than the keyword threshold, enabling the compression distribution module to perform lossy compression on the communication data file i, and distributing the communication data file i to a lossy compression unit for performing lossy compression;
step S3: the information processing module calls a calculation formula for calculating the storage coefficient Cci from the data storage module, wherein the calculation formula is
Wherein a1, a2 and a3 are fixed values of preset proportionality coefficients;
step S4: setting a storage coefficient threshold, if the storage coefficient Cci is larger than or equal to the storage coefficient threshold, sending a temporary storage signal to a data storage module, and after receiving the signal, the data storage module stores a communication data file i in a temporary storage unit; and if the storage coefficient Cci is smaller than the storage coefficient threshold value, sending a long-term storage signal to the data storage module, and storing the communication data file i in a long-term storage unit after the data storage module receives the signal.
Compared with the prior art, the invention has the beneficial effects that:
1. the invention is provided with an information processing module, wherein the information processing module is used for judging a compression mode and a storage position, and collecting communication data information through an information collecting module, and the collected communication data information comprises data storage time, data storage size, calling times after data storage and data importance; sending the acquired communication data information to an information processing module; and obtaining the current time T; after receiving the communication data information, the information processing module marks the communication data information as Ti, Di, Ci and Zi respectively; wherein i represents a communication data file number, i is 1, …, n; the information processing module calls a calculation formula for calculating the compression coefficient Kyi from the data storage module, wherein the calculation formula is
Wherein alpha, beta and gamma are preset proportionality coefficient values; the information processing module sets a compression coefficient threshold, when the compression coefficient Kyi is greater than or equal to the compression coefficient threshold, the communication data file i is compressed, and a compression signal is sent to the compression distribution module; if the compression coefficient Kyi is smaller than the compression coefficient threshold, the communication data file i is not compressed, and the next step is executed; the information processing module calls a calculation formula for calculating the storage coefficient Cci from the data storage module, wherein the calculation formula is
Wherein a1, a2 and a3 are fixed values of preset proportionality coefficients; setting upIf the storage coefficient Cci is larger than or equal to the storage coefficient threshold, sending a temporary storage signal to the data storage module, and storing the communication data file i in a temporary storage unit after the data storage module receives the signal; and if the storage coefficient Cci is smaller than the storage coefficient threshold value, sending a long-term storage signal to the data storage module, and storing the communication data file i in a long-term storage unit after the data storage module receives the signal.
2. The importance Zi is obtained by the following steps: acquiring keywords appearing in enterprise communication, storing the keywords in a data storage module, acquiring the number of different keywords appearing in a communication data file i, wherein the same keywords are marked as one time, and the number of times of the keywords appearing is marked as importance Zi; the compression distribution module is used for performing data compression on the communication data file, the compression distribution module comprises a lossy compression unit and a lossless compression unit, and the specific distribution mode comprises the following steps: when the compression distribution module receives the compression signal sent by the information processing module, the compression distribution module sends a data acquisition signal to the information acquisition module; the information acquisition module acquires a keyword j appearing in the communication data file i and the occurrence frequency Kj of the keyword j; and when the K is less than the keyword threshold, the compression distribution module performs lossy compression on the communication data file i and distributes the communication data file i to a lossy compression unit for lossy compression.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a schematic block diagram of the present invention.
Detailed Description
The technical solutions of the present invention will be described clearly and completely with reference to the following embodiments, and it should be understood that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, an enterprise communication data compression system includes an information acquisition module, an information processing module, a compression distribution module, a storage distribution module, a history recording module, a server, a data storage module, and a data sharing module; the server is an enterprise internal communication server, the information acquisition module is directly connected with the server and used for acquiring communication data information in an enterprise, the information processing module is used for processing the communication data information acquired by the information acquisition module, the information processing module outputs the processed result to the compression distribution module and the storage distribution module, the compression distribution module is used for adopting different data compression modes according to the data processing result, and the compression modes comprise lossy compression and lossless compression; the storage allocation module is used for performing storage allocation according to the data processing result; the data storage module is divided into a temporary storage unit and a long-term storage unit, and the data calling speed of the temporary storage unit is greater than that of the long-term storage unit;
the history recording module is used for recording compression and storage records of communication data information and sharing the communication data information in the server through the data sharing module;
the information processing module is used for receiving and processing the data information acquired by the information acquisition module and outputting the processed result to the compression distribution module and the storage distribution module, and the specific information processing process comprises the following steps:
the method comprises the following steps: the communication data information is acquired through the information acquisition module, and the acquired communication data information comprises data storage time, data storage size, calling times after data storage and data importance; sending the acquired communication data information to an information processing module; and obtaining the current time T;
step two: after receiving the communication data information, the information processing module marks the communication data information as Ti, Di, Ci and Zi respectively; wherein i represents a communication data file number, i is 1, …, n;
step three: the information processing module calls a calculation formula for calculating the compression coefficient Kyi from the data storage module, wherein the calculation formula is
Wherein alpha, beta and gamma are preset proportionality coefficient values;
step four: the information processing module sets a compression coefficient threshold, when the compression coefficient Kyi is greater than or equal to the compression coefficient threshold, the communication data file i is compressed, and a compression signal is sent to the compression distribution module; if the compression coefficient Kyi is smaller than the compression coefficient threshold value, it indicates that the communication data file i is not compressed, and step five is executed;
step five: the information processing module calls a calculation formula for calculating the storage coefficient Cci from the data storage module, wherein the calculation formula is
Wherein a1, a2 and a3 are fixed values of preset proportionality coefficients;
step six: setting a storage coefficient threshold, if the storage coefficient Cci is larger than or equal to the storage coefficient threshold, sending a temporary storage signal to a data storage module, and after receiving the signal, the data storage module stores a communication data file i in a temporary storage unit; and if the storage coefficient Cci is smaller than the storage coefficient threshold value, sending a long-term storage signal to the data storage module, and storing the communication data file i in a long-term storage unit after the data storage module receives the signal.
The method for acquiring the importance Zi in the step one comprises the following steps:
the method comprises the steps of obtaining keywords appearing in enterprise communication, storing the keywords in a data storage module, obtaining the number of different keywords appearing in a communication data file i, recording the keywords as one time when the keywords are the same, and marking the number of times of the keywords appearing as importance Zi.
The compression distribution module is used for performing data compression on the communication data file, the compression distribution module comprises a lossy compression unit and a lossless compression unit, and the specific distribution mode comprises the following steps:
when the compression distribution module receives the compression signal sent by the information processing module, the compression distribution module sends a data acquisition signal to the information acquisition module;
the information acquisition module acquires a keyword j appearing in the communication data file i and the occurrence frequency Kj of the keyword j; and when the K is less than the keyword threshold, the compression distribution module performs lossy compression on the communication data file i and distributes the communication data file i to a lossy compression unit for lossy compression.
The history recording module is used for recording whether the communication data file i is compressed or not and the storage position, recording the compression time, the compressed size and the compressed storage position of the communication data file i in a history recording table, and logging enterprise staff in an enterprise communication data compression system to inquire the history recording.
The data storage module is divided into a temporary storage unit and a long-term storage unit according to storage duration, and the data calling speed of the temporary storage unit is greater than that of the long-term storage unit; the data storage module is also used for storing the communication data information acquired by the information acquisition module and the data processed by the information processing module, storing a calculation formula for calculating the storage coefficient Cci and a calculation formula for calculating the compression coefficient Kyi, and after receiving the call of the information processing module, using the calculation formulas
And
and sending the information to an information processing module.
The compression process of the enterprise communication data compression system comprises the following steps:
step S1: acquiring data storage time, data storage size, calling times after data storage and data importance by an information acquisition module; sending the acquired communication data information to an information processing module; and obtaining the current time T; after receiving the communication data information, the information processing module marks the communication data information as Ti, Di, Ci and Zi respectively; wherein i represents a communication data file number, i is 1, …, n;
step S2: the information processing module calls a calculation formula for calculating the compression coefficient Kyi from the data storage module, wherein the calculation formula is
The information processing module sets a compression coefficient threshold, when the compression coefficient Kyi is greater than or equal to the compression coefficient threshold, the communication data file i is compressed, and a compression signal is sent to the compression distribution module; if the compression coefficient Kyi is smaller than the compression coefficient threshold, the communication data file i is not compressed, and the next step is executed;
step S21: when the compression distribution module receives the compression signal sent by the information processing module, the compression distribution module sends a data acquisition signal to the information acquisition module;
step S22: the information acquisition module acquires a keyword j appearing in the communication data file i and the occurrence frequency Kj of the keyword j; calculating the total occurrence frequency K of the summation keywords;
step S23: setting a keyword threshold, when K is greater than or equal to the keyword threshold, enabling the compression distribution module to perform lossless compression on the communication data file i, distributing the communication data file i to a lossless compression unit for performing lossless compression, when K is less than the keyword threshold, enabling the compression distribution module to perform lossy compression on the communication data file i, and distributing the communication data file i to a lossy compression unit for performing lossy compression;
step S3: the information processing module calls a calculation formula for calculating the storage coefficient Cci from the data storage module, wherein the calculation formula is
Wherein a1, a2 and a3 are fixed values of preset proportionality coefficients;
step S4: setting a storage coefficient threshold, if the storage coefficient Cci is larger than or equal to the storage coefficient threshold, sending a temporary storage signal to a data storage module, and after receiving the signal, the data storage module stores a communication data file i in a temporary storage unit; and if the storage coefficient Cci is smaller than the storage coefficient threshold value, sending a long-term storage signal to the data storage module, and storing the communication data file i in a long-term storage unit after the data storage module receives the signal.
The above formulas are all calculated by taking the numerical value of the dimension, the formula is a formula which obtains the latest real situation by acquiring a large amount of data and performing software simulation, and the preset parameters in the formula are set by the technical personnel in the field according to the actual situation.
The working principle of the invention comprises the following processes: acquiring data storage time, data storage size, calling times after data storage and data importance by an information acquisition module; sending the acquired communication data information to an information processing module; and obtaining the current time T; after receiving the communication data information, the information processing module marks the communication data information as Ti, Di, Ci and Zi respectively; wherein i represents a communication data file number, i is 1, …, n; the information processing module calls a calculation formula for calculating the compression coefficient Kyi from the data storage module, wherein the calculation formula is
The information processing module sets a compression coefficient threshold, when the compression coefficient Kyi is greater than or equal to the compression coefficient threshold, the communication data file i is compressed, and a compression signal is sent to the compression distribution module; if the compression coefficient Kyi is smaller than the compression coefficient threshold, the communication data file i is not compressed, and the next step is executed; when the compressed distribution module receives the informationAfter the compression signal is sent by the processing module, the compression distribution module sends a data acquisition signal to the information acquisition module; the information acquisition module acquires a keyword j appearing in the communication data file i and the occurrence frequency Kj of the keyword j; calculating the total occurrence frequency K of the summation keywords; setting a keyword threshold, when K is greater than or equal to the keyword threshold, enabling the compression distribution module to perform lossless compression on the communication data file i, distributing the communication data file i to a lossless compression unit for performing lossless compression, when K is less than the keyword threshold, enabling the compression distribution module to perform lossy compression on the communication data file i, and distributing the communication data file i to a lossy compression unit for performing lossy compression;
the information processing module calls a calculation formula for calculating the storage coefficient Cci from the data storage module, wherein the calculation formula is
Wherein a1, a2 and a3 are fixed values of preset proportionality coefficients;
setting a storage coefficient threshold, if the storage coefficient Cci is larger than or equal to the storage coefficient threshold, sending a temporary storage signal to a data storage module, and after receiving the signal, the data storage module stores a communication data file i in a temporary storage unit; and if the storage coefficient Cci is smaller than the storage coefficient threshold value, sending a long-term storage signal to the data storage module, and storing the communication data file i in a long-term storage unit after the data storage module receives the signal.
In the description herein, references to the description of "one embodiment," "an example," "a specific example" or the like are intended to mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
The preferred embodiments of the invention disclosed above are intended to be illustrative only. The preferred embodiments are not intended to be exhaustive or to limit the invention to the precise forms disclosed. Obviously, many modifications and variations are possible in light of the above teaching. The embodiments were chosen and described in order to best explain the principles of the invention and the practical application, to thereby enable others skilled in the art to best utilize the invention. The invention is limited only by the claims and their full scope and equivalents.
Claims (5)
1. An enterprise communication data compression system is characterized by comprising an information acquisition module, an information processing module, a compression distribution module, a storage distribution module, a history recording module, a server, a data storage module and a data sharing module; the server is an enterprise internal communication server, the information acquisition module is directly connected with the server and used for acquiring communication data information in an enterprise, the information processing module is used for processing the communication data information acquired by the information acquisition module, the information processing module outputs the processed result to the compression distribution module and the storage distribution module, the compression distribution module is used for adopting different data compression modes according to the data processing result, and the compression modes comprise lossy compression and lossless compression; the storage allocation module is used for performing storage allocation according to the data processing result; the data storage module is divided into a temporary storage unit and a long-term storage unit, and the data calling speed of the temporary storage unit is greater than that of the long-term storage unit;
the history recording module is used for recording compression and storage records of communication data information and sharing the communication data information in the server through the data sharing module;
the information processing module is used for receiving and processing the data information acquired by the information acquisition module and outputting the processed result to the compression distribution module and the storage distribution module, and the specific information processing process comprises the following steps:
the method comprises the following steps: the communication data information is acquired through the information acquisition module, and the acquired communication data information comprises data storage time, data storage size, calling times after data storage and data importance; sending the acquired communication data information to an information processing module; and obtaining the current time T;
step two: after receiving the communication data information, the information processing module marks the communication data information as Ti, Di, Ci and Zi respectively; wherein i represents a communication data file number, i is 1, …, n;
step three: the information processing module calls a calculation formula for calculating the compression coefficient Kyi from the data storage module, wherein the calculation formula is
Wherein alpha, beta and gamma are preset proportionality coefficient values;
step four: the information processing module sets a compression coefficient threshold, when the compression coefficient Kyi is greater than or equal to the compression coefficient threshold, the communication data file i is compressed, and a compression signal is sent to the compression distribution module; if the compression coefficient Kyi is smaller than the compression coefficient threshold value, it indicates that the communication data file i is not compressed, and step five is executed;
step five: the information processing module calls a calculation formula for calculating the storage coefficient Cci from the data storage module, wherein the calculation formula is
Wherein a1, a2 and a3 are fixed values of preset proportionality coefficients;
step six: setting a storage coefficient threshold, if the storage coefficient Cci is larger than or equal to the storage coefficient threshold, sending a temporary storage signal to a data storage module, and after receiving the signal, the data storage module stores a communication data file i in a temporary storage unit; and if the storage coefficient Cci is smaller than the storage coefficient threshold value, sending a long-term storage signal to the data storage module, and storing the communication data file i in a long-term storage unit after the data storage module receives the signal.
2. An enterprise communication data compression system according to claim 1, further comprising: the acquisition mode of the importance Zi in the step one is as follows:
the method comprises the steps of obtaining keywords appearing in enterprise communication, storing the keywords in a data storage module, obtaining the number of different keywords appearing in a communication data file i, recording the keywords as one time when the keywords are the same, and marking the number of times of the keywords appearing as importance Zi.
3. An enterprise communication data compression system according to claim 1, further comprising: the compression distribution module is used for performing data compression on the communication data file, the compression distribution module comprises a lossy compression unit and a lossless compression unit, and the specific distribution mode comprises the following steps:
when the compression distribution module receives the compression signal sent by the information processing module, the compression distribution module sends a data acquisition signal to the information acquisition module;
the information acquisition module acquires a keyword j appearing in the communication data file i and the occurrence frequency Kj of the keyword j; and when the K is less than the keyword threshold, the compression distribution module performs lossy compression on the communication data file i and distributes the communication data file i to a lossy compression unit for lossy compression.
4. An enterprise communication data compression system according to claim 1, further comprising: the history recording module is used for recording whether the communication data file i is compressed or not and the storage position, recording the compression time, the compressed size and the compressed storage position of the communication data file i in a history recording table, and logging enterprise staff in an enterprise communication data compression system to inquire the history recording.
5. An enterprise communication data compression system according to claim 1, further comprising: the data storage module is divided into a temporary storage unit and a long-term storage unit according to storage duration, and the temporary storage unitThe data calling speed of the storage unit is greater than that of the long-term storage unit; the data storage module is also used for storing the communication data information acquired by the information acquisition module and the data processed by the information processing module, storing a calculation formula for calculating the storage coefficient Cci and a calculation formula for calculating the compression coefficient Kyi, and after receiving the call of the information processing module, using the calculation formulas
And
and sending the information to an information processing module.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011608497.7A CN112600938B (en) | 2020-12-30 | 2020-12-30 | Enterprise communication data compression system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011608497.7A CN112600938B (en) | 2020-12-30 | 2020-12-30 | Enterprise communication data compression system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112600938A CN112600938A (en) | 2021-04-02 |
| CN112600938B true CN112600938B (en) | 2022-01-11 |
Family
ID=75206251
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011608497.7A Active CN112600938B (en) | 2020-12-30 | 2020-12-30 | Enterprise communication data compression system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112600938B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106372181A (en) * | 2016-08-31 | 2017-02-01 | 东北大学 | Big data compression method based on industrial process |
| CN111277274A (en) * | 2020-01-13 | 2020-06-12 | 平安国际智慧城市科技股份有限公司 | Data compression method, device, equipment and storage medium |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8194736B2 (en) * | 2008-04-15 | 2012-06-05 | Sony Corporation | Video data compression with integrated lossy and lossless compression |
| CN102437894B (en) * | 2011-11-04 | 2015-01-28 | 百度在线网络技术（北京）有限公司 | Method, device and equipment for compressing information to be sent |
| KR102332782B1 (en) * | 2014-12-15 | 2021-11-30 | 삼성전자주식회사 | Video data compression considering a vision system |
| GB2575437B (en) * | 2018-06-29 | 2022-04-27 | Imagination Tech Ltd | Guaranteed data compression |
-
2020
- 2020-12-30 CN CN202011608497.7A patent/CN112600938B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106372181A (en) * | 2016-08-31 | 2017-02-01 | 东北大学 | Big data compression method based on industrial process |
| CN111277274A (en) * | 2020-01-13 | 2020-06-12 | 平安国际智慧城市科技股份有限公司 | Data compression method, device, equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112600938A (en) | 2021-04-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101853287B (en) | Data compression quick retrieval file system and method thereof | |
| CN104090889B (en) | Data processing method and system | |
| CN105989076A (en) | Data statistical method and device | |
| CN107390650A (en) | A kind of data collecting system based on Internet of Things and the data compression method based on the system | |
| CN110781184A (en) | Data table construction method, device, equipment and storage medium | |
| CN103064933A (en) | Data query method and system | |
| CN102906751A (en) | Method and device for data storage and data query | |
| CN101876932A (en) | Method, system and device for in-memory database monitoring | |
| CN113297196A (en) | Intelligent agricultural AIOT distributed big data storage platform | |
| CN108563776A (en) | Off-line data acquisition methods and system, server and storage medium | |
| CN112600938B (en) | Enterprise communication data compression system | |
| CN102075964A (en) | Method and equipment for acquiring performance data by using network management system | |
| CN1703041A (en) | Method for parallel processing big quantities of data in a communication server | |
| CN115033764A (en) | Metadata acquisition method and device, storage medium and electronic equipment | |
| CN110570255A (en) | DMP-based real-time analysis method and system for market demand data of multi-source system | |
| CN207424932U (en) | A kind of agricultural machinery information system management platform | |
| EP1349087A3 (en) | Remote database access through a table entry | |
| CN110582091A (en) | Method and apparatus for locating wireless quality problems | |
| CN112380164B (en) | Electric power system scene data management method, device and system based on snapshot technology | |
| CN115952030A (en) | Data tracing method and system | |
| CN112214290B (en) | Log information processing method, edge node, center node and system | |
| CN115756889A (en) | Task center architecture method for double data buses of aerospace measurement and control ground station network | |
| CN112463793A (en) | EMS information display system and method based on influxdb database | |
| CN117056134B (en) | Method for quickly backing up database data in energy consumption monitoring system | |
| CN103198068A (en) | Application method of dynamic information database in distribution automation system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20211221 Address after: 518100 first floor, second floor, third floor, No. 12, Yumin Road, Shatou Industrial Zone, Shajing street, Bao'an District, Shenzhen, Guangdong Applicant after: SHENZHEN LANSHUO TELECOM EQUIPMENT CO.,LTD. Address before: 529000 room 106, No.35, North Lianghua new village, Pengjiang district, Jiangmen City, Guangdong Province Applicant before: Jiangmen guokehe Road Information Technology Co.,Ltd. |
|
| TA01 | Transfer of patent application right | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |