CN104868922A - Data compression method and device - Google Patents

Abstract

Embodiments of the present invention provide a data compression method and device, the device comprising: a splitting module, for splitting data to be compressed according to preset splitting rules, obtaining the split data and sending it to the compression module, so that the compressed The algorithm is applicable to the split data, wherein the preset split rule is determined according to the characteristics of the data to be compressed; the compression module is used to receive the split data sent by the split module, and use the compression algorithm to split Compress the final data, output the compressed data, and send it to the storage module; the storage module is used to receive the compressed data sent by the compression module, and store the compressed data. The embodiment of the present invention splits the data to be compressed according to the characteristics of the data to be compressed, so that the data to be compressed that is not suitable for using a special compression algorithm can be processed by a special compression algorithm to obtain a high compression ratio after splitting, thereby saving data storage space and reduce costs.

Description

Data compression method and device

technical field

Embodiments of the present invention relate to data processing technologies, and in particular to a data compression method and device.

Background technique

With the rapid development of science and technology, the amount of information has increased rapidly. Therefore, the storage and mining of information has also attracted widespread attention, for example, big data sought after by information technology (Information Technology, abbreviated: IT) manufacturers. If the information is stored without compression, the storage cost will be high. Therefore, the compression method has attracted much attention.

If divided according to whether the information is lossy, the compression methods include lossy compression methods and lossless compression methods. Among them, the lossless compression method means that the information can be completely restored after decompression; while the data compressed by the lossy compression method cannot be completely restored, for example, audio compression and video compression cannot be completely restored. Common lossless compression methods include dedicated compression algorithms, such as dictionary compression algorithms, equivalence distance compression algorithms, difference compression algorithms, and general bit (bit) compression algorithms, among others.

Dedicated compression algorithms usually solve the compression of a certain type of data, and the compression of some data may lead to an increase in information storage space.

Contents of the invention

Embodiments of the present invention provide a data compression method and device to solve the problem of increased information storage space caused by using a dedicated compression algorithm to compress data.

In a first aspect, an embodiment of the present invention provides a data compression device, including:

A splitting module, configured to split the data to be compressed according to preset splitting rules, obtain the split data and send it to the compression module, so that the compression algorithm is applicable to the split data, wherein the preset The splitting rule is determined according to the characteristics of the data to be compressed;

The compression module is configured to receive the split data sent by the split module, use the compression algorithm to compress the split data, output the compressed data, and send it to the storage module;

The storage module is configured to receive the compressed data sent by the compression module, and store the compressed data.

With reference to the first aspect, in a first possible implementation manner of the first aspect, the splitting module is specifically used for:

The data to be compressed is split by using a predetermined value division method, wherein the number of segments to be split is determined according to a sampling test of the data to be compressed.

With reference to the first aspect or the first possible implementation of the first aspect, in the second possible implementation of the first aspect, the compression module is specifically configured to:

receiving the split data sent by the split module, using the compression algorithm to perform at least one compression process on the split data, and outputting the compressed data, wherein the The same or different compression algorithms.

With reference to the first aspect or the first or second possible implementation manner of the first aspect, in a third possible implementation manner of the first aspect, the storage module is specifically configured to:

The compressed data is stored using a self-describing compression code to obtain coded data. The self-describing compression code includes a code length identification part and a data value code part. The number of sections, the data value coding part is used to represent the part that codes the compressed data, wherein the coded length of the compressed data is identified by its first N bits, where the size of N is equal to the In the binary representation of the above coded data, add 1 to the number of non-zero bits between the highest bit and the first bit whose value is zero from the highest bit.

With reference to the first aspect or the first or second possible implementation manner of the first aspect, in a fourth possible implementation manner of the first aspect, the device further includes:

A determination module, configured to trigger the compression module to perform compression processing on the pre-compressed data if it is determined that the size of the pre-compressed data in the compression process is applicable to the deformed bit compression algorithm;

The compression module is specifically used to compress the pre-compressed data by using the deformed bit compression algorithm to obtain the compressed data, wherein the deformed bit compression algorithm means that different bits are used to compress each data Perform compression processing.

With reference to the fourth possible implementation of the first aspect, in a fifth possible implementation of the first aspect, the determining module is specifically configured to:

Obtaining the minimum number of bytes for encoding the maximum data in the pre-compressed data;

determining a first number of bytes for encoding the pre-compressed data according to the size of the pre-compressed data and the minimum number of bytes;

If the pre-compressed data is bit-coded using a preset number of bits, then determine the second number of bytes for bit-coding the pre-compressed data;

If it is determined that the second number of bytes is smaller than the first number of bytes, then it is determined that the size of the pre-compressed data is suitable for a deformed bit compression algorithm, and the compression module is triggered to perform compression processing on the pre-compressed data.

With reference to the fourth or fifth possible implementation of the first aspect, in a sixth possible implementation of the first aspect, the storage module is used for:

One or a combination of positive number encoding and negative number encoding is used to store the compressed data to obtain encoded data, wherein the positive number encoding means to encode the part of the compressed data that uses byte compression processing, and the negative number Encoding refers to encoding a portion of the compressed data that is subjected to bit compression processing.

With reference to the sixth possible implementation manner of the first aspect, in a seventh possible implementation manner of the first aspect, the encoded data includes the first encoded data, and when the compressed data is stored using a positive number encoding, the The storage module is specifically used for:

Acquiring the minimum number of bytes for encoding the maximum data according to the maximum data in the compressed data;

If the first bit value of the first byte of the largest data is zero, it is determined to use the minimum number of bytes to encode the part of the compressed data that uses byte compression processing to obtain the first encoded data;

If the first bit value of the first byte of the maximum data is non-zero, it is determined to use the number of bytes after adding 1 to the minimum number of bytes to encode the part of the compressed data that uses byte compression processing, and obtain The first encoded data.

With reference to the sixth or seventh possible implementation manner of the first aspect, in an eighth possible implementation manner of the first aspect, the encoded data includes second encoded data, and the storage module is specifically configured to:

Negative encoding is performed on the part of the compressed data that adopts bit compression processing according to the negative encoding rule to obtain the second encoded data, the negative encoding rule includes a code length identification part and a data encoding part, and the code length identification part is used In order to represent the number of coded data, the data coding part is used to represent the coded value after the coded data is coded with bits of a preset length, wherein the length of the coded length identifier is determined by the second code The first M bits of each data in the data are identified, wherein the size of M is equal to the value between the first bit with a value of zero and the highest bit in the binary representation of each data in the second encoding The number of non-zero bits.

In a second aspect, an embodiment of the present invention provides a data compression method, including:

Split the data to be compressed according to a preset split rule to obtain the split data, so that the compression algorithm is applicable to the split data, wherein the preset split rule is based on the data to be compressed Characterized;

compressing the split data by using the compression algorithm, and outputting the compressed data;

The compressed data is stored.

In combination with the second aspect, in the first possible implementation manner of the second aspect, the splitting the data to be compressed according to the preset splitting rules, and obtaining the split data includes:

The data to be compressed is split by using a predetermined value division method, wherein the number of segments to be split is determined according to a sampling test of the data to be compressed.

With reference to the second aspect or the first possible implementation of the second aspect, in the second possible implementation of the second aspect, the compression algorithm is used to compress the split data, and the output compressed data, including:

Using the compression algorithm to perform at least one compression process on the split data to output compressed data, wherein the compression algorithm used in each compression process is the same or different.

With reference to the second aspect or the first or second possible implementation manner of the second aspect, in a third possible implementation manner of the second aspect, the storing the compressed data includes:

The compressed data is stored using a self-describing compression code to obtain coded data. The self-describing compression code includes a code length identification part and a data value code part. The number of sections, the data value coding part is used to represent the part that codes the compressed data, wherein the coded length of the compressed data is identified by its first N bits, where the size of N is equal to the In the binary representation of the above coded data, add 1 to the number of non-zero bits between the highest bit and the first bit whose value is zero from the highest bit.

In combination with the second aspect or the first or second possible implementation of the second aspect, in the fourth possible implementation of the second aspect, the compression algorithm is used to compress the split data Before processing, also include:

If it is determined that the scale of the pre-compressed data in the compression process is applicable to the deformed bit compression algorithm, then the deformed bit compression algorithm is used to compress the pre-compressed data to obtain the compressed data, wherein the deformed bit The compression algorithm means that each data is compressed using different bit numbers.

With reference to the fourth possible implementation of the second aspect, in the fifth possible implementation of the second aspect, the determination of the size of the pre-compressed data in the compression process applies a deformed bit compression algorithm, including:

Obtaining the minimum number of bytes for encoding the maximum data in the pre-compressed data;

determining a first number of bytes for encoding the pre-compressed data according to the size of the pre-compressed data and the minimum number of bytes;

If the pre-compressed data is bit-coded using a preset number of bits, then determine the second number of bytes for bit-coding the pre-compressed data;

If it is determined that the second number of bytes is smaller than the first number of bytes, then it is determined that the size of the pre-compressed data is applicable to a deformed bit compression algorithm.

With reference to the fourth or fifth possible implementation manner of the second aspect, in a sixth possible implementation manner of the second aspect, the storing the compressed data includes:

One or a combination of positive number encoding and negative number encoding is used to store the compressed data to obtain encoded data, wherein the positive number encoding means to encode the part of the compressed data that uses byte compression processing, and the negative number Encoding refers to encoding a portion of the compressed data that is subjected to bit compression processing.

With reference to the sixth possible implementation manner of the second aspect, in a seventh possible implementation manner of the second aspect, the encoded data includes the first encoded data, and when the compressed data is stored using a positive number encoding, the One of or a combination of positive codes and negative codes is used to store the compressed data and obtain coded data, including:

Acquiring the minimum number of bytes for encoding the maximum data according to the maximum data in the compressed data;

If the first bit value of the first byte of the largest data is zero, it is determined to use the minimum number of bytes to encode the part of the compressed data that uses byte compression processing to obtain the first encoded data;

If the first bit value of the first byte of the maximum data is non-zero, it is determined to use the number of bytes after adding 1 to the minimum number of bytes to encode the part of the compressed data that uses byte compression processing, and obtain The first encoded data.

With reference to the sixth or seventh possible implementation manner of the second aspect, in an eighth possible implementation manner of the second aspect, the encoded data includes the second encoded data, and when the compressed data is stored using negative encoding When, the compressed data is stored in one or a combination of the positive code and the negative code, and the coded data is obtained, including:

Negative encoding is performed on the part of the compressed data that adopts bit compression processing according to the negative encoding rule to obtain the second encoded data, the negative encoding rule includes a code length identification part and a data encoding part, and the code length identification part is used In order to represent the number of coded data, the data coding part is used to represent the coded value after the coded data is coded with bits of a preset length, wherein the length of the coded length identifier is determined by the second code The first M bits of each data in the data are identified, wherein the size of M is equal to the value between the first bit with a value of zero and the highest bit in the binary representation of each data in the second encoding The number of non-zero bits.

The embodiment of the present invention splits the data to be compressed according to the characteristics of the data to be compressed, so that the data to be compressed that is not suitable for using a special compression algorithm can be processed by a special compression algorithm to obtain a high compression ratio after splitting, thereby saving data storage space and reduce costs.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description These are some embodiments of the present invention. For those skilled in the art, other drawings can also be obtained according to these drawings without any creative effort.

FIG. 1 is a schematic structural diagram of Embodiment 1 of the data compression device of the present invention;

Fig. 2 is a group of data example of the present invention;

Fig. 3 is a schematic diagram of the format of the self-describing compression coding of the present invention;

Fig. 4 is an example diagram of a result after the present invention carries out self-describing compression coding to data;

Fig. 5 is another result example figure after the present invention carries out self-describing compression coding to data;

Fig. 6 is an example diagram of code stream data corresponding to the second group of data processed by the present invention using a general bit compression algorithm;

Fig. 7 is another group of data example of the present invention;

Fig. 8 is a schematic diagram of the present invention before and after encoding the second group of data;

9 is a schematic structural diagram of Embodiment 2 of the data compression device of the present invention;

FIG. 10 is a flow chart of Embodiment 1 of the data compression method of the present invention.

Detailed ways

In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments It is a part of embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

FIG. 1 is a schematic structural diagram of Embodiment 1 of the data compression device of the present invention. An embodiment of the present invention provides a data compression device, which can be integrated into a communication device, where the communication device can be any terminal device such as a personal computer (Personal Computer, PC for short), a notebook computer, or a server. As shown in FIG. 1 , the device 10 includes: a splitting module 11 , a compression module 12 and a storage module 13 .

Wherein, the splitting module 11 is used to split the data to be compressed according to the preset splitting rules, obtain the split data and send it to the compression module 12, so that the compression algorithm is applicable to the split data, wherein the preset split The split rule is determined according to the characteristics of the data to be compressed; the compression module 12 is used to receive the split data sent by the split module 11, and uses a compression algorithm to compress the split data, output the compressed data, and send it to Storage module 13; the storage module 13 is used to receive the compressed data sent by the compression module 12, and store the compressed data.

In practical applications, some data is compressed by existing compression methods but increases the information storage space. Therefore, in the embodiment of the present invention, it is firstly preprocessed according to the characteristics of the data to be compressed, for example, according to the characteristics of the data to be compressed, The preset splitting criterion is set in advance; and then the data to be compressed is split according to the preset splitting rule, so that the split data can be compressed by the compression method provided by the communication device to obtain a high compression ratio.

In the above-mentioned embodiment, the splitting module 11 may be specifically configured to: split the data to be compressed by using a preset value division method, wherein the number of segments to be split is determined according to a sampling test of the data to be compressed. Here, the preset value can be, for example, 10 ⁿ or 256 ⁿ , without limitation.

Specifically, after the data compression device 10 reads the original data, its built-in splitting module 11 splits the raw data into multiple groups according to preset splitting rules, and then triggers the compression module 12 to compress the split data deal with. If no preset splitting rule is set in advance, the original data is directly compressed by the compression module 12 .

Wherein, the preset splitting rules may have multiple types. The preset splitting rules are described below.

In the field of big data, some data is coded according to certain rules, for example, mobile phone number, International Mobile Subscriber Identification Number (IMSI) number, Internet Protocol (Internet Protocol, IP) address and domain name etc. Taking a mobile phone number in Mainland China as an example, the mobile phone number has a total length of 11 digits and consists of three parts: 1st to 3rd digits: network identification number; 4th to 7th digits: area code; 8th to 11th digits: user number, Therefore, before compressing the mobile phone number data, the mobile phone number can be split according to this rule.

For example, table 1 is to obtain high compression ratio, after a bill data is sorted by some fields, the segment of the part field data that obtains, wherein, SGSN_USR_IP represents in Serving GPRS Support Node (Serving GPRS Support Node, referred to as: The IP address assigned to the user on the SGSN), and GGSN_USR_IP indicates the IP address assigned to the user on the GPRS Gateway Support Node (Gateway GPRS Support Node, referred to as: GGSN).

Table 1 Fragments of a bill data after sorting

Analyze the data in Table 1:

1. The calling number and called number are not suitable for compression using a special compression algorithm, but if the calling number and called number are split into three sections (or two sections) according to their coding rules, the first two sections (split When it is divided into two segments, it is the previous segment) data can be compressed using the equivalent distance compression algorithm to obtain a high compression ratio, for example, split the calling number 1345439747 into three segments, 134, 5439 and 747; The number 1345439747 is split into two segments, 134 and 5439747, or 1345439 and 747. And because using an integer is more space-saving than a character string to store a mobile phone number, therefore, the preset splitting rule can use 10n integer division, and n is 0 or a positive integer. Optionally, the preset splitting rules corresponding to the calling number and the called number may be the same or different, which is not limited here.

2. The two IP address fields of SGSN_USR_IP and GGSN_USR_IP are composed of four dotted numbers, and the value range of each number is 0-256. These two fields can be compressed using a dictionary compression algorithm. However, due to the large value range of the IP address, the dictionary compression algorithm requires a 32-bit (bit) integer to complete the encoding. In practical applications, an IPV4 address is usually represented by 4 bytes, that is, a 32-bit integer, and each byte is a dotted number, which saves more space than an IP address directly represented by a string. Therefore, the above two IP address fields can be split using 256 ^{n integer} division.

For example, the rules for splitting an IPV4 address into two segments are: first segment: (IPV4&0xFFFF0000)>>16, second segment: IPV4&0x0000FFFF; or, first segment: (IPV4&0xFF000000)>>24, second segment: IPV4&0x00FFFFFF. The rules for splitting an IPV4 address into three segments are: first segment: (IPV4&0xFF000000)>>24, second segment: (IPV4&0x00FF0000)>>16, third segment: IPV4&0x0000FFFF, or other splitting rules.

In addition, it can also be formulated according to the characteristics of actual data. For another example, the domain name of a Uniform Resource Locator (Uniform Resource Locator, referred to as: URL) can be split by "/" or "." contained in the URL.

It should also be noted that, in any embodiment of the present invention, the number of split segments may be determined through a sampling test, or may be specified by a user. When determining the number of segments to be split by sampling test, test and compare all or part of the data to be compressed, and determine the split rule with high compression ratio and/or short compression time after splitting as the preset split rule.

In the compression module 12, specifically, the split data is compressed according to the compression algorithm provided by the communication device. There are many ways to obtain the compression algorithm. For example, the compression algorithm can be specified, or the compression algorithm can be obtained heuristically, or the compression algorithm can be obtained completely probingly. Among them, specifying the compression algorithm refers to specifying the compression algorithm through command line parameters or configuration files; heuristically obtaining the compression algorithm means notifying the program of the type and characteristics of the above-mentioned split data through command line parameters or configuration files Or the recommended compression algorithm, and then according to the preset compression algorithm, the program internally selects a suitable compression algorithm through sampling and splitting the data compression estimation; the complete detection acquisition compression algorithm refers to the sampling and splitting of the data Obtain the sampled data, use various compression algorithms to compress and estimate the sampled data, and then select an appropriate compression algorithm based on the above compression estimation results, and the user does not need to specify the compression algorithm to be used through parameters or configuration.

Further, the compression module 12 may be specifically configured to: receive the split data sent by the split module 11, use a compression algorithm to perform at least one compression process on the split data, and output the compressed data, wherein, in each compression process The compression algorithm used is the same or different.

It should be noted that after the compression module 12 compresses the split data once, it can also compress the compressed data again, that is, multi-level compression. In general, the more the number of compression stages, the incremental reduction of the compression ratio that can be obtained, and the increase of the compression time cost. Usually, the setting of the number of compression stages should not be too large. Therefore, the setting of compression stages needs to consider the balance between compression time and compression efficiency, but if there is no requirement for compression time, the number of compression stages can be increased appropriately. to obtain the highest possible compression ratio.

The embodiment of the present invention splits the data to be compressed according to the characteristics of the data to be compressed, so that the data to be compressed that is not suitable for using a special compression algorithm can be processed by a special compression algorithm to obtain a high compression ratio after splitting, thereby saving data storage space and reduce costs.

In addition, because the compressed data itself needs to be described with data of a certain size, for example, the equivalent distance compression algorithm is used to describe a set of repeated data as shown in Figure 2 using the <repeated data, number of repetitions> two-tuple, so that This group of repeated data is compressed, and the compressed data is 5 integers of "2, 10000, 100, 20000, 90", where the first number "2" represents two binary groups, and the following "10000, 100, 20000, 90" are two 2-tuples, which respectively represent repeated data of 10000 and 20000, and repeated times of 100 and 90. The above-mentioned compressed data needs to be described with a certain size of data. If the data used is too large, such as 4 bytes, the storage space will be wasted; if the data used is too small, such as 1 byte, the storage space will be insufficient. Therefore, the embodiment of the present invention improves the method for storing compressed data.

In an implementation manner, the storage module 13 may be specifically configured to: use self-describing compression coding to store compressed data and obtain coded data. Wherein, FIG. 3 is a schematic diagram of the format of the self-describing compression coding of the present invention. As shown in FIG. 3 , the self-describing compression coding may include a coding length identification part 31 and a data value coding part 32, and the coding length identification part 31 is used to represent the compressed data The number of bytes to encode, the data value encoding part 32 is used to represent the part that encodes the compressed data. The encoded length of the compressed data is identified by its first N bits, where the size of N is equal to the value between the first bit with a value of "0" and the highest bit in the binary representation of the encoded data The number of bits that are "1" plus 1, N is the data encoding length, see Table 2 for details.

Specifically, "0" is used to separate the encoding length identification part 31 and the data value encoding part 32 . For example: the binary number 1011111111111111, its encoding length identification part 31 is the number of "1" bits before the first "0" bit of the encoded data from the high position, at this time, the number of bits that are valued as "1" is 1, so N=1+1=2. The data value coding part 32 is the remaining part of the coded data "11111111111111", which are separated by the bit "0" of the code length identification part 31.

Table 2 Example encoding rules for self-describing compression encoding

Using the encoding rules shown in Table 2, self-describing compression encoding is performed on the compressed data "2,10000,100,20000,90", the hexadecimal result is shown in Figure 4, it can be seen that the result only needs 8 bytes That is, compared to using unsigned integers to store compressed data, there is no need to predetermine how many bytes of integers should be used to describe the number of compressed data, so as to avoid the waste of the predetermined number of bytes and realize storage space rational distribution and utilization. Wherein, the ellipsis "..." in Table 2 indicates that the content shown is deduced sequentially and so on, and details will not be repeated here.

In another implementation manner, the data compression device 10 may also include: a determination module (not shown) used to trigger the compression module 12 to perform compression on the pre-compressed data if it is determined that the scale of the pre-compressed data in the compression process is applicable Perform compression processing. At this time, the compression module 12 is specifically configured to perform compression processing on the pre-compressed data by adopting a modified bit compression algorithm to obtain compressed data, wherein the modified bit compression algorithm indicates that each data is compressed using a different number of bits.

Optionally, the determining module may be specifically configured to: obtain the minimum number of bytes for encoding the largest data in the precompressed data; A number of bytes; if the pre-compressed data is bit-coded using the number of bits of the preset value, then determine the second number of bytes for bit-coding the pre-compressed data; if it is determined that the second number of bytes is less than the first byte number, then it is determined that the scale of the pre-compressed data is applicable to the deformed bit compression algorithm, and the compression module 12 is triggered to compress the pre-compressed data; Compress data or end compression processing on it.

Among them, the deformed bit compression algorithm is defined relative to the general bit compression algorithm. The general bit compression algorithm is to find the maximum value in the data to be compressed, determine the minimum number of bits required to encode the maximum value, and then use this for all the data to be compressed The number of bits is used for encoding; the deformed bit compression algorithm is to encode each data in the data to be compressed using different number of bits. The following example illustrates the application of the deformed bit compression algorithm.

For example, compress a piece of continuous group identification code (GROUP_ID) data in a bill data. This paragraph GROUP_ID data (70 in total) is: 153 153 153 153 153 153 153 153 153 153 153 153 153 151 151 151 151 151 151 1511 153 153 153 152 153 153 153 152 153 153 151 151 151 153151 153 153 153 153 153 153 153 153 153 152 153 153 152 152 152 151 151 151153 153 151 152 152 152.

First, use the equivalent compression algorithm to compress the above GROUP_ID data to obtain the first compressed data, that is, 19 binary groups <repeated data, number of repetitions>: <153,16><151,10><153,6>< 152,2><151,1><153,3><152,2><153,1><151,3><153,1><151,1><153,9><152,1>< 153,2><152,3><151,3><153,2><151,1><152,3>.

Then, by encoding and storing the repeated data and the number of repetitions separately, two sets of data can be obtained: the first group is a collection of repeated data: [153,151,153,152,151,153,152,153,151,153,151,153,152,

153,152,151,153,151,152]; the second group is the combination of repetitions: [16,10,6,2,1,3,2,1,3,1,1,9,

1,2,3,3,2,1,3], wherein each repetition number corresponds to each repetition data in the first group one-to-one.

The difference compression algorithm is used to compress the data in the first group twice. Based on 151, the compressed data obtained is: (151), 2,0,2,1,0,2,1,2,0,2,0,2,1,2; the compression is described by self-describing compression coding Among them, 151 uses self-describing compression encoding, and all subsequent data uses 2-bit encoding, and the encoded result is shown in Figure 5.

The data in the second group is compressed twice using a modified bit compression algorithm. If the general bit compression algorithm is used, since the maximum value of the data in the second group is 16, the corresponding binary number is 10000, that is, at least 5-bit coding is required, therefore, all data in the second group must use 5-bit coding. At this time, if other compression coding information is not considered, the encoding of the second group of data itself needs round((5*19+7)/8)=12 bytes, where the round() function means rounding processing, 5*19/ The calculation result of 8 has a part after the decimal point. If you calculate round(5*19/8), the part after the decimal point will be lost and storage distortion will occur. Therefore, here, add 7 to the molecular part in the brackets, that is, to (5* 19+7)/8 is rounded up to avoid the above distortion and ensure the accuracy of the stored results. Figure 6 shows the corresponding code stream (binary) data after processing the second set of data using the general bit compression algorithm.

Observing the data in the second group above, it can be found that only a few data are relatively large, and most of the data are relatively small. Therefore, the modified bit compression algorithm is used to compress the second group of data to obtain a smaller code stream. For example, use byte compression for 16, 10, and 9, and bit compression of less than 5 bits for other data. In the following, different encoding rules are used to distinguish whether compressed data adopts bit compression.

On the basis of the above, the storage module 13 can be used to: adopt one or a combination of positive codes and negative codes to store compressed data and obtain coded data, wherein the positive code represents the part of the compressed data that uses byte compression processing Encoding is carried out, and the negative number encoding indicates that the part of the compressed data that is processed by bit compression is encoded.

Further, the coded data includes the first coded data, and when the compressed data is stored using a positive code, the storage module 13 can be specifically used to: obtain the minimum number of bytes for encoding the maximum data according to the maximum data in the compressed data; if If the value of the first bit of the first byte of the largest data is zero, it is determined to use the minimum number of bytes to encode the part of the compressed data that uses byte compression processing to obtain the first encoded data; if the first bit of the first byte of the largest data If the value is non-zero, it is determined to use the minimum number of bytes plus 1 to encode the part of the compressed data that is subjected to byte compression processing to obtain the first encoded data.

On the basis of the above, the encoded data includes the second encoded data. When negative encoding is used to store the compressed data, the storage module 13 can be specifically used to: perform negative encoding on the part of the compressed data that uses bit compression according to the negative encoding rule to obtain the first Two-coded data, the negative number coding rule includes a code length identification part and a data coding part, the code length identification part is used to represent the number of coded data, and the data code part is used to represent that the coded data is coded with bits of a preset length The coded value after, wherein, the length of the coded length identifier is identified by the first M bits of each data in the second coded data, wherein, the size of M is equal to the first bit from the highest bit in the binary representation of each data in the second coded The number of non-zero bits between the zero-valued bits and the most significant bit.

Specifically, first find out the maximum number in the compressed data, and obtain the minimum number of bytes to encode it, assuming that the minimum number of bytes is m, and m is a positive integer. If the first bit in the first byte of the maximum number is 0, all compressed data that uses byte compression processing will use m bytes for positive encoding, otherwise use m+1 bytes for positive encoding , and the number of bytes encoded with a positive number needs to be stored in the compressed code stream to facilitate identification and processing during decompression.

For the data processed by bit compression, use negative encoding to distinguish it from the previous byte compression result. The method is to set the value of the first bit to "1" for the result of bit compression, and the name of the negative number encoding also comes from this (because the first bit is 1, it is a negative number for signed numbers), and it is exactly the same as Correspond to the positive number code in front.

The bit compression result is as follows: the beginning is the encoding of the number of compressed data (use the negative number encoding rule, as shown in Table 3), followed by the bit encoding data of each compressed data according to the specified bit value.

It should also be noted that since the number of data to be coded with a negative number cannot be 0, the coded data is decremented by 1 during actual coding, and correspondingly, 1 is added when decoding it. For example, 1 is coded with 0 , 2 is encoded by 1, 3 is encoded by 2, ..., and so on; when decoding, the data to be decoded is reversed, and the decoding result must be added by 1. For example, 0 is decoded as 1, 1 is decoded as 2, and 2 is decoded as 3, ..., and so on.

Table 3 Negative encoding rules

Since the number of data processed by bit compression must be added when using bit coding, the more data that can be coded, the more conducive to compression. Next, it is illustrated by two derivations.

First, deduce when the size of the coded data is suitable for using bit coding.

Assuming that the data size is n, the maximum value requires at least m bytes for encoding. If bit encoding is not used, the product of m and n (m*n) bytes are required for encoding.

Assuming that this group of numbers can be encoded using b bits, a total of round((m*b+7)/8) bytes are required; in addition, at least 1 byte is required to identify the number of encoded data, therefore, The condition suitable for bit coding is: m*n>1+round((n*b+7)/8), that is, n>round(15/(8m-b)), where round is the rounding down function . It can be seen that if there are less than round(15/(8m-b)) consecutive data that can be compressed using b bits, even if there is data that can be compressed using b bits, this part of the data is not suitable for bit coding.

Fig. 7 is an example of another set of data in the present invention. As shown in FIG. 7 , a set of data examples is given, and the part of the set of data suitable for bit compression is described below. The maximum number in this group of data is 16, and the minimum 1-byte encoding is required. If 3-bit compression encoding is used, according to n>round(15/(8*1-3))=3, at least 3 consecutive data are suitable for use 3-bit encoding, this group of data is suitable for bit encoding, and thus the result shown in Figure 7 can be obtained.

Second, obtain the optimal coded bit value of the bit code.

For a given set of data to be compressed, first determine the bit value and number of bytes required for the largest data encoding in the set of data, the number of bytes required to bit-encode the set of data using the bit value, and the set The number of data in the data that can be encoded continuously using the number of digits of the bit value; then, according to the above predetermined value, the bit value is decremented by "1", and when the bit value is judged one by one to take different values, encode the group of data to be compressed The number of bytes required, in each cycle, judge the size relationship between the number of bytes obtained in this cycle and the number of bytes obtained in the previous cycle, and take the smaller value of the two as the final value of this cycle The number of bytes; after the last cycle ends, that is, after the judgment that the bit value is "1", the optimal number of bytes and its corresponding bit value, that is, the optimal encoding bit value, are determined. Wherein, the method for obtaining the optimal coded bit value can be realized by different algorithms, which is not limited in the present invention.

For the above second group of data, it can be found that using 2-bit encoding is the best solution, wherein the data before and after encoding are shown in FIG. 8 . It can be seen that using the 2-bit modified bit compression algorithm can obtain a 10-byte compression result, which is 2 bytes less than the 12-byte compression result of the previous general-purpose 5-bit compression algorithm, and is better than directly using 1-byte encoding for each data. (A total of 19 bytes are required) 9 bytes are missing.

Therefore, compared with the general bit compression algorithm that uses the number of bits of the largest data in the data to be compressed to encode, this implementation method uses different numbers of bits to encode different data in the data to be compressed, which can effectively solve the problem that only When there are very few large data and the others are small data, the problem of waste of storage resources caused by encoding with the maximum number of bits of data.

FIG. 9 is a schematic structural diagram of Embodiment 2 of the data compression device of the present invention. As shown in FIG. 9 , the data compression device 20 includes a processor 91 and a memory 92 . Wherein, the memory 92 stores execution instructions. When the data compression device 20 is running, the processor 91 communicates with the memory 92, and the processor 91 calls the execution instructions in the memory 92 to perform the following operations:

Split the data to be compressed according to a preset split rule to obtain the split data, so that the compression algorithm is applicable to the split data, wherein the preset split rule is based on the data to be compressed Characterized;

The split data is compressed using the compression algorithm, and the compressed data is output and sent to the memory 92 .

The memory 92 is used for receiving the compressed data sent by the processor 91 and storing the compressed data.

In the above embodiment, splitting the data to be compressed according to the preset splitting rules, and obtaining the split data may include: splitting the data to be compressed by using a preset value division method, wherein the number of segments to be split is It is determined by performing a sampling test on the data to be compressed.

In the above embodiment, the compressing the split data by using the compression algorithm and outputting the compressed data may include: compressing the split data at least once by using the compression algorithm, and outputting the compressed data. data, wherein the compression algorithms used in each compression process are the same or different.

Optionally, in an implementation manner, the storing the compressed data may include: storing the compressed data using a self-describing compression code to obtain coded data, the self-describing compression code including a code length identification part and a data value An encoding part, the encoding length identification part is used to represent the number of bytes for encoding the compressed data, and the data value encoding part is used to represent the part for encoding the compressed data, wherein the compressed data The coded length of is identified by its own first N bits, where the size of N is equal to the value between the first zero bit from the highest bit and the highest bit in the binary representation of the encoded data. Add 1 to the number of zero bits.

In another implementation manner, if the compression algorithm used in the compression processing is a deformed bit compression algorithm, the processor 91 may also be configured to: before compressing the split data using the compression algorithm, If it is determined that the scale of the pre-compressed data in the compression process is applicable to the deformed bit compression algorithm, then the deformed bit compression algorithm is used to compress the pre-compressed data to obtain the compressed data, wherein the deformed bit The compression algorithm means that each data is compressed using different bit numbers.

Further, the determination that the size of the pre-compressed data in the compression process is applicable to the deformed bit compression algorithm may include: obtaining the minimum number of bytes for encoding the largest data in the pre-compressed data; according to the pre-compressed data size and the minimum number of bytes, determine the first number of bytes for encoding the pre-compressed data; A second number of bytes for which the pre-compressed data is bit-coded; if it is determined that the second number of bytes is smaller than the first number of bytes, it is determined that the scale of the pre-compressed data is suitable for a deformed bit compression algorithm.

On the basis of the above, the storing of the compressed data may include: storing the compressed data by using one or a combination of positive encoding and negative encoding to obtain encoded data, wherein the positive encoding indicates that the compression The part of the data that is processed by byte compression is coded, and the negative number coding indicates that the part of the compressed data that is processed by bit compression is coded.

Optionally, the coded data includes first coded data, and when the compressed data is stored using positive code, one or a combination of positive code and negative code is used to store the compressed data to obtain coded data , including: according to the maximum data in the compressed data, obtain the minimum number of bytes for encoding the maximum data; if the first bit value of the first byte of the maximum data is zero, then determine that the minimum number of bytes The number of sections encodes the part of the compressed data that uses byte compression to obtain the first encoded data; if the first bit value of the first byte of the largest data is non-zero, it is determined to use the smallest word The number of bytes obtained by adding 1 to the number of sections encodes the part of the compressed data that adopts byte compression processing to obtain the first encoded data.

Further, the encoded data includes the second encoded data. When the compressed data is stored with negative encoding, one or a combination of positive encoding and negative encoding is used to store the compressed data, and the encoded data can be obtained. It includes: performing negative coding on the part of the compressed data that adopts bit compression processing according to a negative coding rule to obtain the second coded data, the negative coding rule includes a coding length identification part and a data coding part, and the coding length identification The part is used to represent the number of coded data, and the data coding part is used to represent the coded value after coding the coded data with bits of a preset length, wherein the length identified by the coded length is determined by the first The first M bits of each data in the second coded data are identified, wherein the size of M is equal to the distance between the first zero value from the highest bit and the highest bit in the binary representation of each data in the second code The number of non-zero bits.

The implementation principles and technical effects of the components of the data compression device in this embodiment are similar to those of the corresponding modules in the embodiment shown in FIG. 1 , and will not be repeated here.

FIG. 10 is a flow chart of Embodiment 1 of the data compression method of the present invention. An embodiment of the present invention provides a data compression method, which can be executed by the above-mentioned data compression device, and the device can be integrated into a communication device, wherein the communication device can be any terminal device such as a PC, a notebook computer, or a server. As shown in Figure 10, the method includes:

S101. Split the data to be compressed according to a preset split rule, and obtain the split data, so that the compression algorithm is applicable to the split data, wherein the preset split rule is determined according to the characteristics of the data to be compressed .

Specifically, after the data compression device reads the original data, it splits the original data into multiple groups according to a preset split rule, and then executes S102. If no preset splitting rule is set in advance, the original data is directly compressed, and S102 is executed.

S102. Compress the split data by using a compression algorithm, and output the compressed data.

S103. Store the compressed data.

The method in this embodiment can be executed by the device shown in FIG. 1 or FIG. 9 , and its implementation principle and technical effect are similar, and will not be repeated here.

In the above-mentioned embodiment, S101 may include: splitting the data to be compressed by using a preset value division method, wherein the number of segments to be split is determined according to a sampling test of the data to be compressed.

Optionally, S102 may include: performing at least one compression process on the split data by using a compression algorithm, and outputting the compressed data, wherein the compression algorithm used in each compression process is the same or different.

On the basis of the above, in an implementation manner, S103 may specifically include: using self-describing compression coding to store the compressed data to obtain coded data, the self-describing compression coding includes a coding length identification part and a data value coding part, the The encoding length identification part is used to represent the number of bytes for encoding the compressed data, and the data value encoding part is used to represent the part for encoding the compressed data, wherein the encoding length of the compressed data is determined by its The first N bits of itself are used to identify, wherein, the size of N is equal to the number of non-zero bits between the first zero value from the highest bit and the highest bit in the binary representation of the encoded data Add 1 to the number.

Further, before S102, the data compression method may also include: if it is determined that the size of the pre-compressed data in the compression process is applicable to the deformed bit compression algorithm, then compressing the pre-compressed data by using the deformed bit compression algorithm processing to obtain the compressed data, wherein the deformed bit compression algorithm means that different bit numbers are used to perform compression processing on each data.

Wherein, determining that the size of the pre-compressed data in the compression process is applicable to the deformed bit compression algorithm may include: obtaining the minimum number of bytes for encoding the maximum data in the pre-compressed data; according to the size of the pre-compressed data and the minimum number of bytes, determine The first number of bytes for byte-encoding the pre-compressed data; if the pre-compressed data is bit-encoded using the number of bits of the preset value, then determine the second number of bytes for bit-encoding the pre-compressed data; if determined If the second number of bytes is smaller than the first number of bytes, it is determined that the size of the pre-compressed data is applicable to the deformed bit compression algorithm.

On the basis of the foregoing embodiments, S103 may include: storing the compressed data by using one or a combination of positive encoding and negative encoding to obtain encoded data. Wherein, the positive number encoding means encoding the part of the compressed data that adopts byte compression processing, and the negative number encoding represents encoding the part of the compressed data that adopts bit compression processing.

Optionally, the coded data includes the first coded data. When the compressed data is stored using positive codes, the compressed data is stored using one or a combination of positive codes, and the coded data is obtained, which may include: according to The maximum data in the compressed data is obtained by obtaining the minimum number of bytes for encoding the maximum data; if the first bit value of the first byte of the maximum data is zero, it is determined that the minimum number of bytes is used to encode the Encode the part of the compressed data that uses byte compression processing to obtain the first encoded data; if the first bit value of the first byte of the largest data is non-zero, then determine to use the minimum number of bytes plus 1 Encode the part of the compressed data that is subjected to byte compression processing to obtain the first encoded data.

Further, the encoded data includes the second encoded data. When the compressed data is stored with negative encoding, one or a combination of positive encoding and negative encoding is used to store the compressed data, and the encoded data can be obtained. It includes: performing negative coding on the part of the compressed data that adopts bit compression processing according to a negative coding rule to obtain the second coded data, the negative coding rule includes a coding length identification part and a data coding part, and the coding length identification The part is used to represent the number of coded data, and the data coding part is used to represent the coded value after coding the coded data with bits of a preset length, wherein the length identified by the coded length is determined by the first The first M bits of each data in the second coded data are identified, wherein the size of M is equal to the distance between the first zero value from the highest bit and the highest bit in the binary representation of each data in the second code The number of non-zero bits.

The embodiment of the present invention splits the data to be compressed according to the characteristics of the data to be compressed, so that the data to be compressed that is not suitable for using a special compression algorithm can be processed by a special compression algorithm to obtain a high compression ratio after splitting, thereby saving data storage space and reduce costs.

Those of ordinary skill in the art can understand that all or part of the steps for implementing the above method embodiments can be completed by program instructions and related hardware. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps including the above-mentioned method embodiments; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk and other various media that can store program codes.

Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention, rather than limiting them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: It is still possible to modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the technical solutions of the various embodiments of the present invention. scope.

Claims (18)

1. a data compression device, is characterized in that, comprising:

Splitting module, for splitting rule according to presetting, splitting data to be compressed, the Data Concurrent obtained after splitting gives compression module, data after making compression algorithm be applicable to described fractionation, wherein, described default fractionation rule determines according to the feature of described data to be compressed;

Described compression module, for receiving the data after described fractionation that described fractionation module sends, adopting described compression algorithm to carry out compression to the data after described fractionation and processing, output squeezing data, and send to memory module;

Described memory module, for receiving the described packed data that described compression module sends, and stores described packed data.

2. device according to claim 1, is characterized in that, described fractionation module specifically for:

Adopt preset value to divide exactly method, split described data to be compressed, wherein, the hop count of fractionation is determined according to carrying out sampling test to described data to be compressed.

3. device according to claim 1 and 2, is characterized in that, described compression module specifically for:

Receive the data after the described fractionation of described fractionation module transmission, adopt described compression algorithm to carry out at least first compression process to the data after described fractionation, export described packed data, wherein, the compression algorithm adopted in every second compression process is identical or different.

4. the device according to claim 1 or 2 or 3, is characterized in that, described memory module specifically for:

Self-described compressed encoding is adopted to store described packed data, obtain coded data, described self-described compressed encoding comprises code length identification division and data value coded portion, described code length identification division is for characterizing the byte number of encoding to described packed data, described data value coded portion is for characterizing the part of encoding to described packed data, wherein, the code length of described packed data is identified by the N number of bit of the head of himself, wherein, it is that between the position of zero and highest order, value is that the number of the position of non-zero adds 1 again that the size of N to equal in the binary representation of described coded data from highest order first value.

5. the device according to claim 1 or 2 or 3, it is characterized in that, described device also comprises:

Determination module, if for determining that the scale of the pre-compressed data in described compression process is suitable for deformation ratio extreme pressure compression algorithm, then trigger described compression module and carry out compression process to described pre-compressed data;

Then described compression module is specifically for adopting described deformation ratio extreme pressure compression algorithm to carrying out compression process in described pre-compressed data, obtain described packed data, wherein, described deformation ratio extreme pressure compression algorithm represents that adopting different number of bits to carry out compression to each data processes.

6. device according to claim 5, is characterized in that, described determination module specifically for:

Obtain the minimum byte number that maximum data in described pre-compressed data is encoded;

According to scale and the described minimum byte number of described pre-compressed data, determine the first byte number described pre-compressed data being carried out to byte code;

If use the number of bits of default value to carry out bits of encoded to described pre-compressed data, then determine the second byte number described pre-compressed data being carried out to bits of encoded;

If determine, described second byte number is less than described first byte number, then determine that the scale of described pre-compressed data is suitable for deformation ratio extreme pressure compression algorithm, and trigger described compression module to described pre-compressed data carry out compression process.

7. the device according to claim 5 or 6, is characterized in that, described memory module is used for:

One or its combination in positive number encoder and negative coding is adopted to store described packed data, obtain coded data, wherein, described positive number coded representation is encoded to adopting the part of byte-code compression process in described packed data, and described negative coded representation is encoded to adopting the part of bit compression process in described packed data.

8. device according to claim 7, is characterized in that, described coded data comprises first coding data, when adopting packed data described in positive number code storage, described memory module specifically for:

According to the maximum data in described packed data, obtain the minimum byte number that described maximum data is encoded;

If the first bit value of the first byte of described maximum data is zero, then determines to adopt described minimum byte number to encode to adopting the part of byte-code compression process in described packed data, obtaining described first coding data;

If the first bit value of the first byte of described maximum data is non-zero, then determine that adopting described minimum byte number to add the byte number after 1 encodes to adopting the part of byte-code compression process in described packed data, obtains described first coding data.

9. the device according to claim 7 or 8, is characterized in that, described coded data comprises the second coded data, when adopting packed data described in negative code storage, described memory module specifically for:

Negative coding is carried out to adopting the part of bit compression process in described packed data according to negative coding rule, obtain described second coded data, described negative coding rule comprises code length identification division and data encoding portion, described code length identification division is for characterizing by the number of coded data, described data encoding portion adopts the bit of preset length to the encoded radio after being encoded by coded data for characterizing, wherein, the length of described code length mark is identified by first M bit of each data in described second coded data, wherein, it is that between the position of zero and highest order, value is the number of the position of non-zero that the size of M to equal in the binary representation of each data in described second coding first value from highest order.

10. a data compression method, is characterized in that, comprising:

According to default fractionation rule, split data to be compressed, obtain the data after splitting, the data after making compression algorithm be applicable to described fractionation, wherein, described default fractionation rule determines according to the feature of described data to be compressed;

Described compression algorithm is adopted to carry out compression process to the data after described fractionation, output squeezing data;

Store described packed data.

11. methods according to claim 10, is characterized in that, described basis is preset and split rule, splits data to be compressed, obtains the data after splitting, comprising:

Adopt preset value to divide exactly method, split described data to be compressed, wherein, the hop count of fractionation is determined according to carrying out sampling test to described data to be compressed.

12. the method according to claim 10 or 11, is characterized in that, described employing compression algorithm carries out compression process to the data after described fractionation, and output squeezing data, comprising:

Adopt described compression algorithm to carry out at least first compression process to the data after described fractionation, output squeezing data, wherein, the compression algorithm adopted in every second compression process is identical or different.

13. methods according to claim 10 or 11 or 12, it is characterized in that, the described packed data of described storage, comprising:

Self-described compressed encoding is adopted to store described packed data, obtain coded data, described self-described compressed encoding comprises code length identification division and data value coded portion, described code length identification division is for characterizing the byte number of encoding to described packed data, described data value coded portion is for characterizing the part of encoding to described packed data, wherein, the code length of described packed data is identified by the N number of bit of the head of himself, wherein, it is that between the position of zero and highest order, value is that the number of the position of non-zero adds 1 again that the size of N to equal in the binary representation of described coded data from highest order first value.

14. the method according to claim 10 or 11 or 12, is characterized in that, the described compression algorithm of described employing also comprises before carrying out compression process to the data after fractionation:

If determine, the scale of the pre-compressed data in described compression process is suitable for deformation ratio extreme pressure compression algorithm, then adopt described deformation ratio extreme pressure compression algorithm to carrying out compression process in described pre-compressed data, obtain described packed data, wherein, described deformation ratio extreme pressure compression algorithm represents that adopting different number of bits to carry out compression to each data processes.

15. methods according to claim 14, is characterized in that, describedly determine that the scale of pre-compressed data in described compression process is suitable for deformation ratio extreme pressure compression algorithm, comprising:

Obtain the minimum byte number that maximum data in described pre-compressed data is encoded;

According to scale and the described minimum byte number of described pre-compressed data, determine the first byte number described pre-compressed data being carried out to byte code;

If use the number of bits of default value to carry out bits of encoded to described pre-compressed data, then determine the second byte number described pre-compressed data being carried out to bits of encoded;

If determine, described second byte number is less than described first byte number, then determine that the scale of described pre-compressed data is suitable for deformation ratio extreme pressure compression algorithm.

16. methods according to claims 14 or 15, it is characterized in that, the described packed data of described storage, comprising:

One or its combination in positive number encoder and negative coding is adopted to store described packed data, obtain coded data, wherein, described positive number coded representation is encoded to adopting the part of byte-code compression process in described packed data, and described negative coded representation is encoded to adopting the part of bit compression process in described packed data.

17. methods according to claim 16, is characterized in that, described coded data comprises first coding data, when adopting packed data described in positive number code storage, the positive number encoder of described employing and negative coding in one or its combination store described packed data, obtain coded data, comprising:

According to the maximum data in described packed data, obtain the minimum byte number that described maximum data is encoded;

If the first bit value of the first byte of described maximum data is zero, then determines to adopt described minimum byte number to encode to adopting the part of byte-code compression process in described packed data, obtaining described first coding data;

If the first bit value of the first byte of described maximum data is non-zero, then determine that adopting described minimum byte number to add the byte number after 1 encodes to adopting the part of byte-code compression process in described packed data, obtains described first coding data.

18. methods according to claim 16 or 17, it is characterized in that, described coded data comprises the second coded data, when adopting packed data described in negative code storage, the positive number encoder of described employing and negative coding in one or its combination store described packed data, obtain coded data, comprising:

Negative coding is carried out to adopting the part of bit compression process in described packed data according to negative coding rule, obtain described second coded data, described negative coding rule comprises code length identification division and data encoding portion, described code length identification division is for characterizing by the number of coded data, described data encoding portion adopts the bit of preset length to the encoded radio after being encoded by coded data for characterizing, wherein, the length of described code length mark is identified by first M bit of each data in described second coded data, wherein, it is that between the position of zero and highest order, value is the number of the position of non-zero that the size of M to equal in the binary representation of each data in described second coding first value from highest order.