WO1999027714A1

WO1999027714A1 - Method and arrangement relating to data compression

Info

Publication number: WO1999027714A1
Application number: PCT/SE1998/002120
Authority: WO
Inventors: Martin Larsson
Original assignee: Telefonaktiebolaget Lm Ericsson
Priority date: 1997-11-21
Filing date: 1998-11-23
Publication date: 1999-06-03
Also published as: SE9704321D0; SE9704321L; AU1358299A; SE521925C2

Abstract

The invention relates to a method and arrangement for compressing, a compressed or non-compressed digital data set. The method comprises a first step of converting the data set, which can be any digital data set represented by a digital representation such as, ASCII characters, Hex, Octal or the like, into a representation of the data set, said representation being a pixel representation, in which a binary one is represented by an active (black) pixel and a binary zero is represented by an inactive (white) pixel or vice versa and a second step, in which a compression procedure is applied onto said representation of the data set at least one time.

Description

TITLE

METHOD AND ARRANGEMENT RELATING TO DATA COMPRESSION

TECHNICAL FIELD OF THE INVENTION

The present invention relates to a method and arrangement for compression of digital data. The invention also relates to a communications network including a data compression arrangement.

BACKGROUND

The amount of produced, processed, transmitted and stored digital data increases for each day. This demands a larger and faster storage and transmission means with a high bandwidth. Consequently, the number of new techniques and methods for compression of data has also increased. It seems that there is a major problem to achieve an optimum compression rate. Many algorithms perform a byte wise compression and not a bit wise compression, which means that incomplete compressed data sets are obtained.

According to the source coding thesis of the information theory, a source is completely characterized by its entropy (H), i.e., the source can be represented as accurately as possible by R bits per time unit if R>H, but not R<H. However, it is possible to combine several sets of codes and represent them with a new code where R < (H, + H₂ +...+H_n), where H_n is the entropy of each source code.

There are many effective algorithms for compression of digitalized pictures, both motion and still pictures, for example GIF (Graphics Interchange Format [both trademarks of CompuServe, Inc.]), TIFF, JPEG and MPEG, which are used on the image data. Image compression reduces the storage requirements of pictorial data. In addition it reduces the time required for access to, communication with, and display of images. Also, data files can be compressed using many techniques in some extend less effective techniques, such as PKZIP, LZW, etc.

The Variable-Length-Code LZW Compression is a variation of the Lempel-Ziv Compression algorithm in which variable-length codes are used to replace patterns detected in the original data. The algorithm uses a code or a translation table constructed from the patterns encountered in the original data; each new pattern is entered into the table and its index is used to replace it in the compressed stream.

The compressor takes the data from the input stream and builds a code or translation table with the patterns as it encounters them; each new pattern is entered into the code table and its index is added to the output stream; when a pattern is encountered, which had been detected since the last code table refresh, its index from the code table is put on the output stream, thus achieving the data compression. When expanding and decompressing, the expander takes input from the compressed data stream and builds the code or translation table from it. When the compressed data stream is processed, codes are used to index into the code table and the corresponding data is put on the decompressed output stream, thus achieving data decompression. The Variable-Length-Code aspect of the algorithm is based on an initial code size (LZW-initial code size), which specifies the initial number of bits used for the compression codes. When the number of patterns detected by the compressor in the input stream exceeds the number of patterns encodable with the current number of bits, the number of bits per LZW code is increased by one.

SUMMERY

The main object of the present invention is to provide a reliable and effective method for compressing, compressed or non-compressed digital data. Another object of the invention is to provide a method to achieve a substantially maximum compression ratio using standard or modified and on the market available compressing techniques, without a need for developing new costly techniques. Another object of the invention is to provide an arrangement for compressing digital data.

The invention also has as an object to provide an arrangement for substantially real time compression of data streams, e.g., in form of digital data in a communications network, preferably a telecommunications network.

For these reasons, the method according to the beginning comprises a first step of converting the data set, which can be any digital data set represented by a digital representation such as, ASCII characters, Hex, Octal or the like, into a representation of the data set, said representation being a pixel representation, in which a binary one is represented by an active (black) pixel and a binary zero is represented by an inactive (white) pixel or vice versa and a second step, in which a compression procedure is applied onto said representation of the data set at least one time.

To achieve substantially maximum compression the first and second steps are repeated. In one preferred embodiment the data set includes binary data and the data representation is a pixel representation, in which a binary one is represented by an active (black) pixel and a binary zero is represented by an inactive (white) pixel or vice versa. It is also possible to represent the data set by ASCII characters, HEX., Octal etc. Preferably, the data set is divided into blocks of data and each block is converted to an image-frame representing the data block and a compression is applied on several frames, preferably resulting in fewer frames. Finally the compressed data set is provided with a header, which includes at least information on number of compressions applied. In an advantageous embodiment conversion and compression are carried out in parallel.

In a preferred arrangement for compressing the data set, the arrangement includes interface means to receive data set to be compressed, conversion means for converting data to a form representing the original data set, compression means to compress the data representation. Advantageously, the arrangement further includes means to execute a loop of conversion- compression for achieving higher compression. The arrangement may include means to convert data to or from digital form.

The invention also relates to a communications network, which includes devices for generating digital data, for example telephone units computer units etc., transceiver device and networking arrangement. Preferably, the networking arrangement and at least one of said devices, include means for converting communication data to a representation of the data and means to compress the data representation at least one time before transmission. Advantageously, the representation is a pictorial representation of the communication data. More advantageously, the communication data is converted to packages which are converted to data-representing frames arranged in sequences and said compression is applied to the several frames. For the compression one or several of MPEG, MJPEG, JPEG, GIF, TIFF or the like are used.

The invention also relates to a method for encryption of a data set containing binary data. The method includes the steps of: converting the binary data in a first stage to an image- representation of the data set, compressing said image-representation of the data set in a second stage, repeating said first and second stages at least a number of times, and creating an encrypted data set having encrypted compressor identity and number of repeating. In an advantage embodiment different compressions are applied. The compression is one or several of MPEG, MJPEG, JPEG, GIF, TIFF, TARGA, PKZIP, LZW or the like.

BRIEF DESCRIPTION OF THE DRAWINGS

In the following, the invention will be further described in a non-limiting way under reference to the accompanying drawings in which:

Fig. 1 is a flow diagram illustrating the main steps in an embodiment according to the present invention.

Fig. 2 illustrates a schematic embodiment of an arrangement according to the invention.

Fig. 3 is a flow diagram illustrating steps in an application using the compression method according to the present invention.

Fig. 4 illustrates schematically a communications network employing an arrangement, according to the invention.

DETAILED DESCRIPTION OF THE EMBODIMENTS

The main steps of the compression method according to the present invention for compressing a set of pre- or non-compressed data is illustrated in the flow diagram of fig. 1. In a first step, 100, the data to be compressed is fetched from a source (not shown). The data may be analogous or digital. In case of analogous data it is converted to digital data. The digital data is then converted, 101, to an "image representation" of the data. The procedure may for example check, 102, value of each bit and then each bit representing binary 1 is converted to a black pixel (e.g., ON - the state of an image memory cell), 103, and each bit representing binary 0 is converted to a white pixel (OFF), 104, or vice versa. The product is a bi-level image containing two "colors." The conversion is executed until all bits are converted, 105.

The result of the conversion is stored as a digital image representation or it is sent to the next step. After the conversion procedure, any kind of image compression can be applied to the image representation (IR) of the data set, 106. The compression may be carried out, for example, by any of compression techniques TIFF (Tag Image File Format), GIF (Graphics Interchange Format), TARGA, PCX (PC Paintbrush), JPEG (Joint Picture Express Group) etc., or other loss less compression methods, as long as some kind of compression is applied on the data-representing image.

Generally, loss less compression looks for areas containing pixels of the same value and encodes the area, which allows a moderate compression rate while still ensuring that a decompressed (extracted or expanded) file will look identical to the original file.

Preferably, TIFF format is used due to versatility and compatibility. It works under several operative systems and supports substantially any picture bit depth and various types of compression.

Also, the GIF form is very interesting for its high compression rate and can be used for 1- to 8- bit images. When an image representation of a data set is obtained, it is possible to reduce the size of the image (representing the data set), which results in a compressed image where "new" bits are placed next to each other so that a new compression can be applied on the new bit combinations. This procedure, not limited to GIF technique, can be executed several times.

Moreover, in many cases Lossy Compressions, such as for example MPEG technique (Moving Picture Express Group) or MJPEG (Moving Joint Picture Express Group) etc., can be employed. The basic MPEG video compression technique is based on a macro block structure, motion compression and conditional replenishment of macro blocks. This compression could be used on varying data, such as speech or the like after conversion to a data representation image or motion picture. It is also possible to select blocks of data and arrange them as representing-image sequences, similar to picture frames in a motion picture. Then a suitable compression, e.g., GIF, MPEG, MJPEG can be applied to the sequences, which may compress several frames (data blocks) to one frame (data block).

The compression is carried out until no further compression is possible, 107. The compressed data is delivered, 108, to another location, e.g., for storage, transmission etc.

Before using the compressed data, it must be decompressed to the original data. For this purpose, the compressed data set is supplied with a header, identifying the type of the compression used and number of compression loops. The decompressed IR is then decrypted to the original data stream by converting the pixels to binary data.

An embodiment of the invention is illustrated in fig. 2, intended for use, e.g., in a computer unit, such as PC or workstation, or any other arrangement with a need for data compression. The compression module 10 in this case includes an input/output interface arrangement 11, converter unit 12 and compression unit 13. The interface unit 11 is arranged to read data from a storage unit 14 or a data stream 16. The interface may function as a buffer to adapt the incoming data stream rate to conversion/compression rate. It can also include A/D-D/ A- converters to convert the input/output signal to digital/analog form. The interface unit supplies the converter unit 12 with the incoming data, which converts the data to an image representation, which is then passed to the compressor unit 13 applying a compression algorithm on the image representation. The compressed data is then send to the output interface 11 to be sent to a storage unit 15 or outputted as a compressed data stream 17. Although, the above embodiment is a hardware implementation of the invention, it is obvious that parts of the invention can be implemented as a software application.

Referring now to figs. 3 and 4, an embodiment of a communications network, preferably a telecommunications network and a data compression method will be described as an example.

The communications network 20, for example, includes one or several telephone connections 21a, 21b (cellular or stationary) or computer units 22a, 22b and transceiver means 23a, 23b connected through a network 24.

The compression arrangement according to the invention may be implemented in any of the devices attached to the network, i.e., telephones, computers etc., but in this case it is arranged in the transmission means 24, which can include switches, routers etc.

According to fig. 3, the digital (data) traffics from a telephone 21a or computer 22a is firstly converted, 301, into pixels or, for example, ASCII characters (hexadecimal notation). The advantage of using character-representation of data is that the characters can be modified or "destroyed" by removing sufficient parts of characters so that each character can still be visible and recognizable but "fuzzy". Then a lossy compression, for example MPEG, MJPEG or the like, can be applied on the data, i.e., the character-representation of data and the fuzzy data takes less place than the bit-information applied to the data.

The converted data is then packed, 302, building an image representation of the data. Then one (or several) compressor procedure(s) is(are) applied, 303, onto the packed data. The result of the compression is then sent to the conversion stage converting it to a new image representation. The loop 304, conversion-packaging-compression-conversion, is executed a number of times until no further compression is possible. The number of loops may also be predetermined.

The final compressed product is then provided with a header, 305, containing the number of compression loops and type of compression(s) applied. Some compression procedures insert a signature identifying the compression algorithm. This can be eliminated, for example to reduce an unnecessary data amount. In case where the data streams are divided into smaller compressed packets, according to the invention, the header may also be supplied with sequential packet identification numbers.

It is also possible to convert each data package to an image frame representing each package. Then by arranging the frames in sequential package order a motion picture is obtained which can be subjected to a suitable compression technique, e.g., MPEG, MJPEG etc., so that a further compression is obtained. Of course, also here a loop of conversion-compression can be executed.

To speed up the compression loop the conversions and compressions may be carried out in parallel.

The compressed data is then transmitted, 306, to the addressee in conventional ways through the network 307.

At the receiver site the transmitted compressed data is received, 308, by the receiver 23b. The received data is decoded, 309, and the header information is identified. The data is decompressed, 310, according to the information contained in the header, i.e., the decompression procedure(s) to be used for decompression and number of decompression loops 311. Then the decompressed image is decoded, 311, by converting the pixels to binary data or characters. Finally, the data packets are converted to a digital data stream before delivery to the receiver 21b, 22b.

The conversion and compression procedures can be implemented both through hardware or software in any part of the communications network as shown.

Yet, another advantage obtained by the invention is the possibility of encrypting large amounts of data. The encryption is achieved by anonymously compressing data, i.e., compression without entering the compression type in the compressed data. Each time a certain compression is applied to a set of data, the original data is changed due to compression. Applying several types of compression algorithms on a set of data in a number of times results in a data set which is not recognizable without the knowledge of which type(s) of compression algorithm(s) and number of times the data set has been compressed by each algorithm. Then, it is possible to encode type(s) of the compression algorithm and the number of compressions in the header of each compressed data set. A user having right keys can then decode the compression types and times and decrypt the data. The encryption may be applied in both cases shown in fig. 1 and fig. 3.

The invention is not limited the shown embodiments but can be varied in a number of ways without departing from the scope of the appended claims and the arrangement and the method can be implemented in various ways depending on application, functional units, needs and requirements etc.

Claims

1. Method for compressing a compressed or non-compressed digital data set, characterized in, that the method comprises a first step of converting the data set, which can be any digital data set represented by a digital representation such as, ASCII characters, Hex, Octal or the like, into a representation of the data set, said representation being a pixel representation, in which a binary one is represented by an active (black) pixel and a binary zero is represented by an inactive (white) pixel or vice versa and a second step, in which a compression procedure is applied onto said representation of the data set at least one time.

2. The method according to claim 1, characterized in, repeating said first and second steps, preferably until a maximum compression ratio is achieved.

3. The method according to claim 1, characterized in, that the data set includes binary data and the data representation is a pixel representation, in which a binary one is represented by an active (black) pixel and a binary zero is represented by an inactive (white) pixel or vice versa.

4. The method according to claim 1, characterized in, that the representation is modified by removing sufficient parts of characters so that each character can still be visible and recognizable but fuzzy, whereby the fuzzy character takes less place than the bit information of the ASCII-character, and that a compression, preferably a lossy compression, is applied on the character-representation of data.

5. The method according to claim 1, characterized in, that the data set is divided into blocks of data and each block is converted to an image frame representing the data block.

6. The method according to claim 5, characterized in, that a compression is applied on several frames, preferably resulting in fewer frames.

7. The method according to any of claims 1-6, characterized in, that the compressed data set is provided with a header,

8. The method according to claim 7, characterized in, that said header includes at least information on number of compressions applied.

9. The method according to any of proceeding claims, characterized in, that conversion and compression are carried out in parallel.

10. An arrangement for compressing a compressed or non-compressed data set, characterized in, that arrangement includes interface means (11) to receive data set to be compressed, conversion means (12) for converting data to a form representing the original data set, compression means (13) to compress the data representation.

11. An arrangement according to claim 10, characterized in, that the arrangement further includes means to execute a loop of conversion-compression.

12. Arrangement according to claim 10 or 11, characterized in, that means are arranged to convert data to or from digital form.

13. A communications network, including devices (21a, 21b, 22a, 22b) for generating digital communication data, transceiver device (23a, 23b ) and networking arrangement (24), characterized in, that the networking arrangement (24) and at least one of said devices (21a, 21b, 22a, 22b, 23a, 23b), include means for converting communication data to a representation of the data and means to compress the data representation at least one time before transmission.

14. A network according to claim 13, characterized in, that said representation is a pictorial representation of the communication data.

15. A network according to any of claims 13 or 14, characterized in, that communication data is converted to packages which are converted to data-representing frames arranged in sequences and said compression is applied to the several frames.

16. A network according to any of claims 13 - 15, characterized in, that said compression is one or several of MPEG, MJPEG, JPEG, GIF, TIFF or the like.

17. Method for encryption of a data set containing binary data, characterized in, that said method includes the steps of:

- converting the binary data in a first stage to an image-representation of the data set,

- compressing said image-representation of the data set in a second stage,

- repeating said first and second stages at least a number of times, and - creating an encrypted data set having encrypted compressor identity and number of repetitions.

18. The method according to claim 17, characterized in, that different compression techniques are applying.

19. The method according to claim 18, characterized in, that said compression is one or several of MPEG, MJPEG, JPEG, GIF, TIFF, TARGA, PKZIP,

LZW or the like.