SE528172C2 - Digital image processing method for mobile phone, involves performing Huffman decoding of preset number of coefficients of data unit, so that rest of coefficient is skipped by jumping to next zeroth/first order coefficient in bitstream - Google Patents

Abstract

The zeroth order coefficient of each data unit of an image block, is accessed. The Huffman decoding is not performed/performed to preset number of coefficients of data unit, so that the rest of the coefficient is skipped by jumping to next zeroth/first order coefficient in bitstream (36) using information (32) related to number of bits in bitstream between coefficients zeroth/first order in adjacent data unit. Independent claims are also included for the following: (1) image representation format; (2) method for encoding raw image data into compresses digital image representation; (3) method for analyzing JPEG-compressed digital image; and (4) method for stitching two digital images.

Description

20 25 30 35 * IIIIIIIIIIIIIIIIIIIIIIlill ---- IIIIIIIII-ll 528 172 2 images are the JPEG (Joint Photographic Experts Group) standard. The JPEG standard is defined in CCITT Rec. T. 81.

For the sake of clarity, however, a brief description of the image file format according to the JPEG standard is presented below.

The JPEG standard defines a destructive baseline coding system, which system is based on the DCT transformation, and an extended coding system for presenting the transformed image in smaller amounts of data. When converting a digital image to the JPEG file format, a DCT transformation and quantization of the image is performed, whereby each component of a color space model for the image is separately DCT-transformed. All color components are represented as blocks, which are processed sequentially. The DCT-transformed blocks are thresholded and quantized so that information from basic functions that have a low impact on how the image is perceived is rejected. The zero order coefficient (DC coefficient) of each component of each block is stored as the difference to the previous DC coefficient using Huffman coding. The higher order coefficients (AC coefficients) are arranged sequentially, the sequence being obtained by a zigzag order from the array. The AC coefficients are zero run length code (zero run length code) and are also coded with Huffman coding.

The JPEG file format is designed to create a standard compression that significantly reduces the storage size of a digital image. Thus, the JPEG file format is not suitable for image manipulation. If there is a desire to process a digital image, it is most convenient to transfer the digital image back to a representation in the space domain. However, when processing images on a device that has a small storage space, such as a mobile phone, it is not possible to handle with certainty the large storage requirement of the digital image represented in the space domain. l fi fšê fl öë - iê 30:56 1 = ': \ jicwšrganisaticrfxstàïuëlïö âB \ PATFïï ~ 1'P \ __ lS-o} ïarnily \ SE \ 2LOZüYBSXZIOZíY / * Išö AppïJcatinnfextïl 30 Icn 30 x8 25 I EP 1 037 165 describes a method for manipulating digital images stored in JPEG format. The bitstream of the JPEG image is scanned to identify positions for image areas in the bitstream. Certain designated modes are stored in a pre-scan table for easy access, allowing selected areas of the image to be accessed without the need to decode the entire bitstream when manipulating a portion of the image. However, there is still a need to further increase the speed of image processing while maintaining low memory requirements.

SUMMARY OF THE INVENTION It is an object of the invention to provide compressed images that can be easily analyzed and / or manipulated. It is a further object of the invention to provide an opportunity to easily merge digital images into a compressed image representation format.

These and other objects of the invention are achieved, according to a first aspect of the invention, by a method of processing a digital image. The method comprises: providing the digital image in a compressed format, in which the digital image is represented as a bitstream representing sequential image blocks, each block comprising one or more components, each of which comprises one or more data units and each data unit is represented as a Huffman-coded stream of base function coefficients, and a zero order coefficient is represented as a difference to the previous zero order coefficient of the corresponding component, and a block information table, comprising: indicators of a zero order or first order coefficient of each image block in the bitstream, information indicating the number of bits in the bitstream between zero or first order coefficients in adjacent data units of the image block and the zero order coefficient for at least one “3006 ~ f; 6 ~ 29 10:56 '=. =': [blo fi xgarxisatiormscialšxßi ) AE- \ ¥? P.'SEZ \ IT \ V_I: 1oFamily \ SE \ 2l O20? 3í fi iïlcàüïïšeš àpp ~ lic'ati'nritszt'l “olnstrutrtczi" CAS ïüíjí - ßß-ll Laws. 10 15 20 25 30 35 528 172 4 data unit of each component, said zero order coefficient being represented in a non-differential form. The method further comprises for each data unit of at least one image block: retrieving the zero order coefficient from the data unit and Huffman decoding no or a predetermined number of coefficients of the data unit, skipping the rest of the coefficients by skipping to the next zero or first order coefficient in the bitstream by means of the information in the block information table regarding the number of bits between coefficients in adjacent data units in the bitstream, whereby a reduced set of Huffman-coded coefficients is decoded.

In the context of this application, the term "image block" is to be construed to represent a portion of the space of an image, said block being capable of having information from various color space components. Each image block can be represented as one or more sets of coefficients for each color space component.

According to a second aspect of the invention, the objects are achieved with an image representation format for representing a digital image. The image representation format comprises: image information stored as a bitstream representing sequential image blocks, each block comprising one or more components, each comprising one or more data units and each data unit being represented as a Huffman-encoded stream of coefficients of basic functions , and wherein a zero order coefficient is represented as a difference to the previous zero order coefficient of the corresponding component; and a block information table, comprising: indicators of a zero order or first order coefficient of each image block in said bitstream; information indicating the number of bits in the bitstream between zero or first order coefficients in adjacent data units of the image block; and the zero order coefficient for at least one Û Û EMEê-Z 9 1813 * É V: \ __ I ~ 1 -.>! § «1¿¿: 1r; l: hat ionï SCEÄLF-: IÜ šB \ PÉ- * CPEXGTX NO -Faï fi i lf; '\ š'-3E “~ \ 2 1 Olí fi fi 2: 1 3,2" LOLÉ 1373 ëPlßllcatiønt: ztToïnstrnfrf-a iïß-.ß SU- ”JE-TJB-ll L-'lo-c 10 15 Data unit of each component, said zero order coefficient being represented in a non-differential form.

According to a third aspect of the invention, there is provided a method of encoding raw image data into a compressed digital image representation. The method comprises: arbitrarily reading image blocks of a specified size of said raw image data and for each image block: transforming the image block into one or more data units for one or more components, said transformation creating a representation of each data unit as coefficients of basic functions; calculate a quantized approximation of said coefficients; representing at least some of the quantized coefficients as a stream of coefficients of sequential image blocks; Huffman encodes said stream of coefficients, wherein a zero order coefficient is represented as a difference to the previous zero order coefficient of the corresponding component; storing said Huffman-encoded stream of coefficients in a bitstream; storing in a block information table indicators to a zero order or first order coefficient for each image block in the bitstream; storing in the block information table information indicating the number of bits in the bitstream between zero or first order coefficients in adjacent data units of the image block; and storing in the block information table the zero order coefficient for at least one data unit of each component, said zero order coefficient being represented in a non-differential form.

According to a fourth aspect of the invention, there is provided a method of analyzing a JPEG-compressed digital image, the JPEG-compressed digital image being represented as a bitstream, said bitstream representing sequential image blocks, each block comprising one or more components, which each comprises one or more data units and each 'Yïüê-üê-fr? 10:56 V: \ __ I »Ic-fï> 1'qar1is = atÅormSCALADO AB \ PATENT \ __ Nc = FamiJ.y“ -, SEX2lO2ü? 3š5 \ 2lÛ2ê736 ÄDplittativntsxtToI: astructor * JAS 3ÜüE ~ ü6 ~ 1l; Data unit is represented as a Huffman-encoded stream of coefficients of base functions, and a zero order coefficient is represented as a difference to the previous zero order coefficient of the corresponding component. The method comprises: sequentially stepping through the bitstream and during the stepping through the bitstream: storing an indicator in a block information table to a zero order or first order coefficient of each image block; decoding the zero order coefficients and storing in the block information table the zero order coefficient for at least one data unit of each component, said zero order coefficient being represented in a non-differential form; and storing in the block information table information indicating the number of bits in the bitstream between zero or first order coefficients in adjacent data units of the image block. The step through the non-zero order coefficients of a data unit in the bitstream, which non-zero order coefficients are represented by a sequence of bitstream records, comprises: looking at a bit sequence of a predetermined number of the following bits in the bitstream; making a table lookup for determining the category and the zero sequence length of at least the first bitstream record in the bit sequence and for determining the bit length of the first bitstream record; skipping the number of bits in the bitstream corresponding to the determined bit length; summing the number of skipped bits for collecting information regarding the number of bits in the bitstream between zero or first order coefficients in adjacent data units; and summing the number of coefficients that have been stepped through until all the coefficients of the data unit have been stepped through or a block end symbol is encountered.

According to a fifth aspect of the invention, there is provided a method of joining two digital images.

The method comprises: determining a relationship in the space between the two digital images; to assign image blocks itlüfš-ßš-S-ZQ 1? z 5 6 v: \ ._! ~ Ief.ï.1: gani ss-.at .if> n'xS (:? \ LADO AB ~ 2Pš \ TENT \ ._ NoFarnilyäßëšäâ1020736 \ 2l02 <ï> 736 àpnlicationtzzxt'ï ' digital image information in the two digital images index according to the ratio in the space between the two digital images; to form a bitstream; representing sequential image blocks according to the assigned indices, each block comprising one or more components, each comprising one or more data units and each data unit being represented as a Huffman-encoded stream of coefficients of basic functions; storing image block information for each image block in a block information table according to the position of the image block, said image block information comprising: indicators of a zero order or first order coefficient of each image block in said bitstream, information indicating the number of bits in the bitstream between coefficients of zero e or the first order in adjacent data units of the image block, and the zero order coefficient of at least one data unit of each component, said zero order coefficient being represented in a non-differential form.

Thanks to at least some of the aspects of the invention, a digital image is represented in an image representation format that requires little storage space, while the digital image in the compressed representation format can still be easily processed. This is especially useful for applications with small storage space and low computing power, such as mobile phones. The invention enables a digital image to be stored in a compressed format, but can still be processed and manipulated in real time while being presented on a screen. Thanks to the image representation format and the method of processing an image according to the invention, the image can be presented very quickly on a reduced scale or with a reduced resolution. The storage of the zero order coefficients of the image blocks means that the zero order coefficient does not need to be calculated using information of the previous zero order 20Ü6 ~ <1 = 6 ~ 29 10:56 VH. Noöšrgani.satíonëSCàLADO AMPATEIJTX IiøFamilyKSEVÉ1Û2ü736 \ 2l () 213736 F-.pplicatisrltcxtTcïaistrucfcox -IAE: IG-HE ll Lrloc _ 10 15 20 25 30 35 528 172 8 coffic. Furthermore, any desired number of non-zero order coefficients can be decoded. The rest of the non-zero order coefficients in the bitstream can be quickly skipped because the block information table provides bit length information between coefficients of adjacent data units, enabling fast access to the next data unit. This means that the image can be decoded quickly on a reduced scale, since there is no need to decode the non-zero order coefficients in the bitstream to find the start of the next data unit in the image block or the start of the next image block. Thus, the image representation format allows quick access to a digital image.

The storage of indicators to a zero-order or first-order coefficient of each image block provides quick access to particular portions of the images, without the need to decode the Huffman-encoded stream of coefficients from the start of the stream.

Instead, the image block can be accessed directly using the indicator. Furthermore, the zero order coefficient of at least one data unit of each component is represented in a non-differential form in the block information table. Thus, the need to calculate the value is avoided. This allows the presentation and manipulation of parts of it from the Huffman-coded stream of coefficients. digital image, while it is in the compressed image presentation format, since parts of the image can be accessed and analyzed at random.

In accordance with the image representation format of the invention, the goal of compressing a digital image to a minimal size is somewhat eased. Thus, the size of the digital image is not optimally compressed, but instead some additional information about the Huffman-encoded stream of coefficients is stored separately to enable rapid acquisition of certain portions of the image. In particular, it is possible to decode the image or parts of an image very quickly on another scale by iüüê-Lmi-ZS * 10:56 v: “xjlasïlrganisaticlnxåflataïëâ) AB \ E> š -.“ ». 'I:; 2-ZT \. léoFaznily \ SE \ 21020736 \ 2lC2 & 736 Appzlifatiorrtextiïfoïrzstructor CAS 3305-08-11 Ldoc 10 15 20 25 30 35 528 1 '72 9 Huffman decoding only a fraction of the non-zero order coefficients and using an inverse discrete fit industry form to a smaller block, such as 4 x 4, for calculating a smaller size block. The image representation format can be easily transferred to a JPEG image, since the image representation format is very similar to the JPEG image format. Thus, it is possible to perform the transfer to a JPEG image and remove the indicators and the stored values of the coefficients, when the image does not need any further manipulation.

The Huffman-coded stream of coefficients need not include the zero-order coefficients stored separately in the block information table. However, the Huffman-encoded stream of coefficients can be in all This may be appropriate if the image representation format is to be transferred to another image format, since the bitstream itself can then be used directly in the second image format.

The term "block information table" should not be interpreted strictly as a table, but only as the fact that cases include all coefficients. Thus, for example, the block information table may be divided into several lists or tables.Furthermore, "Huffman-coded stream of coefficients" does not necessarily mean that the whole stream is Huffman-coded.The stream may include Huffman codes mixed with raw data for the coefficients. For example, in a JPEG compressed file, a zero sequence length and category of coefficients are Huffman encoded and the value of a coefficient within a category is described by uncompressed bits.

By storing indicators and coefficients in a block information table, a distinct structure is obtained to store the image information. 20f26 ~ O6 ~ Z9 read V9-, ëIQOr-'Ltanisationï fi tàlëaïšàï ABU / ATENTK. Iiøïamilv '\ SE \ 2ïO2ü73¿í \ 2] 929736 åvplicanfsw + ax + ma1ns: rutt @ r mas: ena-os-11 1.aac "10 15 20 25 30 35 528 1:72 10 An image block can, for example, include three Representation of the digital image in one luminance component and two color components means that the color components can be represented in a lower resolution without actually losing any information perceived by one eye. the information in the digital image is provided relative to a representation of the digital image sdm three color components.Using such compression, each image block may, for example, comprise four data units for the luminance component and one data unit for each color component.The data units are arranged sequentially in the Huffman coded the stream of coefficients.

According to a sixth aspect of the invention, there is provided an image representation format for representing a digital image. The image representation format comprises: image information stored as a bitstream sdm representing sequential image blocks, each block each comprising one or more components, each comprising one or more data units and each data unit being represented as a Huffman-encoded stream of coefficients of basic functions, wherein a zero order coefficient is represented as a difference to the previous zero order coefficient of the corresponding component, and bitstream information stored adjacent to the bitstream, which bitstream information includes information indicating the number of bits of each data unit in the image blocks.

This image representation format requires a relatively small storage space. The image representation format contains only bitstream information that enables rapid creation of a block information table in accordance with the image representation format of the second aspect of the invention. Therefore, this image representation format according to the sixth aspect of the invention is suitable for long-term storage of an image. When the image is retrieved, Il1fJ6-1 ”J6 ~ 29 l fi zbá- V: \ __ låow3rgar1is ~ ationXSIiRLâDO AB \ PATENTXjJoFatHilyKSEXZIC2IE736 \ 2lO20736 Åpplicati fi vrlteztïtzlxïüllcfs

The information indicating the number of bits of each data unit in the image blocks can be used for quick access of the data units. When the data units are retrieved, the indicators can be created, the information indicating the number of bits between zero or first order coefficients of adjacent data units can be collected and the zero order coefficients can be decoded to store at least one zero order coefficient of each component in a non-differential form.

The bitstream information may be compressed. This means that the image representation format according to the sixth aspect can be stored even more efficiently.

The indicators in the block information table may indicate the bit offset from a static position to the coefficient of said specified order. The static position can, for example, be the start of the bitstream.

Thus, the indicators are realized as pointers to a specific bit position in each image block. This means that each image block can be quickly retrieved, thereby speeding up access to specific parts of the Huffman-encoded stream of coefficients.

Alternatively, the indicators in the block information table may indicate the bit offset to the coefficient from a benchmark in the bitstream. The image representation format can then also include a list that provides information about in which image block each benchmark in the bitstream is located. In this way, the bit offset of the indicator can be stored with a smaller number of bits, since the bit offset from a benchmark in the bit stream is almost always smaller than the bit offset from a static position. This means that the indicators can be stored using less memory. Instead, a list is needed that provides information about in which image block each benchmark in the bitstream is located. When a specific block of images is to be accessed, the following is done '2006- (16-29 10:56 V: Lïießïwgariiz- »atioIYxSCALADO AEWATENTLNOFæni13'“ ~, SE \ 2lO2ü? 36 \ 2ï02ü? 25 30 35 528 172 12 first a check in the list so that the last benchmark in the bitstream before the specific image block is found, then the information about the relevant benchmark in the bitstream and the bit offset from the benchmark is used to find the desired image block in the bitstream.

The indicator can point to a zero-order or first-order coefficient of an image block. When the indicator points to a zero order coefficient, the start of a data unit can be easily accessed. Then the zero order coefficient needs to be decoded to access the non-zero order coefficients, even though the zero order coefficient may already be stored in a non-differential format in the block information table.

When the indicator points to a first-order coefficient, the non-zero-order coefficients can be accessed directly.

The information indicating the number of bits in the bitstream between coefficients of adjacent data units may further indicate the number of bits between any combination of zero order and first order coefficients. This information can be used for fast jump in bitstream from a data unit to an adjacent data unit. As described above, the information of a data unit can be easily accessed either by accessing the zero order coefficient or, when the zero order coefficient is known from the block information table, by directly accessing the first order coefficient. Therefore, the information indicating the number of bits can either indicate the number of bits to the zero order coefficient or to the first order coefficient. Likewise, the image representation format is structured in such a way that it is convenient to indicate either the number of bits from the zero order coefficient or the first order coefficient.

The indicator can also point to a coefficient in any data unit in the image block. Then can 'Exïfßê-üê-Z? 'Líëzišiï iFz \ __ Noñrgarils-: 1t ionkišfilALšxf fl í) AB \ PATEI-Iï fl jlilïarnllyESERZ102 (!? 36 \ 2102 $'? 3G Eepliciati-T-rltextTolnstz-uctfn. 'ÅAS 205 25c 36 l *. Furthermore, the zero order coefficient of the data unit to which the indicator points is preferably stored in the block information table. The zero order coefficients of the other data units of the component can It is appropriate for the indicator to point to the zero or first order coefficient in the first data unit of the image block, thus direct access to the start of the image block is used .

The block information table may include the zero order coefficient represented in a non-differential form for each zero order coefficient represented in the bitstream as a difference to a zero order coefficient of a previous image block. This means that each image block can be accessed independently, since the block information table provides all the information presented in the bitstream as dependent on the previous image block. According to a specific embodiment, the block information table includes indicators indicating the bit shift of the coefficient from a benchmark in the bitstream and only the zero order coefficients in non-differential form for coefficients represented in the bit stream as a difference to a zero order coefficient of a previous one. image blocks. This embodiment provides a block information table of a small size, which is advantageous when storage capacity is limited.

Alternatively, the block information table includes each zero order coefficient represented in a non-differential form. This means that there is no need to calculate the zero order coefficient for data units, whose zero order coefficient Züüê - ízi-'VPQ 10:56 \ 1: \ __ t ~! => O1'ç% fi swïtionßííàlšxßü AEXPÅTBNTK šJoFalnily \ SE \ ¿'LO3Ü736X21Û2Ü? 36 AppliusatioïztaztToïnsLruuiøi. Iüßï - fšzš-ll 1.310: __ 10 15 20 25 30 35 528 172 14 is represented in the bitstream as a difference to the zero order coefficient of a data unit within the image block. Thus, the information of the image block can be accessed more quickly. However, more information is required in the block information table.

Furthermore, the bitstream can represent the digital image in the JPEG format. Thus, a standard JPEG compression or an already compressed JPEG image may be associated with additional information for quick access, manipulation, and / or analysis of specific parts of the image.

According to an embodiment of the method according to the first aspect of the invention, the method further comprises presenting the decoded image blocks to a data handling or data presentation unit, whereby the image or a part of the image is presented on a reduced scale.

The data management or data presentation unit may be, for example, a monitor, a printer or a hardware unit for performing image processing. The image or parts of the image can be quickly presented on, for example, a screen, since the method decodes the relevant parts of the digital image for presentation on the screen in a very fast way. This means that the image can be presented to a user without the user experiencing annoying waiting times.

The method may further comprise performing calculations for image processing on the decoded image blocks. In this way, fewer calculations are needed because the image is decoded into a reduced set of Huffman-coded coefficients. Therefore, the image processing can be performed relatively quickly.

The method may further comprise presenting results of the performed calculations on a screen when the calculations are performed. The method provides presentation of an image stored in a compressed format in real time when the image is selected and presentation of manipulations of the image in real time, as the method provides a very fast way to access relevant parts of the image and fi PíI- “ëfßš-ZQ 93: 56- V: * pjlo fl ïrqazïisitLonESíIALADG AB \ E'ATEI-IT \ _ ~ IIoFanfilyRSE \ 2l02f173i5 \ 2lC20736 àppticarn * nr-taztToïrrstruct-oi "'SAS ÉGüE-'Jíoc-52 8 15 2 L' 25 decode them into a small amount of data so that manipulation can be quickly performed and presented.The method further allows the definition and execution of manipulations of images on a device having a small memory space, such as a mobile telephone.

According to one embodiment of the method according to the first aspect of the invention, the number of coefficients of the data unit used by Huffman is decoded to approximate a decoded image block corresponding to a larger number of coefficients. The approximation allows the Huffman-decoded coefficients to represent more coefficients, whereby a lower quality image can be created or a smaller image that has fewer pixels per image block can be created. Even if the approximation loses the finest details of the image, the quality of the image can still be satisfactory, since the rough features of the image are represented by the first coefficients in each data unit.

The predetermined number of coefficients that Huffman decodes can be, for example, four, nine, thirteen, eighteen or twenty-four. When four and twenty-four coefficients are decoded, respectively, the decoded coefficients represent a scaling of each image block from 8 x 8 pixels to 2 x 2 pixels and 4 x 4 pixels, respectively. When nine, thirteen or eighteen coefficients are Huffman decoded, the decoded coefficients can be used to approximate a representation of an image block in 4 x 4 pixels. These numbers of decoded coefficients are particularly suitable, since the coefficients immediately following the ninth, thirteenth and eighteenth coefficients cannot be used as information in a 4 x 4 pixel image block.

According to an embodiment of the image representation format according to the second aspect of the invention, the indicators, the information indicating the number of bits between coefficients of adjacent data units and the zero order coefficients represented in a non-differential form are stored in a block in block information lüfßfš-Oê fl ïñ 50:56. sš '' ''. 1,311: 10 15 20 25 30 35 528 172 16 tables. In an alternative embodiment, the indicators, the values of the coefficients and the image information are stored in separate memory areas.

According to one embodiment of the method according to the fourth aspect of the invention, each bit sequence viewed comprises sixteen bits. This is especially suitable for the format of bitstream records. A bitstream record encoding the non-zero coefficients consists of two parts. The first part is Huffman-coded and encodes the zero-sequence length and the category for the value of the current coefficient.

The second part is data that represents the value of the current coefficient. The first part of the bitstream record thus contains information about how many coefficients are coded by the record (zero sequence length + 1 coefficient) and how many bits the bitstream record consists of. Since the first part of a bitstream record is a maximum of sixteen bits long, it is advisable to look at sixteen bits. Thus, when looking at sixteen bits at a time, each bit sequence will always comprise at least the first part of a bitstream record and the sixteen bits will therefore contain information about the number of coefficients encoded by the bitstream record and the length of the bitstream record.

The lookup in table may include doing a first table lookup for the first eight bits of the bit sequence. If a table lookup is done for the sixteen bits at once, a table with 65,536 entries is needed, which consumes quite a lot of memory. Furthermore, the first part of the most common bitstream entries is eight bits or shorter. Therefore, a table lookup of the first eight bits will in most cases provide the information needed about the bitstream record.

The first table lookup can return information about the bit length of the first bitstream record and the number of coefficients passed through or return information to a second table lookup. In this way, the first table lookup will either lüíêê- fßë-ffê 10:56 X1: \ _ t ~ lo1ätgarlisxatiOnXSCALADO àB \ E'š = .TEi ~ 1T \ .__ IêoFatnilSASEÅZIOQÜ? 3G \ 2l02ü? Return the required information about the bitstream record or return information for analysis of the first part of the bitstream record using the last eight bits of the bit sequence.

The lookup in the table may further comprise doing a second table lookup for the last eight bits of the bit sequence to determine the bit length of the first bitstream record and the number of coefficients being stepped through. When a second table lookup is needed, the first table lookup can return a pointer to a table to be used in the second lookup depending on the first eight bits. There is only a need for a few different tables for the second table lookup, since there are only a few combinations for the first eight bits, where the first part of the bitstream record is longer than eight bits. Thus, when a table lookup is done in two steps, there is no need for as many table records for analysis of the first bitstream record of each bit sequence.

According to an embodiment of the method according to the fifth aspect of the invention, certain image blocks of one of the digital images are manipulated with information from the other digital image. The manipulation can be a mixture of effects on the image block from the two digital images, if the contents of the image block are imaged in both the digital images. This means that the two digital images can be more seamlessly joined together.

According to a further embodiment, image blocks of a part of a first digital image are first handled, while the image blocks of the rest of the image are temporarily stored in an uncompressed format. Then, these later image blocks can be used to calculate the manipulation of the second digital image, before the image blocks of the second digital image are handled.

The merged image can be further transferred to another image compression format, such as the JPEG format, by sequentially retrieving the image blocks of the .ïüüê-Ûë-'zå l fi zâê V: Agaplicfltioritemïfïoïnstructor CAB 'ZJOÉ-flê-íl Lrloc H 10 15 20 25 30 35 528 172 18 merged the image using the indicators of the image representation format and storing the stream of Huffman-coded coefficients for sequential image blocks with zero order corresponding coefficients corresponding to zero scheme coefficient. Thus, the digital images can be merged and manipulated while requiring little memory capacity, and when the merging is completed, the image can be moved to a representation format that is even better compressed.

Brief Description of the Drawings The invention will now be described in more detail, by way of example, with reference to the accompanying drawings, in which: Fig. 1 is a flow chart of a method for compressing a digital image according to an embodiment of the invention.

Figures 2a ~ c are schematic views of the image representation format according to embodiments of the invention.

Fig. 3 is a flow chart of a method of analyzing a data unit in the bitstream.

Fig. 4 is a flow chart of a method for reading a specific part of a digital image stored in a compressed image file format.

Fig. 5 is a flow chart of a method for decoding a data unit for manipulating a digital image stored in a compressed image file format according to an embodiment of the invention.

Fig. 6 is a flow chart of a method for merging two digital images into a compressed image file format according to an embodiment of the invention.

Fig. 7 is a schematic overview of the joining of two digital images.

Figs. 8-9 are screenshots of a device that captures digital images and merges the digital images into a compressed image file format. 'lüfß-šwê - Zšä 19:56 V: \ __ t ~ le = 2> xganis »aríonXSCALAEO ABXPATEIQT \ èloš- * amilj / XSEAZ1020736 \ 21021}? 36 Äpplicaciorntextík fl zstrutftor CAS 2005-08-11 20 Ldoc 25" 10 Detailed Description of a Preferred Embodiment In the following description, image compression will be described with reference to JPEG compression, although other compressions using other transformers are conceivable. to JPEG compression.

Referring to Fig. 1, a method of compressing a digital image will now be described. First, the digital image is represented in the YUV color model, step 10, with each pixel in the image having three components: luminance, Y and two color components U and V. The two color components U and V represent color features, the finest details of which are difficult to detect for the human eye.

Thus, these components can be represented in a lower resolution than the luminance component. A 16 x 16 pixel block of the image, which is an example of an image block as defined herein, may be represented by four Y x data units of 8 x 8 pixels each, a U x data unit of 8 x 8 pixels and a V data unit of 8 x 8 pixels. This corresponds to a 50% compression of the amount of data compared to an RGB representation of the digital image. However, four U-data units and four V-data units can also be used, whereby no compression is obtained compared to the RGB representation.

The image is treated as discrete 16 x 16 image blocks in any order. Each data unit of each component is transformed using the discrete cosine transform (DCT), step 12. Since each data unit of each component comprises 64 pixels, the DCT transform will generate 64 coefficients for the basic functions of the DCT transform. These coefficients include a zero order coefficient (DC coefficient) and 63 higher order coefficients (AC coefficients).

Then, a thresholded and quantized approximation of the coefficients for each data unit is created, step 14. 2006-136-29 19:56 V: \ _ I ~ I -;> Organizational sutíOxHSCALADO ABAPATENTäWlIaFamilï \ $ E \ Ii1O2O736 \ 2ï <320'73í5 F-.ppliczatiorrtezffslnstructoi CAS 'ZCIQÉ - Üß fl l l.df.-c: D10 15 20 25 30 35 528 172 20 a normalization matrix. This means that the coefficient for basic functions that have been determined to have low perceptual weight is given low weight and many coefficients are given the value 0 (zero).

Then the coefficients are reorganized into a stream of coefficients by means of a zigzag arrangement. According to standard JPEG compression, the DC coefficients are represented as the difference to the previous DC coefficient of the previous data unit by the same color component and the difference is Huffman coded. The AC coefficients are zero-sequence length-coded and further coded with Huffman coding and stored directly after the Huffman-coded DC coefficient. When there are only AC coefficients that have a zero value left in the data unit, a block end code is entered into the stream of coefficients. Thus, a bitstream of Huffman-coded coefficients for sequential data units and image blocks is obtained. According to one embodiment of the invention, the Huffman-coded coefficients are calculated in a similar manner as for standard JPEG compression, step 16. The bitstream constitutes a compressed representation of the digital image.

However, since a zero-sequence length coding and Huffman coding are used, the length of each data unit is unknown. Therefore, the start of a data unit is unknown until the bitstream has been decoded from the start of the bitstream to the start of the data unit. Furthermore, since the representation of the DC coefficient depends on the previous DC coefficient, the DC coefficient of a data unit is not known unless the previous DC coefficient has been determined by decoding the Huffman-encoded stream of previous coefficients.

To enable fast retrieval of specific blocks of the image and thereby manipulation and / or analysis of the specific block, a block information is created .ÜCfJG-ÛfS-Z? 10:56 ** '= \ _ ï ~} o1f =:' garis-aríonßíïšïlšæll fi àPfxPATENT \ g_ïisl “axnilyXSE \ 2l02Ü '? 36XèÉlOZU736 Applicatlorltextffc-ïzzatrutctor» CAS 1005-05-11 17 20 30 52 '5 , step 18, which includes an indicator for each image block, information indicating the number of bits in the bit stream between adjacent coefficients of a specified order and a DC coefficient for each color component of each image block, the DC coefficient being represented in a non-differential form.

According to other embodiments of the invention, the bitstream of Huffman-coded coefficients may have different contents or be stored in other ways. For example, since the DC coefficients due to a previous image block are already stored in the block information table, they do not need to be in the bitstream.

Furthermore, all information about an image block is known from the block information table, which includes the DC coefficients that depend on other image blocks and an indicator of the image block in the bitstream. Thus, the bitstream need not be represented as a specific sequence of image blocks or even be stored in a stream.

Thus, according to an embodiment of the invention, the information is stored in the block information table in a manner where the position of a block information in the table specifies which part of the image it represents.

The block information may also be stored in a manner where the position it represents is stored together with the other block information. The Huffman-encoded stream of image block coefficients may even be stored adjacent to the block information. The stored block information can also have a mechanism for determining whether the block has been coded, which makes it possible to process the non-coded blocks as, for example, black blocks.

A packet of the Huffman-encoded stream of coefficients and the block information table thus constitutes a representation of a digital image, which requires little storage capacity while enabling analysis and manipulation of specific parts of the image without the need to decode the entire image. This also enables compri- J "-06-29 19:56 V:" çlkwargarlisatíOnXSCALAQiD ABXPATEI ~ ITX èëoFamw'1.yRSERZlOEÜÉBSKxVZlOZD7BG ÅPL- * li fi fxticnram-ffnënl ln - ll 528 172 22 of the image in a non-linear manner, where the order in which the blocks are compressed is not essential.

Reference is now made to Fig. 2a, in which the structure of the compressed image representation format is presented. The block information table may be stored in the RAM of a device, while a random access for reading or writing image blocks is required by the device. As shown in Fig. 2b, the block information table 30 and the bitstream of Huffman-coded coefficients 36 can be stored in separate memory spaces. The block information table 30 includes the indicator 31, information 32 indicating the number of bits in the bitstream between zero or first order coefficients in adjacent data units and the DC coefficients stored in a non-differential form 33. The indicator 31 of the block information table 30 then includes an indication of the the first coefficient of a specified order of the image block in the bitstream 36.

As an alternative, as shown in Fig. 2c, the block information table 30 'may be stored as a header or as a special marker to the bitstream 36', which may be a standard JPEG format.

It will also be appreciated that when information is stored in the bitstream header, the header need not include all the information needed in the block information table.

Instead, the header may include information to allow the block information table to be created very quickly.

Thus, the bitstream head 36 'only needs to pour information about the lengths of each data unit. When the image is retrieved, the block information table can then be quickly created and loaded into RAM. Using the lengths of the data units, the information needed in the block information table can be quickly accessed. The data units are incremented while indicators are created to the start of each image block, the DC coefficients are decoded and stored and the information indicating the number of bits in the bitstream between the coefficient- ApplicationteztTcInstrucLul »TAE 20-'15-08-11 lead-c 10 15 20 25 30 35 528 1272 23 ter of zero or first order in adjacent data units are updated.

This head can even be compressed, eg Huffman-coded, so that storage space is saved. The length information has values that only slightly vary. Therefore, Huffman coding of the lengths in a manner corresponding to the coding of DC coefficients would substantially compress the information. The essential compression of the file means that the file can be stored in a space-efficient format while allowing the block information table to be created quickly. The head is then decoded when the image is retrieved.

It should be noted, however, that the invention is in no way limited to these presented ways of representing the image.

The indicator 31 can point to the first zero order coefficient of the image block. However, since the zero order coefficient is provided in the block information table, there is no need to access the zero order coefficient. Thus, the indicator 31 may alternatively point to the first order coefficient of the image block. The indicator can be implemented as a bit offset from a static position in the bit stream to the coefficient of a specified order in the image block. Preferably, the indicator provides information about the bit offset to the coefficient from the start of the bit stream.

The indicator 31 can alternatively indicate the bit offset to the coefficient from a benchmark in the bit stream. If the indicator 31 indicates the bit offset from the start of the bit stream, a bit stream of a size larger than 2 Mbytes would require indicators represented by 4 bytes. The benchmarks in the bitstream can be placed in the bitstream so that an indicator 31 can be represented by 2 bytes. When the offset becomes larger than, for example, 65,536 bits (the largest number represented by 2 bytes), a benchmark is registered in the bitstream. 23136--216--29 30:56 V: ”xjl-: DrganisatiornSCALAItë AB \ EA'JEI ~ IT \ IJcFarnilg fl sEïâl020736 \ 21020? 36 Äïëßl.Lcationteztïøïnstructor CAS .IÛIJE - ÜS-ll 15 xx 25 ' 528 172 24 The benchmarks in the bitstream are sequentially numbered. A list of benchmarks in the bitstream is created to provide information about which image block each benchmark in the bitstream is located in. When an image block is to be accessed using the indicator, access is made in two steps. First, a comparison is made with the list of benchmarks in the bitstream so that the number of the benchmark in the bitstream that is located closest to the bitstream before the image block to be accessed is found. Then the bit offset of the image block from the start of the bit stream can be calculated as (the number of the benchmark in the bit stream) * 65.536 + the bit offset provided by the indicator. Of course, any number of bits could be used between the benchmarks in the bitstream.

The information indicating the number of bits between adjacent coefficients of the specified order could be used for quick access of specific data units within an image block. There is no need to decode the previous data units to know where a data unit starts. This could be used to advantage for fast decoding of image blocks to a reduced set of decoded coefficients, as described in more detail below.

The block information table may include the DC coefficient represented in a non-differential form for each data unit. This means that there is no need to calculate the DC coefficient for any data unit.

However, the block information table may alternatively include only the DC coefficients represented in the bitstream as a difference to a DC coefficient of a previous image block. Since an image block comprises several data units for each component, there is only a need to store the DC coefficient of the first data unit of each component. This means that the block information table requires less storage space.

The structure of the compressed image representation format can also be obtained by starting from a ^ 70fJ6 ~~ l16 ~ 29 10:56 * JU-_ t-lf. “'GanisstionkSCšïr-ÅDO ABXPATBIJTë I fl olfarnil *; \ SE \ 2lOllïïßïtêlüäftVfšiš Applicationtf2ztï'oIn.st.r |.-_. Tcn: CAS Süüï - ÛB-il Ldo: 10 15 20 25 30 35 5-28 compressed JPEG file. The JPEG file is then decoded to determine the indicators for each image block, information indicating the number of bits between adjacent coefficients of specified order and a DC coefficient in a non-differential form for each component of each image block. Thus, the indices and DC coefficients are stored in the block information table, while the Huffman-coded stream of coefficients is kept intact.

The AC coefficients of such a JPEG image can alternatively be copied to a new memory space, whereby all information of the JPEG image will be accessible without the need to keep the original JPEG image stored in the device.

The image representation format created in this way opens up possibilities for performing image processing directly on a JPEG file.

A specific method of analyzing a JPEG image for creating the image representation format will now be described. The method involves sequentially stepping through the bitstream to collect the information needed. During the bypass of the bitstream, the indicators for each image block are stored, the information indicating the number of bits in the bitstream between adjacent coefficients of specified order and the DC coefficients of at least the first data unit of each component in the block information table.

During the traversal of the bitstream, the DC coefficients for the components are decoded and the last coefficient is stored temporarily to enable determination of the next DC coefficient.

Referring to Fig. 3, increment by AC coefficients of a data unit in the bitstream will be described. The analysis only wants to determine where the next data unit starts and the number of bits between the DC coefficients or the first AC coefficients of adjacent data units. Therefore, it is not necessary to decode the AC coefficients.

: Zív fl éé - Giï-Z? 10: 56- V: \ _ t <\ oOrganis = ationkßtïšalåïaü AB \ PATENT \ _IIc «FamiLyïSE \ .2102ü '? 3931020736 âpplicativnteztïcßïlxstrnctrir * SAS JGUE - êâ-lí Ldoc 10 15 20 25 30 35 35 the coefficients consist of two parts. The first part is a Huffman code that encodes how many zero coefficients precede the current coefficient and in which category the value of the current coefficient is. The category determines the number of bits for the second part of the bitstream record, which encodes the current value of the coefficient. Thus, by analyzing the first part of the bitstream record, the number of coefficients encoded by the bitstream record and the number of bits used by the bitstream record can be determined.

The step through the AC coefficients involves looking at a bit sequence of a predetermined number of bits in the bitstream, step 20. Preferably, the bit sequence 16 is a number of bits long, which corresponds to the longest first part of any bitstream record. Thus, by looking at 16 bits at a time, the required information can always be collected for at least one bitstream record.

The information is collected by making a table lookup, step 22. The lookup table thus returns the bit length of at least the first bitstream record that occurs in the bit sequence and the number of coefficients encoded by the Huffman code. A number of bits corresponding to the determined bit length is then skipped, step 24. The accumulated number of skipped bits is summed, step 26, and the accumulated number of skipped coefficients is summed, step 28. If a block end symbol has not been found and the maximum number of coefficients of a data unit has not been skipped, the process is returned to step 20 and a new bit sequence is watched.

The table lookup in step 22 can be performed in two steps. First, a table lookup is performed for the first eight bits of the bit sequence. This lookup is sufficient to determine the first part of the Huffman code for most lookups for normal JPEG images. If the first table lookup is sufficient to determine the first part of the first Huffman code, the last eight bits are not further analyzed at this stage. Om ÉOfJG fl EE-ffíï 1,015? - “. ~ ':“ X__E ~ IoOrqanis¿xtion \ SCALADQ QB \ E> ATEIV3T “-._ Ešaïaznil =,: \ 3Eï2lí) 20? 3öïllß fifi ßä äpplicasIrocrrz ( The first part of the Huffman code is longer than eight bits, an additional table lookup is needed, the first table lookup then returns a pointer to a new table, where a lookup of the last eight bits is to be done. of the first eight bits of the bit sequence.The second table lookup will then determine the number of bits for the Huffman code and the number of coefficients skipped.The second table lookup can be done in different tables or in different parts of the same table. few variants of the first eight bits of Huffman codes that are longer than eight bits, so the number of entries in the second table is limited and much less than 65,536 (ie the total number of possible bit sequences of sixteen bits).

Alternatively, the table lookup in step 22 is performed in one step. Then a table lookup is performed for a 16 bit code. This table lookup may in some cases return information about two or more Huffman codes. Where the bit sequence includes the first part of several Huffman codes, these Huffman codes can thus be skipped at the same time. It is also possible to skip multiple Huffman codes at the same time when an eight-bit lookup is performed.

However, only a few Huffman codes are short enough to provide the required eight-bit information.

Referring to Fig. 4, a method of reading a specific part of a digital image stored in the image representation format will now be described.

First, the position or area of interest in the image is defined, step 40. Then the correct image block or blocks corresponding to the defined position or area are determined by simply correlating the movement to the right and down from the upper left corner to the sequence of the image blocks, step 42 .

Then, the position of the correct image block or blocks in the bitstream is determined by lookup in the block information table, step 44. Further, DC- 20136416-29 “íüzßí-š V: to: CAS: Iilgïwßå-ll lxlsç: 10 15 20 25 30 35 528 172 28 the coefficient of the correct image block or blocks from the block information table, step 46. Then access the position in the bitstream, step 48, and decode the image block at this position, step 50, using the retrieved DC coefficient.

Referring to Fig. 5, a method of processing and manipulating a digital image stored in the image representation format will now be described. The processing and manipulation of the digital image is so fast that it can be performed and displayed on a mobile phone in real time to a user. Thus, the user can define the manipulations to be performed and see them performed within a few seconds.

The user can first define an image or part of an image to be displayed. To allow the image to be quickly decoded for display on a screen, only a reduced amount of Huffman-encoded coefficients is decoded for each data unit. The reduced amount of coefficients can be used to approximate the image with degraded resolution or to display an image with fewer pixels.

The number of AC coefficients decoded can suitably be zero, which reduces an image block of 8 x 8 pixels to 1 pixel, four, which reduces the image block to 2 x 2 pixels or twenty-four, which reduces the image block to 4 x 4 pixels. When the image block is reduced to 4 x 4 pixels, a smaller number of AC coefficients can be decoded to approximate the image block by 4 x 4 pixels. Thus, for example, nine, thirteen or eighteen coefficients can be decoded. Since the main information of each image block is placed in the first coefficients, the information lost in the non-decoded coefficients is not very significant.

Thanks to the image representation format, the reduced amount of Huffman-coded coefficients can be retrieved and decoded very quickly for displaying the image or part of the image. First, the image blocks to be decoded are determined, step 60. Access to each image block is given by Éïlflí-l-ü 6-29 'l C t 5 6 V: \. _i ~ lo 'fi> r gar: i s-ízt .i OIIXSCALÄILÄ šXE' - \ PäTEï fi Tï jIQFavnJLK yVEFS ïlï 'i 031373 6 \ 2 102 0736 Apr. i z ratio-rr: extïzsl nat rutter iÉAS 2 Q TJ fl- ~ 38 - ll l. the use of the indicators in the block information table, step 62. The DC coefficient of the first data unit of each component is also provided by the block information table, step 64. Then the desired number of AC coefficients is decoded , step 66. Then the next data unit of the image block is quickly retrieved by skipping the rest of the AC coefficients using the information indicating the number of bits between, for example, the first AC coefficients in adjacent data units, step 68. Now the next data unit can decoded. In this way, the image is decoded very quickly for presentation on a screen, which reduces annoying waiting times for a user. This is especially useful when using a device with low computing power and small storage space, such as a mobile phone.

The user can then define a manipulation to be performed on the image presented on the screen.

The calculations required for the manipulation can now be performed on the reduced amount of Huffman-decoded coefficients. Thus, the calculations can be performed faster and the results can be displayed on a display in real time, without the user experiencing long waiting times.

Referring to Figs. 6-9, a method of merging two digital images into a compressed image file format will now be described. Fig. 6 shows a flow chart presenting an overview of the method. Fig. 7 shows a schematic overview of the merging of images, while Figs. 8-9 show different steps in the method when it is implemented on a device for capturing images, such as a mobile telephone with a built-in camera.

First, a first digital image 200, step 100, is recorded.

The contents of the digital image are displayed in the viewfinder of a camera in Fig. 8. The captured image is then transformed so that lens corrections are taken into account and a projection of the corrected image on a cylinder is created.

Thus, the captured image is adapted to be merged with another image to create a panoramic image.

A section B of the digital image 200 is compressed "IilfJE-fšêïïf * 10:36 V: * rwtâcflïïgariisationïSCALAI-O AB \ PATEI ~ 1T \ __ lIoFa1nily \ SE \ 2l03073í5 \ 2lO2Ü? U.S. Pat. the second section A of the image is stored, step 104, with the intention of performing a blending operation later.If the user wishes to create a panoramic image, the direction of the panning can be defined in the image recording apparatus. the direction of the panning is then stored as the section stored with the intention of mixing.

This section A of the first image can be presented in the viewfinder of the camera, step 106, when a second image is to be recorded. The section for the first digital image can be presented in a first layer on the viewfinder, each other pixel being transparent so that the object space viewed by the camera and presented in a second layer on the viewfinder can be perceived behind the section of the first digital image, as shown in Fig. 9. If a panorama is made in a right-hand direction, a section on the far right of the first digital image will be presented in the part on the far left of the viewfinder. If the camera itself performs the lens correction and the cylindrical projection, as described above, in real time, the stored section A of the first image can simply be presented in the viewfinder. However, if no real-time correction is performed by the camera, section A is inversely transformed as if it were located in the far left position of the viewfinder so that it better fits the object space displayed in the viewfinder. The user is thus guided to record a second image 202, step 108, which positions corresponding objects in the first and second digital images in an overlapping area of the viewfinder. The captured image is then transformed so that lens corrections are taken into account and a projection of the corrected image on a cylinder is made. 20í16-O6 ~ 29 Ulwâê 259: _ï ~ 1f:. ~ 1 ') \' gar-isatíoIYaSCALAIFO .ÅBUTIPIEEEITX Z fl QPatUlJ.y \ SE * .¿1Q2Ü'I3ü * ßiê fififl ßö Äpplicaticrita-CAS-ust 2005; The second digital image now comprises a section C, which substantially corresponds to the section A of the first digital image. Thus, the camera can easily correlate the two digital images to each other, step 110, so that the two images can be correctly merged with each other.

Of course, the correlation between the two digital images can be obtained in any other way. For example, a computer unit can calculate and find the correlation between the two images or the correlation can be defined by a user, eg when there are no overlapping areas. The correlation determines a displacement of the images in relation to each other.

Then, the section A stored for mixing is mixed into the second image according to the correlation, step 112. Then, a section D of the second image with the correct index is compressed into the image blocks, according to the determined offset, into the compressed the image representation format as described with reference to Fig. 1, step 114. Further, another section E of the second image is stored in an uncompressed format, step 116, with the intention of being used in a mixing operation to be performed if additional images are to be recorded to the panorama. As shown in Fig. 7, section E may include portions which do not contain any image information, due to the displacement of the two recorded images relative to each other. The storage of the image section E thus has a mechanism for determining whether a pixel that is stored represents image data or unknown information due to the offset. This information is used later by the mixture and can also be used in the correlation operation.

Then, steps 106-116 are repeated if desired to merge additional images into the already merged images, step 118. The last inserted image is compressed in its entirety into the image representation format as described with reference to Fig. 1. ÉÛÜ_6--06-29 10 : 56 V: \ __ NoOrganisaïíonXSCÄLADC AB-VEATEiITXWNNOFaYnilyRESEXB'lO2f> 73 fi \ 2 Il.O20 '? 36 àppiic-atierltextffoïnstructoi CAS ÉWJDLI - Uê-ll Ldoc »10 15 20 32 32 det 32 är 32 25 32 25 possible to form from the two or more digital images and the first image block that completely fits into the rectangle in the upper left corner is determined. Then, the image representation format is converted to a JPEG image file of the merged images, step 120, by introducing the Huffman-coded stream of coefficients of the sequential blocks, starting from the determined block and moving from left to right and from top to bottom through it. definite, largest rectangular image. In this way, a large, merged image will be represented in a JPEG image file format. The merged digital images can now be displayed on a screen of the mobile phone that includes the camera. Although the method of merging has been described as a sequence in which a first digital image is first compressed into a compressed image file format and the second digital image is later added to this compressed image representation format, it is conceivable that the compressed image representation format may be created by the digital images are directly incorporated in the compressed image representation format or alternatively that the two digital images can each be represented in a compressed image file format and combined into a large image via a representation in the compressed image representation format.

In contrast to merging images, a method for cropping a JPEG ~ baseline coded image will now be described. First, the JPEG image is analyzed by decoding Huffman-encoded data. During this decoding, indicators for each image block and information indicating the number of bits between data units are stored in the block information table. The DC coefficient of the first data unit of each color component in each image block is further calculated and stored in the block information table.

Now the cropping of a JPEG image to a new JPEG baseline coded representation of a portion of the image is accomplished ¿'Cf. \ 21ÛI3f173í5 \ Z] 92f> f'3iš àpplirzaticnt fi. AzztTolzisrruzrzci: IAS INJÉ-Wß fl l Ld-: fc 10 15 20 25 30 528 172 33 by first determining an area to be retained from pruning. For the image block on the far left of each row of the area, a new difference for the DC coefficients needs to be calculated using the information about the DC coefficient in the block information table.

Then, the representation of the rest of the line can be simply copied bit by bit from the Huffman-encoded data of the original JPEG image to the new JPEG image using the indexes in the block information table to determine the bit length to be copied.

Thus, a new, cropped JPEG image can be created very easily.

If the cropping of the JPEG image is to be performed to an uncompressed representation of the image instead, the image blocks in the area are determined and decoded to the correct position in the uncompressed representation of the image using the indexes in the block information table. The decoding is performed using the DC coefficients in the block information table, so that all Huffman-encoded data does not have to be decoded at this time.

Furthermore, the JPEG image can also be manipulated by processing the image, while the image information is represented as the DCT coefficients. By converting the JPEG image to the image representation format presented herein, manipulations can be performed on the image blocks, while being represented as DCT coefficients. With the help of matrix operations, the image can thus, for example, be rotated or scaled.

It should be emphasized that the preferred embodiments described herein are in no way limiting and that many alternative embodiments are possible within the scope defined by the appended claims.

ZÜFJG-GQZC »10:56 V: ïmblofbrqanis-atiomSCsLADO RB”. Fi íaTàïïñ ^ ß \ _šlfnïarnilyäßßfi

Claims (35)

10 15 20 25 30 35 528 172 34 PATENT REQUIREMENTS A method of processing a digital image, the method comprising: providing the digital image in a compressed format, in which the digital image is represented as a bitstream representing sequential image blocks, each block comprising a or several components, each comprising one or more data units and each data unit being represented as a Huffman-encoded stream of coefficients of base functions, and a zero order coefficient being represented as a difference to the previous zero order coefficient of the corresponding component, and a block information table, comprising: indicators of a zero order or first order coefficient of each image block in the bitstream, information indicating the number of bits in the bitstream between zero or first order coefficients of adjacent data units of the image block, and the zero the order coefficient for at least one data unit of each comm component, said zero order coefficient being represented in a non-differential form, and for each data unit of at least one image block: retrieving the zero order coefficient for the data unit and Huffman decoding no or a predetermined number of coefficients of the data unit, to skip over the rest of the coefficients by skipping to the next zero or first order coefficient in the bitstream using the information in the block information table regarding the number of bits between coefficients in adjacent data units in the bitstream, thereby decoding a reduced set of Huffman-encoded coefficients. 'IUC-E fl bé-ZE? 10:56 V22jloürgar fl sationïSCALAšw AE- “xFATEHTLrJoFamilyïsßïï102fTI3äï2l021T136 ApplicatißnteztTcïzxaftrucrtor» SALä CÛÜE - CB-ll Ldoc 10 15 20 25 30 35 5218 The method of claim 1, wherein the digital image is provided in a compressed format and wherein the indicators in the block information table indicate the bit offset to the coefficient from a static position. The method of claim 1, wherein the digital image is provided in a compressed format, the indicators in the block information table indicating the bit offset to the coefficient from a benchmark in the bitstream, and wherein the digital image is further represented by a list providing information about in which image block each benchmark in the bitstream is located. A method according to any one of the preceding claims, wherein the digital image is provided in a compressed format and wherein the block information table comprises the zero order coefficient represented in a non-differential form for each zero order coefficient represented in the bitstream as a difference to a zero order coefficient of a previous image block. The method of any of claims 1-3, wherein the digital image is provided in a compressed format and wherein the block information table comprises each zero order coefficient represented in a non-differential form. A method according to any one of the preceding claims, further comprising presenting the decoded image blocks to a data management unit, whereby the image or a part of the image is presented on a reduced scale. A method according to any one of the preceding claims, which further comprises performing calculations for image processing on the decoded image blocks. The method of claim 7, further comprising presenting results of the calculations performed on a screen while the calculations are being performed. Method according to one of the preceding claims, in which the number of coefficients of the data unit which Huffman decodes 2006-ßë ~ 29 l fl: 56 Vzï Noürga application: ionrsx + Toïnstrnctor .ms “n \ SCALšDO BRXPATR $ TäñNoFami1y2 $ ER2l0lC2073ö -G8-ll l.dcc 10 15 20 25 30 35 528 172 36 is used to approximate a decoded image block corresponding to a larger number of coefficients. A method according to any one of the preceding claims, wherein the predetermined number of coefficients that Huffman decodes is four, nine, thirteen, eighteen or twenty-four. A method according to any one of the preceding claims, wherein the bitstream represents the digital image in the JPEG format. An image representation format for representing a digital image, the image representation format comprising: image information stored as a bitstream representing sequential image blocks, each block comprising one or more components, each comprising one or more data units and each data unit comprising a Huffman-coded stream of coefficients of base functions, and wherein a zero-order coefficient is represented as a difference to the previous zero-order coefficient of the corresponding component, and a block information table, comprising: indicators of a zero-order or first-order coefficient of each image block in said bitstream, information indicating the number of bits in the bitstream between zero or first order coefficients in adjacent data units of the image block, the zero order coefficient for at least one data unit of each component, said zero order cow fficient is represented in a non-differential form. and The image representation format of claim 12, wherein the indicators in the block information table indicate the bit offset to the coefficient from a static position. iUYJG-írê-fêä 13:56 V: \ ___ HesI> xgar: if: -ariomSKlALADO AB \ P. = \ TL'EI ~ 1T \ __ téok = 'smilyKSERLBl020736 \ 2lO2í_rï3¿5 ZappiicationtextTuJluctor S-lUJstructor CAS - : 10 15 20 25 30 35 528 17 2 37 The image representation format according to claim 12, wherein the indicators in the block information table indicate the bit offset to the coefficient from a benchmark in the bitstream and the image representation format further comprises a list providing information on in which image block each benchmark in the bitstream is located. The image representation format according to any of claims 12-14, wherein the block information table comprises the zero order coefficient represented in a non-differential form for each zero order coefficient represented in the bitstream as a difference to a zero order coefficient of a previous image block . The image representation format according to any of claims 12-14, wherein the block information table comprises each zero order coefficient represented in a non-differential form. The image representation format according to any one of claims 12-16, wherein the bitstream represents the digital image in the JPEG format. A method of encoding raw image data into a compressed digital image representation, the method comprising: arbitrarily reading image blocks of a specified size of said raw image data and for each image block: transforming the image block into one or more data units for one or more components, the transformation creating a representation of each data unit as coefficients for base functions, calculating a quantized approximation of said coefficients, representing at least some of the quantized coefficients as a stream of coefficients of sequential image blocks, Huffman coding said stream of coefficients, wherein a zero coefficient of representation represent- fÛÛí ~ C $ -29 Lüzbê V åpp1ieati: ntextToL: OxqanisationXSCåLâDO äšïPATENTK_ fl oFamily \ SE \ 2l02ü735 \ 3lO¿ü73ü uctor CAS ÉQJE-08-35-l2 las 15. unlike the previous zero order coefficient of the corresponding component, store said Huffman-coded stream of coefficient - cents in a bitstream, store in a block information table indicators of a zero order or first order coefficient for each image block in the bitstream, store in the block information table information indicating the number of bits in the bitstream between zero or first order coefficients in adjacent data units of the image block, and store in the block information table the zero order coefficient of at least one data unit of each component, said zero order coefficient being represented in a non-differential form. The method of claim 18, wherein the indicators in the block information table are stored as indicators of the bit offset to the coefficient from a static position. The method of claim 18, wherein the indicators in the block information table are stored as indicators of the bit offset to the coefficient of a benchmark in the bitstream and the image representation format further comprises a list providing information on in which image block each benchmark in the bitstream is located. The method of any of claims 18-20, wherein the zero order coefficient is stored represented in a non-differential form for each zero order coefficient represented in the bitstream as a difference to a zero order coefficient of a previous image block. An image representation format according to any one of claims 18-20, wherein each zero order coefficient is stored represented in a non-differential form. 'ÉÛCEfOG-íà? lêzäê V: 'awiiclïßnfganisationïs fi íàllšåïli) AB \ E * àTENT \ __; 1 -: \ Fau @ il3' \ SE \ 21020736 \ 210297315 applicatiantazfvñr fl nsfmatar lins: L fl-: xç-: x-15 30-ï5 '25-ï5' 25-ï8 25 I 72 39 A method according to any one of claims 18-22, wherein the bitstream represents the digital image in the JPEG format. A method of analyzing a JPEG compressed digital image, the JPEG compressed digital image being represented as a bitstream, said bitstream representing sequential image blocks, each block comprising one or more components, each comprising one or more data units and each data unit is represented as a Huffman-coded stream of coefficients of base functions, and a zero order coefficient is represented as a difference to the previous zero order coefficient of the corresponding component, which method comprises: sequentially stepping through the bit stream and storing through the bit stream an indicator in a block information table to a zero order or first order coefficient of each image block, decode the zero order coefficients and store in the block information table the zero order coefficient for at least one data unit of each component, said zero order coefficient nt is represented in a non-differential form, and storing in the block information table information indicating the number of bits in the bitstream between zeros or first order coefficients in adjacent data units of the image block, the passing of the non-zero order coefficients of a data unit in the bitstream, which said non-zero order coefficients are represented by a sequence of bitstream records, comprising: looking at a bit sequence of a predetermined number of the following bits in the bitstream, making a table lookup for determining the category and the zero sequence length of at least the first bitstream record in the bit sequence and for iüffš -üírï-Z? 10:56 \ J: \ __ í ~ IeOrganis-aráor fi SCALADO ABK, VATENTXNQFamily \ SE '\ 2lO2O73ë \ 2lOâüïšš špplitatiantextToïnstrugïtor IIAS ÉOYIE-OS-ll Ldoc. Determining the bit length of the first bitstream record, skipping the number of bits in the bitstream corresponding to the determined bit length, summing the number of skipped bits for collecting information regarding the number of bits in the bitstream between coefficients of zero or first order in adjacent data units, and to sum the number of coefficients that have been stepped through until all coefficients of the data unit have been stepped through or a block end symbol is found. The method of claim 24, wherein each bit sequence viewed comprises sixteen bits. The method of claim 25, wherein the table lookup comprises making a first lookup table for the first eight bits of the bit sequence. The method of claim 26, wherein the first table lookup returns information about the bit length of the first bitstream record and the number of coefficients passed through or returns information to a second table lookup. The method of claim 27, wherein the table lookup further comprises making a second table lookup for the last eight bits of the bit sequence to determine the bit length of the first bitstream record and the number of coefficients stepped through. The method of any of claims 24-28, wherein the indicators in the block information table are stored as indicators of the bit offset to the coefficient from a static position. The method of any of claims 24-28, wherein the indicators in the block information table are stored as indicators of the bit offset to the coefficient from a bitstream benchmark and the image representation format further comprises a list providing ZÛCE-iJéWZQ lüzßš Vf. ïäoörgariisationHSCALALO AB \ .PATENT \ tloF-amily * -XSE \ 2l02O736 \ 2l @ 2ü? 36 spplicaticnrez fl oïnstrucrcz »SAS * Jfßíëä-» OS-ll 1.doc ~ 10 15 20 25 30 35 528 17b 41 information om each in the bitstream is located. The method of any of claims 24-30, wherein the zero order coefficient is stored represented in a non-differential form for each zero order coefficient represented in the bitstream as a difference to a zero order coefficient of a previous image block. The method of any of claims 24-30, wherein each zero order coefficient is stored represented in a non-differential form. A method of merging two digital images, the method comprising: determining a relationship in the space between the two digital images, assigning image blocks of digital image information in the two digital images indices according to the relationship in the space between the two digital images the images, to form a bitstream representing sequential image blocks according to the assigned indices, each block comprising one or more components, each comprising one or more data units and each data unit being represented as a Huffman-coded stream of coefficients of basic functions, storing image block information for each image block in a block information table according to the position of the image block, said image block information comprising: indicators of a zero order or first order coefficient of each image block in said bit stream, information indicating the number of bits in the bit stream between coefficients of zero or first or in the adjacent data units of the image block, and the zero order coefficient of at least one data unit of each component, said iüikê-ÜE-ZQ 10:56 '6- * ~ = _! ~ 3 ~> 01'ganis- ationwCALADO AEÅ.Pë-Tlšblïwurésifamilylßß fl åLOZOHPQIQZCVEJ F-.pplicuatifzritext'ï “c-ïnstrçzctoz SAE 2005-08-11 Lclcc. 10 15 20 L .._._____ 528 172 42 the zero coefficient of the order is represented in a non-differential form. An image representation format for representing a digital image, said image representation format comprising: image information stored as a bitstream as sequential image blocks, each block or components being each or more data units and each data unit as a Huffman-encoded stream. of coefficients of base functions, and wherein a zero order coefficient represents comprises a represented cient is represented as a difference to the previous zero order coefficient of the corresponding component, and bitstream information stored in connection with the bitstream, said bitstream information comprising information indicates the number of bits of each data unit in the image blocks. The image representation format of claim 34, wherein the bitstream information is compressed. Ûlïïißflåtï * 19255- VrKjüeürítariís-atiQINSCÄLADO ABKPATEN'I '\ __ tl-9Faïnily \ SE} \ 2lO2Û73ERÉIOZUTBE Applicati-.arltextTolnstrucLoi GAS Z-'Clfšt - Oß cß.