GB2356101A - Image transmission using mobile communication device - Google Patents

Abstract

A portable communication device includes an imaging device capable of scanning a document containing black and white text and/or graphics. The document to be captured is presented to an optics head (100) which focusses the image of the document onto an image sensor (105), the sensor including means for converting the image from an analogue to digital signal. The image data is transferred via a buffer (110) to an image processing algorithm block (115) where it converted into facsimile format. Upon completion of format conversion, the image data is stored in a fax memory storage device (125), where fax data may be viewed. A facsimile engine (135) prepares the document for transmission (140).

Description

2356101 Digital image processing The present invention relates to a

digital image processing, and in particular to digital image processing in a portable radio communication device.

Mobile digital video communication is an area of telecommunications' technology which is currently the focus of heavy research work. Digital video offers a great many advantages over traditional analogue systems, supporting services such as video telephony and multimedia applications. However, a key problem of digital video when compared to analogue systems is the demand it places on communications and storage resources. For example, a bandwidth of approximately 160 Mbps is required in order to transmit a broadcast quality video which compares with a bandwidth of approximately 5 MHz for a comparable quality analogue video. Thus to be able to use digital video, the digital signal requires reduction of the quantity of data. Data reduction is achieved by using compression techniques to remove redundant data while still retaining sufficient information to allow the original image to be reproduced within an acceptable quality level.

Whilst applications such as video telephony are concerned with capturing, compressing and transmitting image information of a scene around a mobile video-phone, the present invention addresses the particular requirements of capturing, processing and transmitting digital image data representative of a document comprising for example text and graphics.

Accordingly, the present invention provides a method of transmitting an image of a document between a portable radio communication device having an imaging device and a network in which the portable radio communication device is operative, 2 the method comprising - providing a dedicated parameter set up for said imaging device for capturing and processing text and/or graphics information of said image of the document to be transmitted, - converting said image into facsimile format data, and transmitting as a facsimile transmission said facsimile data.

It has been recognised by the present inventors that unlike an image taken of an ordinary scene, an image taken of a document consists primarily of text and/or graphics, which text and/or graphics is usually presented in black and white in the document. Furthermore, that in the context of a document, the image has a fixed character to it when compared to say a scene which typically has some moving elements in it. The inventors thus realised that contrast rather than colour is a key aspect to imaging of a document for transmission. From a review of existing digital video imaging devices the inventors noted that these are invariably set up to capture, process and transmit images of a natural scene, and are thus configured to capture and process images that are very different in character to images of a document. For example, existing digital video imaging devices have hardware and software designed to perform image compression and other such processing steps. In recognising this fundamental difference between scene imaging and document imaging, the inventors have proposed that in the context of document imaging a different image processing approach may be more appropriate. Moreover, because text and graphics are displayed in high contrast black and white, many of the processing steps used in colour scene imaging are not per se necessary in document imaging. Accordingly, the inventors have concluded that there may therefore be scope for making a saving in terms of processing steps and power when imaging a document rather than a scene. With this in mind, the inventors have devised a video imaging device concept that configures the video imaging device specifically to receive and transmit images of a document. More particularly, the video 3 imaging device has a dedicated set up optimised for document imaging and transmission via the facsimile standards.

Thus the provision in the present invention of a method and portable radiotelephone apparatus that utilises a dedicated set up for an on-board imaging device for document imaging results in benefits in savings in memory usage and processing power. For example, the invention bypasses standard image compression techniques in favour of optimisation for black and white contrast which leads to a reduced amount of RAM take up of the apparatus.

The invention takes advantage of the observation that in comparison to imaging a natural scene, imaging a document comprising text and/or graphics involves fewer variables.

There are products currently available, such as Applicant's Communicator device, which can send a facsimile transmission remotely from a mobile terminal operating in a cellular network to a facsimile machine with a facsimile number. Sending faxes from these devices is limited to only those which are typed in directly on a computer file or forwarded onwards. The present invention allows a paper document to be captured by an imaging device and transmitted to a facsimile number.

In a preferred embodiment the imaging device is provided with a switch to toggle the imaging device to processing an image of a document.

The present invention will now be described by way of example with reference to the accompanying drawings in which:

Figure 1 shows in block diagram form the components of an image processing arrangement of a preferred embodiment of the present invention; 4 Figure 2 shows in outline a flow diagram for controlling an image processing block diagram of Figure 1; Figure 3 provides a resolution comparison of A4 and A5 pages. 5 Referring initially to Figure 1, there is shown the basic building blocks of a preferred embodiment of a digital video imaging device in accordance with the present invention. The video imaging device comprises at the front end an optics head 100 in association with an image sensor and A/D converter 105.

The sensor and A/D converter is connected to a buffer and interface 110 for the control bus of the video imaging device. This links in to the image processing algorithm block 115 which is controlled by the microprocessor of the digital video imaging device. The image processing algorithm block 115 is connected to a block 120 containing image set up parameters. The image set up parameters block 115 is also directly linked to the buffer 110 and in turn controls the sensor 105. The output of the image processing algorithm block 115 is to a fax document storage memory 125 which includes a fax viewer 130. Connected to the storage memory 125 is the facsimile engine 135 which in turn outputs to sending block 140.

The operation of the system is as follows. When a document to be sent by fax is offered up to a video imaging device of a portable radio communication device, the video imaging device detects that this is a page of a document and switches the imaging device to fax transmission mode. Alternatively the user can switch the imaging device to fax transmission mode. In switching to fax transmission mode, the image sensor is configured on the basis of one or more parameters from the image set up parameters block. To effect this the optimising document imaging parameters are transferred first from the set up parameters block to the buffer and then from there to the high resolution sensor. Dedicated set up parameters include for example gain controls, sensitivity adjustments, and gamma correction. Accordingly, the sensor can be adjusted for optimum contrast of black and white, hence other parameters can be traded off. For example by giving up colour trueness, sensitivity to contrast can be enhanced.

Adjustments that can be made at the sensor level for a CMOS sensor would typically include pixel exposure time and pixel gain. Take the following simple example:

A black and white text document is a high contrast image comprising large areas of white and sharp transitions to dark text. A long exposure time will cause saturation of pixels from the light pixels to the dark pixels. Short exposure times however, incur decreased signal to noise ratio. To compensate, a high pixel gain is required. Because this is complicated by the transformation from RAW image data (Bayer pattern) to other formats, greater leverage can be made at that level when combined with the sensor level parameter adjustment.

The document to be captured and sent by fax from the portable radio communication device is presented to the optics head of the digital video imaging device of the portable radio communication device. The optics head focuses the image of the page of the document onto the image sensor thus capturing an image. The image is then converted from an analogue to digital signal by the A/D converter and the raw image data is transferred from the sensor to the buffer. The sensor provides the image information in an 8 pixel by 8 pixel format. The raw image data is then transferred from the buffer to the image processing algorithm block in which the raw data undergoes conversion from raw image data to fax format data (details of this process will be described later). Once the image data has been converted to fax data it is moved to the fax storage memory device. The fax viewer may view the fax data of the raw image data from the fax memory storage device. When the fax is to be transmitted the fax is loaded into the facsimile engine which prepares the fax document for transmission. This is then ready for transmission from the block captioned sending.

6 From block 140 the image of the document is transmitted as a fax transmission to the network which may be GSM. The network could comprise a Public Switched Telephone Network (PSTN), the Internet or any kind of network cloud, the fax transmission finally arriving at the receiving terminal in analogue form, in the case of a standard facsimile, or in digital form, in the case of a personal computer or Communicator.

Returning to block 115 of Figure 1, the details of the image processing algorithms will now be described with reference to Figure 2. Here, to convert the raw image data into a fax format data the raw image data first has all colour data removed. Then contrast is adjusted (usually increased) and the brightness increased. This is then converted to bitmap format and then converted to fax format. The fax format remains in memory and is added to in 16kB blocks until the complete image is stored.

Once complete it is transferred to the fax storage transmission memory.

Since the data is processed as it is delivered directly from the camera sensor the invention performs data pipelining of the raw image data into fax format.

The use of data pipelining directly from the sensor, in raw format, directly to fax format allows for bypassing any form of a separate compression stage or the need for a large intermediate memory.

Accordingly, the whole process in the preferred embodiment can be summarised as comprising the following sequence of steps; 1. The user action of selecting Fax Document Aapplication from the user interface of the mobile phone, 2. Selecting scan image feature, 3. Triggering the parameter set up application and fetching the fax specific parameters, 4. Carrying out the hardware level set up of the sensor, 5. Scanning the image, 7 6. Pipelining from raw format to fax without any colour interpolation, compression or such function typically used when taking normal landscape pictures with the same sensor, 7. Adding manually some text, lines, drawings if required from the user interface, 8. Selecting recipient from user interface and activating sending of the document.

In contrast, a method available today for sending images of documents by fax might involve the use of a digital camera and Personal Computer as follows.

In this case, the document would be photographed, the camera would then automatically convert from its RAW data format, which could be Bayer matrix, to some intermediate stage, which could be YUV420, to some storage format, such as JPEG which is in common use today. Then the image would be moved via connection cable to a PC. Using PC software tools, like Coral or PaintShop, the image could be converted into a Bitmap image format then using PC facsimile software tools, like WinFax, the image could be converted to fax compatible format so as to fax the image.

Preferred embodiment of the present invention obviates many of these steps by the inclusion of an algorithm to convert directly from RAW format (which could be Bayer matrix) to fax format - hence cutting down the processing steps. Since the algorithm uses a pipeline technique the data is converted in blocks of data, much smaller than the whole image, as the data is produced by an imaging device. The memory requirement is only the size of 1 or 2 blocks, not the whole image.

A camera available for document scanning is typically a universal type colour camera. Most of these cameras today have image sensors which contain so called Bayer colour mosaic RGB pattern; colour image is formed by interpolating different colours from the raw mosaic image.

8 However, this interpolation loses some spatial resolution, i.e. sharpness. Thus the optimum solution is to use the raw Bayer image as a starting point when forming the black and white document image. A simple way to do this is to assume that in document imaging mode the target is black and white, and give a different black/white threshold level respectively for each colour of pixels (R, G or B). This way a better binary image can be formed.

Furthermore, sensor exposure time should be adjusted to give maximum contrast to the raw Bayer image.

Accordingly, since data is streamed directly from the imaging device into conversion process the image only occupies as much as facsimile file would thereby allowing for a minimum use of RAM. A prior art method is to store the image in raw format and then convert it. This would require up to

1,300,000 x 12 bit = 2 Mbytes for a one A5 sheet. A fax image is about only kilobytes. A prior art alternative is to store the image in a typical compressed mode such as JPEG and then convert to fax afterwards. However this method causes much degradation of fax image quality as it compresses away the very information that is most useful -the high spatial frequency element of the image.

Since the whole image transfer is handled at every stage it is optimised without user interactions. Typically a photograph of a white page will appear very dark and grey. This can be adjusted by post processing measures by adding brightness and contrast until the image looks like fax quality. In this case the image can be automatically adjusted since it is known that the input is a document and the output format is fax i.e. black and white.

With regard to the conversion of RAW image data to facsimile compliant data certain issues relating the hardware compliance need to be addressed, and this will now be described with reference to Figure 3.

9 In fax format the pixel element is rectangular, whilst in most currently available digital still cameras image pixel data is based on a square pixel. The pixel resolution varies between the modes and this is only in one dimension, namely height. For example one A4 page (in dots) is: 1725 wide by 1077 high in standard fax mode, 1725 wide by 2156 high in fine fax mode, 1725 wide by 4301 super fine fax mode. Furthermore the aspect ratio of the image varies between modes and is not equivalent to typical sensor aspect ratios of 1. 25:1 common in camera sensors. Additionally, in a facsimile transmission the document is presented to the imager in a controlled manner, whereas a camera is held by a user and subject to multiple variations: the document can be presented to the imager in any angle and distance.

A solution to these issues can be obtained by increasing the resolution of image data. Data of high resolution, higher than any of the facsimile formats, can then be sampled or filtered in any number of techniques to produce the needed fax resolution. This high resolution image can be generated by means of a high resolution sensor - which are presently available.

Alternatively, using a lower resolution sensor and scanning the document multiple times either by presenting different parts of the document at different times or by scanning the whole document several times, in each case, a high resolution image can be generated by a binding process.

For example one A5 page (1/2 of A4) in standard fax mode requires 929K pixels, allowing for a border in the image of say 10%, 1021 K pixel image data is required. If the image is scanned with a 2.31VIeg pixel sensor (available today) then more than enough data is available to generate the needed fax resolution. If however an image sensor of 11 OK pixels is used then multiple images say up to 100 would be needed to generate the required high resolution image in this case. Accordingly, storing 100 images and processing them all would require relatively large memory (- 11 MBytes) and extended processing time.

Against that background, the power of the data pipelining solution of the present invention should therefore now be apparent. By processing the data for each of the images (say 100 as described above) as they are generated and only storing the needed results from that processing that are relevant to resolving a high resolution image or in fact a facsimile image, only the data that is needed to store the one A5 fax page is needed which is approximately 20KBytes.

An advantageous application of the present invention is a business user who with a mobile equipped with the present invention can take copies of documents at any location and store, send or file them immediately with little effort.

The present invention may be embodied in other specific forms without departing from its essential attributes. The digital imaging device may be an optional detachable accessory to be attached only when needed. Accordingly reference should be made to the appended claims and other general statement's herein rather than to the foregoing specific description as indicating the scope of invention.

Furthermore, each feature disclosed in this specification (which term includes the claims) and/or shown in the drawings may be incorporated in the invention independently of other disclosed and/or illustrated features. In this regard, the invention includes any novel features or combination of features disclosed herein either explicitly or any generalisation thereof irrespective of whether or not it relates to the claimed invention or mitigates any or all of the problems addressed.

The appended abstract as filed herewith is included in the specification by reference.

Claims (6)

Claims

1. A method of transmitting an image of a document between a portable radio communication device having an imaging device and a network in which the portable radio communication device is operative, the method comprising - providing a dedicated parameter set up for said imaging device for capturing and processing text and/or graphics information of said image of the document to be transmitted, - converting said image into facsimile format data, and - transmitting as a facsimile transmission said facsimile data.

2. A method according to claim 1, wherein the provision of a dedicated parameter set up for said imaging device for capturing and processing text and/or graphics, comprises optimising at least one sensor parameter for black and white imaging.

3. A method according to claim 2 wherein said at least one sensor parameter comprises pixel exposure time.

4. A method according to claim 2 or claim 3 wherein said at least one sensor parameter comprises pixel gain.

5. A portable radio communication device including a imaging device and having a hardware set up mode for said imaging device dedicated to process black and white text and/or graphics and provide image data, and means for pipe-lining said image data directly to fax format ready for radio transmission.

6. A method of transmitting an image of a document is substantially as hereinbefore described with reference to and as shown in the accompanying drawings.