AU771764B2 - Scaleable resolution motion image recording and storage system - Google Patents
Scaleable resolution motion image recording and storage system Download PDFInfo
- Publication number
- AU771764B2 AU771764B2 AU39983/00A AU3998300A AU771764B2 AU 771764 B2 AU771764 B2 AU 771764B2 AU 39983/00 A AU39983/00 A AU 39983/00A AU 3998300 A AU3998300 A AU 3998300A AU 771764 B2 AU771764 B2 AU 771764B2
- Authority
- AU
- Australia
- Prior art keywords
- stream
- components
- image
- image input
- input stream
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Ceased
Links
- 238000003860 storage Methods 0.000 title claims description 16
- 238000000034 method Methods 0.000 claims description 27
- 230000002123 temporal effect Effects 0.000 claims description 23
- 238000012545 processing Methods 0.000 claims description 20
- 239000000872 buffer Substances 0.000 claims description 12
- 238000006243 chemical reaction Methods 0.000 claims description 12
- 238000013139 quantization Methods 0.000 claims description 3
- 238000007906 compression Methods 0.000 claims description 2
- 230000006835 compression Effects 0.000 claims description 2
- 230000002596 correlated effect Effects 0.000 claims description 2
- 241000894007 species Species 0.000 description 9
- 230000002093 peripheral effect Effects 0.000 description 6
- 241000023320 Luma <angiosperm> Species 0.000 description 4
- OSWPMRLSEDHDFF-UHFFFAOYSA-N methyl salicylate Chemical compound COC(=O)C1=CC=CC=C1O OSWPMRLSEDHDFF-UHFFFAOYSA-N 0.000 description 4
- 238000011002 quantification Methods 0.000 description 4
- 230000003139 buffering effect Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 3
- 230000009977 dual effect Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 2
- 230000000750 progressive effect Effects 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 101100328887 Caenorhabditis elegans col-34 gene Proteins 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/01—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/46—Embedding additional information in the video signal during the compression process
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
- H04N19/61—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N9/00—Details of colour television systems
- H04N9/79—Processing of colour television signals in connection with recording
- H04N9/7921—Processing of colour television signals in connection with recording for more than one processing mode
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
- Television Signal Processing For Recording (AREA)
- Television Systems (AREA)
Description
2 SCALEABLE RESOLUTION MOTION IMAGE RECORDING AND STORAGE
SYSTEM
Technical Field This invention pertains to the field of digital signal compression and quantification. More specifically, the present invention related to a system and method that provides scaleable resolution image recording and storage.
Background Electronic motion image recorders have traditionally been designed for one or at most only a few motion image formats. With the advent of various high definition video, medical, scientific, and industrial formats, there is a requirement for single systems which can support a broad range of motion image formats. As motion image frame rates, image sizes, and pixel resolutions increase, there oo is a tremendous increase in the amount of data that must be maintained to represent the motion image stream in the sample domain. This places a large burden on the processing, storage, and communications costs to support these data sets at the desired resolution.
Summary of Invention The present invention provides an efficient and cost effective system which can be configured to effectively support all motion image formats. The system can be S. modularly expanded to increase throughput, and can trade off system throughput between the number of image streams, their frame rate, frame resolution, and pixel resolution.
The system uses subband methods throughout to support variable image size and frame rate recording. This allows a single image stream to be divided into multiple lower rate streams to allow more reasonable processing rates.
The present invention also enables the application of a variable sized array of standardized image processing components.
H:\jolzik\keep\Speci\39983-00.doc 11/02/04 3 More specifically, optimized storage and efficient communication is achieved by storing image streams in the information domain, rather than the sample domain. This typically provides a dramatic reduction in storage and communication requirements, while still providing a guaranteed recording quality. Unlike conventional video tape recorders, this system can also place recordings onto the same removable storage medium at any desired image resolution, frame rate, and quality.
According to one aspect of the present invention, there is provided a system for scaling resolution of an image input stream, the system comprising: an input for receiving the image input stream; a frame buffer for storing image frames coupled to the input; a router coupled to the frame buffer for decorrelating the image input stream based on the 20 throughput of the image input stream and dividing the image input stream into frequency bands wherein each frequency band has one or more components using either spatial and/or temporal sub-band transforms; coupled to the router, for each frequency band, a processor that at least quantizes and entropy encodes the components; and a storage device for storing each of the encoded components.
According to a further aspect of the present invention, there is provided a method for processing an image input stream having a throughput, the method comprising the steps of: receiving an image input stream; calculating a frame conversion rate; if the throughput is greater than the frame conversion rate, stopping the processing; H:\jolzik\keep\Speci\39983-OO.doc 11/02/04 4 if the frame conversion rate is lower than the throughput and the image input stream components are correlated, splitting the image input stream into one or more representational streams of components using sub-band coding; routing the representational streams to one or more signal processors; performing quantization and entropy encoding on the representational stream components; and storing the encoded representational stream components.
Brief Description of the Drawings Figure 1 shows a system for scaling the resolution of an image input stream according to an embodiment of the present invention.
j: Figure 2 shows a method for processing an image input stream according to an embodiment of the present invention.
Detailed Description The following description describes the method of the present invention as performed by the system of the present invention. The method described, however, could also be performed by an alternate image processing device.
The following provides a list of components that may .be included in the system of the present invention. The •function accomplished by each component is also provided.
As will be appreciated by one skilled in the art, this list is neither exhaustive nor inclusive and other components that provide similar functionality, such as other band limiters, band splitters or color space conversion modules, may also be added.
H:\jolzik\kee\Speci\39983-OO.doc 11/02/04 5 Component Function Input Bandsplit Deframer Color space Bandlimit Bandsplit CbYCr CbLCrH CbYCr
MM
MMMM
CbLCrD
L--H
LHVD
0 0 o o o.
o o Framebuffer Interlace Processing Router Temporal IP Interlace Processing Spatial IP Quantification Entropy coding Input converters and optional analog bandsplit digital data stream in optional RGB- CbYCr 1 D or 2D Bandlimit and arbitrary rate conversion of components Full band 2 pixel color to halfband chroma dual pixel"quad" Full band 4 pixel color to visual space"quad" Monochrome to bandsplit"quad" Monochrome to 2D bandsplit"quad" Inverse Pulldown Sample only a portion of the input fields or frames Buffer and optional time multiplex frames Interlaced image processing, to allow interlaced image to be processed as progressive Routes N streams to N IP channels Using time multiplexed frames Using simple (Harr) temporal bandsplit of frames Temporal transform and optional time multiplex of frames Interlaced image processing, to allow interlaced image to be processed as progressive Spatial transform Component and temporal quantification as required Entropy coding, potentially for each temporatial and spatial component H:\jolzik\keep\Speci\39983-OO.doc 11/02/04 6 DMA channels RAM Queues Disk Queues Disk, Tape, Provides channels for data movement Peak buffering Peak buffering Final output or Channel The IO subsystems accept motion image input streams in one of numerous pixel formats.
RGBA
YCbCrA
YC
M
MM
MMMM
Red, Green, Blue, and Alpha Luminance, Blue color difference, Red Color difference, and Alpha Luminance and alternating color difference Monochrome Two sequential monochrome samples Four sequential monochrome samples oe o eo The input analog converter 2 can operate at up to a fixed rate of N conversions per second, where N is determined by the particular implementation. If a 20 conversion rate 24 higher than N is desired, the input system can use an analog bandsplit of the input signal into two or four streams at one half or one fourth rate to allow multiple channel conversion at or above N 26.
In one embodiment shown in Figures 1 and 2, the digital IO system converts these input pixel streams 22 into quad component representational streams. In each case, it is assumes that the components have been decorrelated and can be processed separately 28. If the components have not been decorrelated, however, color space conversion may be used to decorrelate the color components. This could also involve subband splitting 26 certain color components in order to further decorrelate the components and spread the bandwidth evenly among the available channels. While splitting the pixel stream into quad component representations is described as one embodiment, the image processing resources can also H:\jolzik\keep\Speci\39983-00.doc 11/02/04 7 process the pixel stream by splitting it into a number of different component representations 26. For example, the present system may be used to process dual channel image streams. In these cases, the first and third components are assigned to the first channel, and the second and fourth are assigned to the second.
CbYCrA Single color pixel, four component full band CbLCrH Dual color pixel, two subband luma component (L and half band chroma Luma is bandsplit using either a Harr transform, an Odd near orthagonal 7 tap filter, or an Odd near orthagonal 9 tap filter.
CbLCrD Quad color pixel, two halfband luma components (L and quarter band chroma. Luma is limited to a diagonal square providing 1/2 the original bandwidth, but full horizontal and vertical response. This is bandsplit using a 2D filter of 3x3 or 9x9 taps into a low half and a diagonal 20 half. The color is bandlimited to half band in both dimensions.
LMMH Quad monochrome pixel, 1D four subband components LHVD Quad monochrome pixel, 2D four subband components The frame buffers 4 can store 1 to N frames of the input stream 22 and distribute 1 to N frames among 1 to N signal processors 8,28. This allows time multiplexing image processing resources, or the support of temporal processing in the frame buffer 4 or router 6. N is determined by the required quad throughput divided by the quad throughput of the individual signal processors 8.
This also allows for support of temporal processing in the frame buffer 4 or router 6.
H:\jolik\keep\Speci\39983-OOdoc 11/02/04 8 The router 6 can accomplish simple spatial or temporal bandsplits. This accomplishes additional decorrelation and allows subdivision of the signal stream 26 among computing resources 8,28. It can also be used to support image resolutions or throughputs that exceed the physical buffers 4 or individual throughput supported by the image processing modules.
This system can also perform temporal transform processing. In the preferred embodiment, a temporal processor 8 is required for each quad stream which will require processing. The input to a temporal processor 8 is a stream of image frames. The output is a stream of shuffled temporal transformed frames. For example, if one temporal transform is selected, an alternate stream of low and high temporal component frames is produced. If two temporal transforms are selected, a representational stream of low, midlow, midhigh, and high temporal component frames is produced. Temporal transforms can be inserted as desired between the router 6 and the spatial transform processor 8.
From 1 to N spatial signal processors 8 accept the quad streams of frames. They perform a spatial multiband transform, quantify the data to the specified recording e "signal quality and entropy encode it 30. From 4 to 4N DMA channels deliver the encoded data to a RAM buffer 10 which handles peak transfer loads.
The data can then be transferred to queuing or output files on disks, tapes, or other peripherals 12 that are capable of storing digital information 32. If the data rate is higher than the specified peripheral can support, the information is queued to a faster storage device until the desired peripheral can accept the data. In the case of peak tolerant recording at a fixed rate, a phase delay is specified and a hierarchy of processes related to the H:\jolik\keep\Speci\39983-OO.doc 11/02/04 9 hierarchy of buffer devices 10 monitor the rate and potentially control the quality level. This allows the process to be as tolerant of peaks as possible. Each process in charge of buffering determines whether the data budget is being exceeded over the time scale of interest to this process. If the local average rate exceeds a set point, the quality level of the images being received by the output or storage device is reduced until the local average rate no longer exceeds the set point. If the average is below the set point, and the quality level is below the target, the quality level of the images is increased.
Example: Here is an example of this unique storage process. The S: target data rate is 12 MB/S and a quality level may be 66 dB. This would allow peaks rates as high as 80 megabyte :per second to occur for one frame time, peaks as high as 40 to occur for up to about a second, and peaks up to 20 megabyte per second for a few seconds. The steps up or down are all added together and to the target to obtain the next quality level. The quality level is re-computed for every frame.
S 25 Process Timescale Setpoint Peak Rate dB Steps To RAM 4 Frames 40 MB/S 80 MB/S 3 dB To Disk 4 second 20 MB/S 40 MB/S 1 dB To Tape/Channel 30 second 10 MB/S 12 MB/S 1/2 dB The playback process is a reversal of the recording process, with the exception that playing from a slower peripheral than required induces a preroll time while a faster peripheral is used to queue the required amount of data. A preroll time is the time needed to transfer data from a slower peripheral to a faster storage device. For example, if the data was stored on a compact disc, the H:\jolzik\keep\Speci\39983-OO.doc 11/02/04 10 data could be prerolled to RAM in order to provide playback at the appropriate frame rate and quality level.
Although the description above contains many detailed descriptions, these descriptions should not be construed as limiting the scope of the invention but merely as providing illustrations of some of the presently preferred implementations of this invention. For example, although this method was described with reference to standard motion and still images, this method can be used to optimize quantification of any signal stream. Thus the scope of the invention should be determined by the appended claims and their legal equivalents, rather than by examples given.
In the claims which follow and in the preceding description of the invention, except where the context requires otherwise due to express language or necessary :0 implication, the word "comprise" or variations such as 20 "comprises" or "comprising" is used in an inclusive sense, i.e. to specify the presence of the stated features but not to preclude the presence or addition of further features in various embodiments of the invention.
0* It is to be understood that, if any prior art 0.* publication is referred to herein, such reference does not 0. constitute an admission that the publication forms a part of the common general knowledge in the art, in Australia or any other country.
H:\jolzik\keep\Speci\39983-OO.doc 11/02/04
Claims (12)
1. A system for scaling resolution of an image input stream, the system comprising: an input for receiving the image input stream; a frame buffer for storing image frames coupled to the input; a router coupled to the frame buffer for decorrelating the image input stream based on the throughput of the image input stream and dividing the image input stream into frequency bands wherein each frequency band has one or more components using either spatial and/or temporal sub-band transforms; coupled to the router, for each frequency band, a processor that at least quantizes and entropy encodes the components; and *on* a storage device for storing each of the encoded components. 20
2. The system of claim 1, wherein the system further comprises an analog band splitter for splitting the image input stream into multiple streams and an analog to digital converter coupled to the frame buffer.
3. The system of claim 1, wherein the system further comprises a color space converter for decorrelating the color components. ooo
4. The system of claim 1, wherein the system comprises four compression channels. The system of claim 1, wherein the router further comprises a processor capable of performing a temporal transform on a plurality of image frames producing a plurality of temporal components wherein each temporal component is provided to the processor for performing quantization and entropy encoding.
H:\jolzik\keep\Speci\39983-OO.doc 11/02/04 12
6. The system of claim 1, wherein the system further comprises, coupled to the router and each processor, a temporal transform module.
7. A method for processing an image input stream having a throughput, the method comprising the steps of: receiving an image input stream; calculating a frame conversion rate; if the throughput is greater than the frame conversion rate, stopping the processing; if the frame conversion rate is lower than the throughput and the image input stream components are correlated, splitting the image input stream into one or more representational streams of components using sub-band coding; routing the representational streams to one or more ^signal processors; performing quantization and entropy encoding on the representational stream components; and storing the encoded representational stream components.
8. The method of claim 7, further comprising splitting the image stream into four quad component representations.
9. The method of claim 7, further comprising applying an analog bandsplit to the stream.
10. The method of claim 7, wherein after routing the image streams to one or more signal processors using a separate temporal processor to process each representational stream.
11. The method of claim 7, after routing the image streams to one or more signal processors, performing a spatial multiband transform on the stream. H:\jolzik\keep\Speci\39983-OOdoc 11/02/04 13
12. The method of claim 7, wherein the step of storing the streams comprises: calculating the data budget of a first storage device used to store the data; responsive to the stream requiring a data budget higher than is supported by the first storage device, queuing the data to a second storage device until the first storage device is able to accept the stream. Dated this 11th day of February 2004 QUVIS, INC. By their Patent Attorneys GRIFFITH HACK Fellows Institute of Patent and Trade Mark Attorneys of Australia H:\jolzik\keep\Speci\39983-OO.doc 11/02/04
Applications Claiming Priority (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US11855699P | 1999-02-04 | 1999-02-04 | |
| US60/118556 | 1999-02-04 | ||
| PCT/US2000/003103 WO2000046978A2 (en) | 1999-02-04 | 2000-02-04 | Scaleable resolution motion image recording and storage system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| AU3998300A AU3998300A (en) | 2000-08-25 |
| AU771764B2 true AU771764B2 (en) | 2004-04-01 |
Family
ID=22379344
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| AU39983/00A Ceased AU771764B2 (en) | 1999-02-04 | 2000-02-04 | Scaleable resolution motion image recording and storage system |
Country Status (6)
| Country | Link |
|---|---|
| EP (1) | EP1157540A4 (en) |
| JP (1) | JP2002536919A (en) |
| KR (1) | KR100629808B1 (en) |
| AU (1) | AU771764B2 (en) |
| CA (1) | CA2361474A1 (en) |
| WO (1) | WO2000046978A2 (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US4601055A (en) * | 1984-04-10 | 1986-07-15 | The United States Of America As Represented By The Secretary Of Commerce | Image processor |
| US4851826A (en) * | 1987-05-29 | 1989-07-25 | Commodore Business Machines, Inc. | Computer video demultiplexer |
| US5881245A (en) * | 1996-09-10 | 1999-03-09 | Digital Video Systems, Inc. | Method and apparatus for transmitting MPEG data at an adaptive data rate |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6195465B1 (en) * | 1994-09-21 | 2001-02-27 | Ricoh Company, Ltd. | Method and apparatus for compression using reversible wavelet transforms and an embedded codestream |
| US6865291B1 (en) * | 1996-06-24 | 2005-03-08 | Andrew Michael Zador | Method apparatus and system for compressing data that wavelet decomposes by color plane and then divides by magnitude range non-dc terms between a scalar quantizer and a vector quantizer |
| US5768445A (en) * | 1996-09-13 | 1998-06-16 | Silicon Graphics, Inc. | Compression and decompression scheme performed on shared workstation memory by media coprocessor |
| FR2758670B1 (en) * | 1997-01-20 | 1999-03-26 | France Telecom | PROCESS FOR SYNTHESIZING DIGITAL FILTERS BY WAVELET AND CORRESPONDING DIGITAL FILTERS |
| US6154493A (en) * | 1998-05-21 | 2000-11-28 | Intel Corporation | Compression of color images based on a 2-dimensional discrete wavelet transform yielding a perceptually lossless image |
-
2000
- 2000-02-04 CA CA002361474A patent/CA2361474A1/en not_active Abandoned
- 2000-02-04 AU AU39983/00A patent/AU771764B2/en not_active Ceased
- 2000-02-04 EP EP00919270A patent/EP1157540A4/en not_active Ceased
- 2000-02-04 KR KR1020017009881A patent/KR100629808B1/en not_active IP Right Cessation
- 2000-02-04 JP JP2000597943A patent/JP2002536919A/en not_active Withdrawn
- 2000-02-04 WO PCT/US2000/003103 patent/WO2000046978A2/en active IP Right Grant
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US4601055A (en) * | 1984-04-10 | 1986-07-15 | The United States Of America As Represented By The Secretary Of Commerce | Image processor |
| US4851826A (en) * | 1987-05-29 | 1989-07-25 | Commodore Business Machines, Inc. | Computer video demultiplexer |
| US5881245A (en) * | 1996-09-10 | 1999-03-09 | Digital Video Systems, Inc. | Method and apparatus for transmitting MPEG data at an adaptive data rate |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2000046978A2 (en) | 2000-08-10 |
| KR20010101779A (en) | 2001-11-14 |
| EP1157540A1 (en) | 2001-11-28 |
| CA2361474A1 (en) | 2000-08-10 |
| JP2002536919A (en) | 2002-10-29 |
| WO2000046978A3 (en) | 2000-12-07 |
| KR100629808B1 (en) | 2006-09-29 |
| EP1157540A4 (en) | 2003-05-21 |
| AU3998300A (en) | 2000-08-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US6823014B2 (en) | Video decoder with down conversion function and method for decoding video signal | |
| US5363097A (en) | Direct sequential-bit variable length decoder | |
| EP1542476A2 (en) | Color space coding framework | |
| US6301304B1 (en) | Architecture and method for inverse quantization of discrete cosine transform coefficients in MPEG decoders | |
| US7684489B2 (en) | Method and apparatus for video compression | |
| WO2013058541A1 (en) | Method and apparatus for encoding/decoding image | |
| US20100061643A1 (en) | Encoding device and encoding method, and decoding device and decoding method | |
| JPH06326615A (en) | Method and equipment for decoding code stream comprising variable-length codes | |
| GB2422262A (en) | Decoding dual video signals | |
| EP1573469B1 (en) | Apparatus and method for multiple description encoding | |
| US20020141499A1 (en) | Scalable programmable motion image system | |
| CN114245989A (en) | Encoder and method of encoding a sequence of frames | |
| US6823129B1 (en) | Scaleable resolution motion image recording and storage system | |
| AU771764B2 (en) | Scaleable resolution motion image recording and storage system | |
| US6507673B1 (en) | Method and apparatus for video encoding decision | |
| US20090296822A1 (en) | Reduced Memory Mode Video Decode | |
| JP4339784B2 (en) | A method for decoding compressed bitstream encoded video encoded as a plurality of blocks. | |
| EP0843483B1 (en) | A method for decoding encoded video data | |
| JP2012004898A (en) | Storage device, encoding device, encoding method, and program | |
| CN106954073B (en) | Video data input and output method, device and system | |
| JP2005508100A (en) | Scalable programmable video system | |
| JPH06113140A (en) | Picture processor | |
| JP2000092512A (en) | Video signal encoding device, video signal decoding device and video signal conversion device | |
| Mahmudov Ergash et al. | Building Principles and Application of Multifunctional Video Information System | |
| JPH01260990A (en) | Color video signal processing method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| SREP | Specification republished | ||
| FGA | Letters patent sealed or granted (standard patent) |