EyeLytics Main/Baseline Profile HD H264 Encoder FPGA/ASIC IP

Combines low cost and high quality

EyeLytics Main/Baseline Profile High Definition H264 Encoder IP core has been developed using an optimized heavily pipelined architecture. This architecture gives our encoder a 2x performance advantage over our competitors using a similar amount of logic. Our encoder is developed with the whole system in mind. This makes our interface system optimized and easy to use. The encoder is fully verified in the lab with a real life surveillance system and application. IP Features: Supports image sizes from 176x144 up to 2000x2000 pixels. Multi-channel is supported by time sharing the encoder. o Each channel can have different resolution, quality and frame rate. o Main Profile and Baseline Profile dynamically switchable between channel o There is no limit on the number of channels sharing an encoder. Constant quality. Scene complexity does not degrade the image quality. Instead, the frame rate is adjusted to achieve the best image quality. This feature is very important in surveillance applications which often prefer a clear image over smooth motion. Variable bit-rate with peak limit Low latency. Less than 32 lines of delay from the end of video input frame to the end of encoded bitstream. High compression o CABAC / CAVLC o o o o o

April 20th, 2010

Deblock all 13 Intra Luma modes all 4 Intra Chroma modes single slice operation all 4 inter prediction macroblock partitions o all 4 inter prediction submacroblock partitions o skip macroblock o search window of 240x128 pixels o half pel and quarter pel search High performance Low Gate Count Altera CycloneIII -8 speed ~32K LC ~1.3Mbit ~191 M9K 5 DSP 150MHz 720p30 1080p15 Altera Stratix III -2 speed ~25K ALUT ~1.3Mbit ~154 M9K ~5 M144K 5 DSP 300MHz 720p60 1080p30 0.13um standard cell ~300K gates ~1.3Mbit N/A 300MHz 720p60 1080p30

Tech. Logic Count Memory Count DSP Count Perform.

IP camera demo

Contact: sales@eyelytics.com www.eyelytics.com

EyeLytics
1 Introduction

Main/Baseline Profile HD H264 Encoder

FPGA
Video Input EyeLytics H264 Encoder Bit Stream Output

Multi-port Video Frame Buffer (user provided or EyeLytics)

External Memory (DDR-SDRAM, DDR2-SDRAM or DDR3-SDRAM)

EyeLytics Main/Baseline Profile High Definition H264 Encoder IP core is designed to receive video input and to send bit stream output thru a separate multi-port video frame buffer module. An EyeLytics multi-port video frame buffer IP core is available. Please contact sales@eyelytics.com. Besides the multi-port video frame buffer connections, the Encoder IP core only requires clocks, reset, some static signals potentially come from software programmable registers and a StartFrame pulse to start encode of an image frame. There are also status signals FrameDone and BitStreamByteCount to notify designers that the encode is finished and to report the byte count of the output bit stream.

April 20th, 2010 Eyelytics Inc., reserves the right to change products or specifications without notice. 2008 Eyelytics, Inc. All rights reserved

EyeLytics
2 Interface Signals
Signal Name
clk CabacClk

Main/Baseline Profile HD H264 Encoder

Direction Bit width

in in

Descriptions
system clock clock for the CABAC module. CABAC module runs slower than the rest of the system. To avoid slowing down the whole system by the CABAC module, CABAC module runs with this separate CabacClk. CabacClk should run at less than 2/3 of the system clock frequency. A high CabacClk frequency allows the encoder to support high bit rate applications. Not used in CAVLC mode system reset

System Signals

reset StartFrame StartSeq CavlcMode PFrame

in in in in in

Control Input Signals

A single pulse of StartFrame starts the encoder engine Indicates the current frame is the first frame of a video sequence. This signal should be valid when StartFrame is asserted. 1: current frame enables using CAVLC 0: current frame enables using CABAC 1: current frame is a P Frame 0: current frame is an I Frame This signal should be valid one cycle before StartFrame is asserted and remain static until FrameDone is asserted. Quantisation parameter. Setting it 0 to generate the highest quality video and higher bit rate. Setting it to larger values to generate lower quality video and lower bit rate. Valid range is 0 to 51. This signal should be valid one cycle before StartFrame is asserted and remain static until FrameDone is asserted. Defines the width of the image. The value should be the width in pixel divided by 16 and minus one. For example, a SD image of 720x480 should have this input set to 720/16-1 = 44. This signal should be valid one cycle before StartFrame is asserted and remain static until FrameDone is asserted.

QP

in

5:0

HSizeInMBMinusOne

in

6:0

April 20th, 2010 Eyelytics Inc., reserves the right to change products or specifications without notice. 2008 Eyelytics, Inc. All rights reserved

EyeLytics
VSizeInMBMinusOne in 6:0

Main/Baseline Profile HD H264 Encoder Defines the height of the image. The value should be the height in pixel divided by 16 and minus one. For example, a SD image of 720x480 should have this input set to 480/16-1 = 29. This signal should be valid one cycle before StartFrame is asserted and remain static until FrameDone is asserted. Defines the amount of data in the image. The value should be the width in pixel times the height in pixel times 1.5 divided by 128. The 1.5 factor is to calculate the number of bytes in a 4:2:0 video. For example, a SD image of 720x480 should hav this input set to 720x480x1.5/128 = 4050. This signal should be valid one cycle before StartFrame is asserted and remain static until FrameDone is asserted. This should be set to 0 for the first frame of a sequence. It should then be incremented by one for every frame in the same stream. When 15 is reached, the frame_num of the stream should wrap around and be 0 again. This signal should be valid one cycle before StartFrame is asserted and remain static until FrameDone is asserted. Set to 1 to flush the bitstream data at the end of encode. This should always be set to 1 for multistream encode. For single stream encode, user has the option to keep this 0 and only flushing the encoder at the very last frame of a sequence. This signal should be valid one cycle before StartFrame is asserted and remain static until FrameDone is asserted. Set the external memory address where the bitstream should be written to. Each address location is a 32-bit word. This signal should be valid one cycle before StartFrame is asserted and remain static until FrameDone is asserted. Set the external memory address where the video input data should be read from. Each address location is a 32-bit word. This signal should be valid one cycle before StartFrame is asserted and remain static until FrameDone is asserted.

PicSizeIn128ByteWords in

13:0

frame_num

in

3:0

flush_frame

in

BS_DmaAddr

in

24:0

IN_BaseAddr

in

24:0

April 20th, 2010 Eyelytics Inc., reserves the right to change products or specifications without notice. 2008 Eyelytics, Inc. All rights reserved

EyeLytics
REF_BaseAddr in 24:0

Main/Baseline Profile HD H264 Encoder Set the external memory address where the video reference data should be read from. Each address location is a 32-bit word. This signal should be valid one cycle before StartFrame is asserted and remain static until FrameDone is asserted.. Set the external memory address where the video decoded / reference data should be written to. Each address location is a 32-bit word. This signal should be valid one cycle before StartFrame is asserted and remain static until FrameDone is asserted. BSData is valid Compressed bit stream output A signal indicates the encoder is ready for the start of the next encode. This signal is incremented for every bit-stream byte sent out. When it reaches 0xFFFFFF, it will wrap around back to 0x000000. Video input frame buffer read request. Video input frame buffer read ready Video input frame buffer address. Each address location is a 32-bit word. Video input frame buffer read data Video input frame buffer read data valid Video input frame buffer read data read enable Motion estimation frame buffer read request. Motion estimation frame buffer read ready Motion estimation frame buffer address. Each address location is a 32-bit word. Motion estimation frame buffer read data Motion estimation frame buffer read data valid Motion compensation chroma frame buffer read request. Motion compensation chroma frame buffer read ready Motion compensation chroma frame buffer address. Each address location is a 32-bit word. Motion compensation chroma frame buffer read data Motion compensation chroma frame buffer read

OUT_BaseAddr

in

24:0

Bitstream Output Signals

BSValid BSData FrameDone BitStreamByteCount out out out out 23:0 7:0

Status Output Signals

Frame Buffer Interface Signals

IN_FB_read_req IN_FB_ready IN_FB_address IN_FB_rdata IN_FB_rdata_valid IN_FB_rdata_ren ME_FB_read_req ME_FB_ready ME_FB_address ME_FB_rdata ME_FB_rdata_valid MCC_FB_read_req MCC_FB_ready MCC_FB_address MCC_FB_rdata MCC_FB_rdata_valid out in out in in out out in out in in out in out in in 24:0 63:0 24:0 63:0

24:0 63:0

April 20th, 2010 Eyelytics Inc., reserves the right to change products or specifications without notice. 2008 Eyelytics, Inc. All rights reserved

EyeLytics
MCC_FB_rdata_ren DB_FB_write_req DB_FB_ready DB_FB_address DB_FB_wdata DB_FB_be DB_FB_wdata_req BS_FB_write_req BS_FB_ready BS_FB_address BS_FB_wdata BS_FB_be BS_FB_wdata_wr BS_FB_wdata_wen BS_FB_wdata_pause out out in out out out in out in out out out out in in

Main/Baseline Profile HD H264 Encoder data valid Motion compensation chroma frame buffer read data read enable Deblock frame buffer write request Deblock frame buffer read ready Deblock frame buffer address. Each address location is a 32-bit word. Deblock frame buffer write data Deblock frame buffer write data byte enable Deblock frame buffer write data request Bit stream frame buffer write request Bit stream frame buffer read ready Bit stream frame buffer address. Each address location is a 32-bit word. Bit stream frame buffer write data Bit stream frame buffer write data byte enable Bit stream frame buffer write data write Bit stream frame buffer write data write enable Bit stream frame buffer write data pause

24:0 63:0 7:0

24:0 63:0 7:0

April 20th, 2010 Eyelytics Inc., reserves the right to change products or specifications without notice. 2008 Eyelytics, Inc. All rights reserved

EyeLytics
3 External Frame Buffer Requirement

Main/Baseline Profile HD H264 Encoder

An external multi-port frame buffer is required to build the encoding system. The following sections describe the read / write operations.

3.1 Frame Buffer Request Operation

1) Encoder sets the FB_address with the target frame buffer address location. 2) Encoder then asserts the FB_write_req signal or the FB_read_req signal to request frame buffer write operation or read operation respectively. 3) Each frame buffer request is for eight bursts of 16 bytes or for a total of 128 bytes. Each burst can have a different burst starting address. As a result, there are eight FB_ready pulses to latch eight starting addresses. 4) The first FB_ready signal generated by the frame buffer is used as both a grant signal as well as an address advance signal. FB_write_req or FB_read_req should be deasserted after the first FB_ready if there are no more requests afterward. 5) Encoder will change its address the next cycle after each FB_ready pulse. 6) FB_ready should never be asserted two consecutive cycles in a row.

April 20th, 2010 Eyelytics Inc., reserves the right to change products or specifications without notice. 2008 Eyelytics, Inc. All rights reserved

EyeLytics

Main/Baseline Profile HD H264 Encoder

3.2 Standard Frame Buffer Data Write Operation

1) Frame buffer is expected to assert the FB_wdata_req signal one cycle for each 64-bit data. This means FB_wdata_req should be asserted for two cycles for each starting address to transfer 16 bytes. 2) Encoder presents valid data on FB_wdata and FB_be two cycles after each FB_wdata_req assertion. 3) Encoder also zeros out the FB_wdata and FB_be buses when there is no data being transferred. This is to make it easier for the frame buffer designer to meet timing. Simply use or logic to multiplex all the FB_wdata and FB_be buses together.

April 20th, 2010 Eyelytics Inc., reserves the right to change products or specifications without notice. 2008 Eyelytics, Inc. All rights reserved

EyeLytics

Main/Baseline Profile HD H264 Encoder

3.3 Bitstream Frame Buffer Data Write Operation

The Bitstream frame buffer interface is different from the standard frame buffer interface. This is done to reduce memory usage for the whole system. Designers are highly recommended to use the fb_wrbuf_share module to convert the Bitstream interface to the standard frame buffer interface. The fb_wrbuf_share module can be downloaded from www.eyelytics.com/download/download.html. 1) Frame buffer asserts BS_FB_wdata_wen signal to allow data transfer. If Frame buffer can accept data all the time, BS_FB_wdata_wen can be tied to high. 2) Encoder asserts BS_FB_wdata_wr signal to send data to Frame Buffer. 3) The BS_FB_wdata and BS_FB_be are valid only when both BS_FB_wdata_wen and BS_FB_wdata_wr are asserted. 4) Unlike the standard frame buffer write operation, BS_FB_wdata and BS_FB_be wont be zeroed out by the encoder. 5) Frame buffer is expected to have an internal buffer of 64 64-bit words. When this internal buffer is filled more than half full, the frame buffer interface is expected to assert the BS_FB_wdata_pause signal.

April 20th, 2010 Eyelytics Inc., reserves the right to change products or specifications without notice. 2008 Eyelytics, Inc. All rights reserved

EyeLytics
3.4 Frame Buffer Data Read Operation

Main/Baseline Profile HD H264 Encoder

1) Frame buffer is expected to assert the FB_rdata_valid signal one cycle for each 64-bit data. This means FB_rdata_valid should be asserted for two cycles for each starting address to transfer 16 bytes. 2) Frame buffer should also present the FB_rdata during the same cycle that FB_rdata_valid is asserted. 3) When FB_rdata_ren signal is present in the encoder interface, Frame Buffer should only assert FB_rdata_valid when FB_rdata_ren is asserted in the same cycle. This means Frame Buffer is required to have buffering for each port that uses FB_rdata_ren.

To simplify the users design and to reduce memory usage, a fb_rdbuf_share module can be downloaded from www.eyelytics.com/download/download.html. This fb_rdbuf_share module allows multiple clients (e.g. IN and MCC of the encoder) to share a single buffer. This reduces amount of memory needed in the whole system.

3.5 Frame Buffer Performance Recommendations

Frame buffer performance directly affects the encoder performance. As a result, the frame buffer designer should pay careful attention to the following: 1) Allow multiple pending requests. This can be done by using a small FIFO to buffer up the requests. 2) Arbitration has to be as fair as possible. 3) Frame buffer should be optimized to do long bursts and random accesses. Low latency is not as important. 4) A well designed high efficiency Frame buffer with 32-bit 150MHz DDR2-SDRAM should have adequate memory bandwidth to support the encoder.

April 20th, 2010 Eyelytics Inc., reserves the right to change products or specifications without notice. 2008 Eyelytics, Inc. All rights reserved

EyeLytics

Main/Baseline Profile HD H264 Encoder

4 Video In Frame Buffer Storage Formats

Video input data is passed to the encoder through the frame buffer. Video input data should be stored inside the frame buffer with luma and chroma component separated. Chroma component should be stored 0x800000 bytes after the luma component counterpart. Both Luma and Chroma frames are partitioned into 128 byte columns. Each column has 2K lines. Each column is stored in frame buffer after the column from the left. If the frame has a horizontal size that is not multiple of 128 bytes or the frame has a vertical size that is less than 2K lines, the storage should be left unpacked. User can choose either not writing to those storage spaces, to fill them with any content, or to use the storage space for other applications.

April 20th, 2010 Eyelytics Inc., reserves the right to change products or specifications without notice. 2008 Eyelytics, Inc. All rights reserved

EyeLytics

Main/Baseline Profile HD H264 Encoder

Within each 128 bytes x 2K lines column, the Luma data should be stored in a linear raster scan order. However, the Chroma data is Cb/Cr component interleaved. Also, since the encoder uses 4:2:0 format, every other chroma line is not used.

5 Reference Frame Frame Buffer Storage Requirement

When doing a P frame encode, the previous decoded output is used as the current encode reference frame. The decoded output is stored to the frame buffer using the DB_FB signals. The ME_FB signals read the reference frame to perform motion estimation. The MCC_FB signals read the reference frame to perform motion compensation. The luma and chroma components of the reference frame are stored in separate locations. The luma frame is always stored with bit 23 of the FB_address and base_address set to zero. The chroma frame is always stored with bit 23 of the FB_address and base_address set to one. Since each address location is a 32-bit word, the chroma reference will be 0x2000000 byte away from the luma reference counter part. Both luma and chroma reference data are built with 1Kbyte reference building blocks. Each 1K byte reference building block stores reference info for a 16x64 pixel region. The following shows how these reference building blocks are arranged in the Frame Buffer.

April 20th, 2010 Eyelytics Inc., reserves the right to change products or specifications without notice. 2008 Eyelytics, Inc. All rights reserved

EyeLytics

Main/Baseline Profile HD H264 Encoder

April 20th, 2010 Eyelytics Inc., reserves the right to change products or specifications without notice. 2008 Eyelytics, Inc. All rights reserved