JP5547198B2 - Digital video coding using interpolation filters and offsets - Google Patents

Description

  This application claims the benefit of US Provisional Patent Application No. 61 / 102,789, filed Oct. 3, 2008, which is incorporated herein by reference in its entirety.

  The present disclosure relates to digital video coding, and more particularly to video coding techniques in which interpolation filters and offsets are used.

  Digital video functions include digital television, digital direct broadcast system, wireless broadcast system, personal digital assistant (PDA), laptop or desktop computer, digital camera, digital recording device, video game device, video game console, cellular phone or satellite radio It can be incorporated into a wide range of devices, including telephones. Digital video devices are MPEG-2, MPEG-4, or ITU-T H.264. Implement video compression techniques, such as those defined by H.264 / MPEG-4, Part 10, Advanced Video Coding (AVC), or video compression techniques described in other standards, to transmit digital video information more efficiently And receive. In video compression techniques, spatial prediction and / or temporal prediction can be performed to reduce or remove redundancy inherent in video sequences.

  Intra coding relies on spatial prediction to reduce or eliminate spatial redundancy between video blocks within a given coding unit. Intercoding relies on temporal prediction to reduce or eliminate temporal redundancy between video blocks in consecutive coding units of a video sequence. In the case of intercoding, the video encoder identifies in the reference unit a predictive block that closely matches the block in the unit to be encoded and generates a motion vector that indicates the relative displacement between the coded block and the predictive block And perform motion estimation and compensation. The difference between the coded block and the prediction block constitutes residual information. Thus, an inter-coded block can be characterized by one or more motion vectors and residual information.

  In some coding processes, motion vectors may have fractional pixel values that allow the video coder to track motion with a higher accuracy than integer pixel values. In order to support the identification of predicted blocks using fractional pixel values, the encoder applies an interpolation operation to the reference unit to generate values at sub-pixel positions, such as quarter-pixel positions or half-pixel positions. H. The H.264 standard specifies the use of a fixed interpolation scheme for subpixel positions. In some cases, various interpolation filters can be selected to improve coding efficiency and prediction quality. For example, the encoder can selectively apply a different set of fixed or adaptive interpolation filters. Also, in order to further improve the quality of prediction, the encoder can add an offset to the sub-pixel position value after interpolation.

  In general, this disclosure describes techniques for encoding digital video data using interpolation filters and offsets. The encoder can be configured to select an interpolation filter for sub-pixel accuracy motion estimation based on historical interpolation results obtained for previously encoded video units, such as frames or slices. The encoder can also be configured to calculate and assign an offset to the interpolated sub-pixel position based on the unit and / or block difference between the reference unit and the unit to be coded. Offset calculation and assignment can be performed prior to motion estimation.

  In addition, motion estimation can be improved so that motion search considers subpixel locations where offsets were previously added and evaluates subpixel locations with non-zero offsets. In some cases, interpolation filter selection, offset calculation, and / or improved motion estimation for a given unit can be performed in a single coding pass. The encoder, in some examples, allows the encoder to select an interpolation filter that gives the lowest interpolation error at each sub-pixel position given historical information about one or more previously encoded units. , Errors for each interpolation filter and each sub-pixel position on each unit can be accumulated.

  In one example, this disclosure interpolates sub-integer pixels of a reference video unit using a selected interpolation filter and performs motion estimation of the current video unit before performing motion estimation of the current video unit. Video code comprising: applying an offset to at least some of the sub-integer pixels of the reference video unit; and encoding a block of the current video unit using motion estimation based on the reference video unit Provide a method

  In another example, this disclosure interpolates sub-integer pixels of a reference video unit using a selected interpolation filter and performs motion estimation of the current video unit before performing motion estimation of the current video unit. Before performing, apply an offset to at least some of the sub-integer pixels of the reference video unit and encode a block of the current video unit using motion estimation based on the reference video unit A video encoding device including a video encoder configured as described above is provided.

  In additional examples, the present disclosure interpolates sub-integer pixels of a reference video unit using a selected interpolation filter and performs motion estimation of the current video unit before performing motion estimation of the current video unit. Programmable to apply an offset to at least some of the sub-integer pixels of the reference video unit and to encode a block of the current video unit using motion estimation based on the reference video unit before performing A computer readable storage medium encoded with instructions for causing a processor to perform is provided.

  In some cases, encoding a block may comprise performing block motion estimation only once such that encoding is performed in a single pass. A set of interpolation filters can be selected based on historical interpolation results for one or more previously encoded video units. The interpolation filters correspond to respective sub-integer pixels, and the interpolation filters include different interpolation filters for at least some of the sub-integer pixels. The offset can be determined in frame units or block units.

  The details of one or more examples are set forth in the accompanying drawings and the description below. Other features, objects, and advantages will be apparent from the description and drawings, and from the claims.

1 is a block diagram illustrating an example video encoding and decoding system. FIG. 2 is a block diagram illustrating an example of a video encoder configured to perform video encoding using interpolation and offset. FIG. 3 is a block diagram illustrating an example of a video decoder configured to decode video data encoded by the encoder of FIG. 2. FIG. 5 is a conceptual diagram illustrating integer pixel locations associated with prediction data and sub-integer pixel locations associated with interpolated prediction data. FIG. 5 is a conceptual diagram illustrating integer pixel locations of reference units along with assignment of offset values. FIG. 5 is a conceptual diagram illustrating integer pixel locations of reference units along with assignment of offset values. FIG. 5 is a conceptual diagram illustrating integer pixel locations of reference units along with assignment of offset values. 6 is a flowchart illustrating an exemplary operation of a video encoder when encoding video data using interpolation and offset. 7 is a flowchart illustrating an example method for identifying an optimal set of interpolation filters for a history unit to calculate an interpolation value for a sub-pixel of the current unit.

Detailed description

  This disclosure describes techniques for encoding digital video data using interpolation and offset. For example, the encoder can be configured to select an interpolation filter and assign an offset to a sub-pixel location in the reference video unit to support motion estimation of the unit to be coded. A coding unit may be, for example, a video frame or slice that includes a video block, such as a macroblock. Filters and offsets can be selected prior to motion estimation for a given unit based on the historical interpolation results of one or more previously coded units so that coding can be performed in a single pass. Thus, in some cases, encoding a block may comprise performing block motion estimation only once, ie, only once, such that encoding is performed in a single pass.

  Encoding video data in a single pass requires that motion estimation be applied only once to a given block in the unit to be coded. Instead of performing motion estimation and compensation once to select an interpolation filter for each sub-pixel location and then re-applying the selected interpolation filter, the encoder can use one or more previously encoded units An interpolation filter can be selected based on the history interpolation result obtained for the subpixel positions.

  Further, the encoder can be configured to calculate and assign an offset to the interpolated sub-pixel position before motion estimation of the unit to be coded is performed. The offset can be determined based on the unit and / or block difference between the reference unit and the unit to be coded. In some examples, after interpolation and offset selection, consider the offset previously added to the subpixel location and force the motion search to evaluate subpixel locations with non-zero offset values. Motion estimation can be improved. In some implementations, a combination of these features may allow encoding to be performed in a single pass with improved processing speed and little or no performance loss.

  In some cases, the video encoder may be based on an interpolation error generated by motion estimation of a previously coded unit (eg, N-1), such as a coding unit that is currently used as a reference unit for unit N. An interpolation filter to be used for unit N motion estimation can be selected. As an example, after the previously coded reference unit N-1 is encoded, the interpolation error between the block in the reference unit N-1 and the block in the previous reference unit (eg, N-2) is , At each of the subpixel locations. Using this historical error information in unit N-1, the video encoder selects the optimal interpolation filters that would have reduced the error value, and then uses those interpolation filters to determine in unit N's motion estimation. The subpixel values in unit N-1 for use can be interpolated.

  In other cases, assuming a set of interpolation filters that may be selected for use in sub-pixel motion estimation, the video encoder may, for each interpolation filter and each sub-pixel position, determine the previously encoded unit and its Interpolation errors with one or more reference units (ie, cumulative interpolation error values of subpixel positions over frames N-1, N-2, N-3, etc.) can be accumulated. This cumulative interpolation error value can serve as a historical interpolation result that can be used to select an interpolation filter for the interpolation of subpixel positions in the reference unit to be used in motion estimation for the next unit to be coded. it can. For example, for a given subpixel position, the interpolation filter with the lowest cumulative error value can be selected for interpolation.

  The accumulated error value can be scaled down over time to prevent overflow and to provide a windowing effect that weights more heavily on the contribution of more recently coded units than on previously coded units. For example, the cumulative error value of each interpolation filter applied to a given subpixel position is the weighted sum of the individual error values for each interpolation filter and subpixel position on a series of multiple coding units, and more The error value of the recently coded unit can be weighted more heavily. The encoder can evaluate the error value after coding each unit and select an interpolation filter that yields the lowest cumulative error for each sub-pixel position.

  Thus, to select an interpolation filter, the video encoder can rely on the historical interpolation error of one previously coded video unit or multiple previously coded video units. In particular, as described above, the video encoder selects a set of interpolation filters that would have produced the optimal interpolation error for previously coded units and applies those set of interpolation filters to A motion estimation of the coding units of can be performed. In this sense, if interpolation filters that would have produced optimal results were used in the motion estimation of previously coded units, the current unit motion estimation is performed using those interpolation filters. . Alternatively, as described above, the video encoder can select an interpolation filter that has generated a cumulative error value on a series of units. In each case, the video encoder relies on the historical interpolation results of one or more previously coded units.

  For example, a DC offset value can be applied to the interpolated pixel value to compensate for illuminance changes between different video coding units. The DC offset can be applied at the frame level so that all pixels in the frame use the same DC offset, or can be applied individually at the sub-pixel location level. The calculation and assignment of the DC offset value can be based on the DC frame and block difference between the reference unit and the current unit to be encoded. The calculation can be performed on an “a priori” basis before the motion estimation of the current unit to be encoded is performed. By analyzing the DC frame and block differences between the current unit and the reference unit, when no motion vector information is available, an offset can be applied before encoding the current unit, It becomes possible.

  H. Some video encoders, such as an encoder built to perform according to the H.264 standard, can predict a frame from multiple previously encoded and transmitted frames. These reference frames are typically stored in one or two lists, and each list can contain several frames indexed with a positive integer. In general, a frame indexed with 0 (1 in each list if two lists are used) is a frame that is closer in time to the current frame. The sub-pixel offset determined using the procedure described in this disclosure is generally used only for the reference frame closest to the current frame (the frame indexed with index 0). On all other frames, a simple DC offset can be used regardless of subpixel location. This offset can be calculated as the difference in visibility between the reference frame and the current frame and can be applied on the current frame.

  An improved motion estimation process can be applied to take into account the DC offset previously added to the subpixel location and force the evaluation of the subpixel location with a non-zero DC offset in the motion search. Predictive motion estimation can reduce the complexity of motion search by predicting locations that are more likely to correspond to blocks to be coded and by structuring the search into patterns. However, the predictive motion estimation process may skip testing several subpixel positions that have been assigned a DC offset value. This limitation can be overcome by performing a motion search that explicitly searches for subpixel locations with a defined DC offset during distortion estimation. In this way, motion search can be configured to require that subpixel locations with non-zero offsets be explicitly searched.

Further, the motion search can be configured to calculate a virtual block DC offset. For example, the SAD error calculated during motion search can take into account all possible offsets. Motion search compares blocks (calculates SAD error between blocks) before applying an offset. It may be possible to improve performance by giving the motion search an offset that is calculated so that it can be used to take this possibility into account. For example, as shown in Table 1 below, each time the algorithm determines that a subpixel offset must be used for each location, a “virtual” offset {−2, −1,0,1, After adding one of 2,3} to the reference block, the SAD calculation of the two blocks can be calculated six times.

The motion vector with the smallest SAD can be selected. An offset value that appears in more than one location (eg, offset “2” that appears in both subpixel location 1 and subpixel location 11) can only be attempted once. That is, the motion estimation unit can search only one of a plurality of subpixels each having the same calculated offset value.

  FIG. 1 is a block diagram illustrating an example video encoding and decoding system 10 that may implement the techniques of this disclosure. As shown in FIG. 1, the system 10 includes a source device 12 that transmits encoded video to a destination device 16 via a communication channel 15. Source device 12 and destination device 16 may comprise any of a wide range of devices, including wired devices and wireless devices. In some cases, source device 12 and destination device 16 may communicate video information via a wireless communication device, such as a so-called cellular or satellite radiotelephone wireless handset, or communication channel 15, in which case communication Channel 15 may comprise any wireless device that includes a wireless communication medium.

  In the example of FIG. 1, the source device 12 may include a video source 18, a video encoder 20, a modulator / demodulator (modem) 22, and a transmitter 24. The destination device 16 can include a receiver 26, a modem 28, a video decoder 30, and a display device 32. The video encoder 20 of the source device 12 may be configured to apply one or more coding techniques described in this disclosure as part of the video encoding process. Similarly, the video decoder 30 of the destination device 16 can be configured to apply one or more coding techniques described in this disclosure as part of the video decoding process.

  The encoding techniques described in this disclosure may be performed by any encoding device that supports motion compensated interpolation to subpixel resolution. In the example of FIG. 1, the source device 12 generates coded video data for transmission to the destination device 16. Devices 12, 16 can operate substantially symmetrically such that each of devices 12, 16 includes a video encoding and decoding component. Accordingly, the system 10 can support one-way or two-way video transmission between the video device 12 and the video device 16 for, for example, video streaming, video playback, video broadcast, or video telephony communication. .

  The video source 18 of the source device 12 can include a video capture device, such as a video camera, a video archive containing previously captured video, or a video feed from a video content provider. As a further alternative, video source 18 may generate computer graphic-based data as source video, or a combination of live video, archive video, and computer-generated video. In some cases, if video source 18 is a video camera, source device 12 and destination device 16 may form so-called camera phones or video phones. In each case, the captured video, pre-captured video, or computer-generated video can be encoded by video encoder 20. The encoded video data can be modulated by the modem 22 according to the communication standard and transmitted to the destination device 16 via the transmitter 24. The modem 22 can include various mixers, filters, amplifiers or other components designed for signal modulation. The transmitter 24 may include circuitry designed to transmit data, including amplifiers, filters, and one or more antennas.

  The receiver 26 of the destination device 16 receives the information via the channel 15 and the modem 28 demodulates the information. The video decoding process performed by video decoder 30 may utilize interpolation filters, offset values, motion vectors, and residual information to decode and play back video data received from device 12. For example, information communicated over channel 15 may include offset information defined by video encoder 20 for a particular frame, slice and block. The display device 32 displays the decoded video data to the user and may be a variety of displays, such as a cathode ray tube, liquid crystal display (LCD), plasma display, organic light emitting diode (OLED) display, or another type of display device. Any of the devices can be provided.

  In the example of FIG. 1, the communication channel 15 comprises any wireless or wired communication medium, such as a radio frequency (RF) spectrum or one or more physical transmission lines, or any combination of wireless and wired media. be able to. Communication channel 15 may form part of a packet-based network, such as a local area network, a wide area network, or a global network such as the Internet. Communication channel 15 generally represents any communication medium or collection of various communication media suitable for transmitting video data from source device 12 to destination device 16. Communication channel 15 may include routers, switches, base stations, or any other equipment useful for enabling communication from source device 12 to destination device 16.

  The video encoder 20 and the video decoder 30 are alternatively described in ITU-T H.264 as MPEG-4, Part 10, Advanced Video Coding (AVC). It can operate according to a video compression standard, such as the H.264 standard. However, the techniques of this disclosure are not limited to any particular coding standard. Although not shown in FIG. 1, in some aspects, video encoder 20 and video decoder 30 can be integrated with an audio encoder and decoder, respectively, and include appropriate MUX-DEMUX units, or other hardware and software. Can handle both audio and video encoding in a common data stream or separate data streams. Where applicable, the MUX-DEMUX unit is ITU H.264. It can be compliant with other protocols such as the H.223 multiplexer protocol or User Datagram Protocol (UDP).

  ITU-TH. The H.264 / MPEG-4 (AVC) standard was developed by ITU-T Video Coding Experts Group (C) as a result of a joint partnership known as Joint Video Team (JVT) together with ISO / IEC Moving Picture Experts Group (MPEG). It was. In some aspects, the techniques described in this disclosure are generally described in H.264. It can be applied to a device conforming to the H.264 standard. H. The H.264 standard is an ITU-T recommendation H.264 dated March 2005 by the ITU-T Study Group. H.264 "Advanced Video Coding for generic audioservices". H.264 standard or H.264 standard. H.264 specification or H.264 H.264 / AVC standard or specification. Joint Video Team (JVT) H.264 / MPEG-4 AVC is continuously being expanded.

  Video encoder 20 and video decoder 30 each include one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, software, hardware , Firmware, or any combination thereof. Each of video encoder 20 and video decoder 30 can be included in one or more encoders or decoders, each of which is part of a combined encoder / decoder (codec), respectively, mobile device, subscriber device, broadcast device, Can be integrated into a server.

  A video sequence typically includes a series of video frames. Video encoder 20 operates on video blocks within individual video frames to encode video data. Video blocks can be fixed or change in size, and may vary in size depending on the specified coding standard. Each video frame includes a series of slices. Each slice can include a series of macroblocks, which can be arranged in sub-blocks. As an example, ITU-T H.I. The H.264 standard supports intra prediction of various block sizes, such as 16 × 16, 8 × 8, or 4 × 4 for luma components and 8 × 8 for chroma components, and for luma components Inter-predict various block sizes, including 16x16, 16x8, 8x16, 8x8, 8x4, 4x8 and 4x4, and corresponding scaled sizes for chroma components to support. The video block may comprise a block of pixel data or a block of transform coefficients after a transform process, such as a discrete cosine transform or a conceptually similar transform process.

  Smaller video blocks provide better resolution and can be used for video unit locations that contain higher levels of definition. In general, macroblocks and various sub-blocks can be considered video blocks. Further, a slice or frame can be considered a video unit comprising a series of video blocks such as macroblocks and / or sub-blocks. Each frame may be a unit that can be decoded independently of a video sequence, and each slice may be a unit that can be decoded independently of a video frame. The term “coding unit” refers to any unit that can be decoded alone, such as an entire frame, a slice of a frame, or another independently decodable unit defined according to applicable coding techniques.

  After predictive coding and Following any transformation, such as a 4 × 4 or 8 × 8 integer transform used in H.264 / AVC or Discrete Cosine Transform (DCT), quantization can be performed. Quantization generally refers to the process of quantizing a coefficient to reduce the amount of data used to represent the coefficient. The quantization process can reduce the bit depth associated with some or all of the coefficients. For example, a 16-bit value can be truncated to a 15-bit value during quantization. After quantization, entropy coding may be performed according to, for example, content adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), or another entropy coding process.

  According to the techniques of this disclosure, video encoder 20 may encode a video unit in a single pass using an interpolation filter and an offset value. Thus, in some cases, encoding a block may comprise performing block motion estimation only once such that encoding is performed in a single pass. In one example, video encoder 20 may include a set of interpolation filters with fixed coefficients. That is, the set of interpolation filters does not necessarily include an adaptive filter that the encoder will dynamically determine the coefficients. Alternatively, the encoder 20 can select an interpolation filter for each subpixel position from a plurality of alternative fixed interpolation filters.

  Different interpolation filters can be selected for interpolation at different subpixel positions. Each sub-integer pixel location can have a default interpolation filter and a plurality of other interpolation filters that can be selected to interpolate the respective sub-integer pixel location based on the historical interpolation results. Video encoder 20 first calculates a sub-integer pixel value of the reference unit using a default interpolation filter at each sub-integer pixel location, thereby providing a first inter-coded video unit, eg, a first P frame. Can be encoded.

  When the sub-integer pixel values in the reference unit are interpolated, video encoder 20 uses both the integer and fractional sub-pixel positions of the reference unit to move the macroblock of the next P frame being encoded. A vector can be calculated. In particular, the video encoder 20 may determine, based on pixel difference calculations such as, for example, absolute difference sum (SAD), sum of square differences (SSD), etc., before it most closely matches each block in the current unit being coded. Blocks in the coded reference unit can be searched.

  After generating a motion vector for each macroblock of the first P unit to be coded for the prediction block in the reference unit, video encoder 20 may, for example, Based on the difference in pixel values from the actual macroblock of the P unit, the error value of each macroblock of the first P unit can be determined. As described above, the error value can comprise, for example, an absolute value difference sum (SAD) or a square difference sum (SSD) value.

  Video encoder 20 can then determine whether a different set of interpolation filters would have resulted in a lower error value. In particular, video encoder 20 recalculates the value of the sub-integer pixel value of the reference macroblock for each subpixel position, and calculates the error value of the recalculated macroblock identified by the motion vector. An interpolation filter that would have resulted in an error value can be identified. Video encoder 20 can then determine which interpolation filter provided the lowest absolute error value or lowest average error value for the unit at each sub-integer pixel location.

  In this way, video encoder 20 interpolates to interpolate the sub-integer pixel positions of the reference unit used for the current unit based on the interpolation filter that would have produced the optimal result for the reference unit. A filter can be selected. In some examples, video encoder 20 may further calculate a cumulative error value for each interpolation filter at each sub-integer pixel location over a series of code units. Video encoder 20 can use the accumulated error value to select an interpolation filter that provides a historically optimal value for the sub-integer pixel location.

  Video encoder 20 may identify which interpolation filter produced the lowest error value for the previous frame or over a series of previous frames. The error value can be tracked for each interpolation filter and each subpixel position over a series of frames. Each sub-pixel position can be calculated using a respective interpolation filter. Multiple interpolation filters may be available for selection for each subpixel location. A set of interpolation filters can be selected to support interpolation of multiple respective subpixel positions.

  In one example, before video encoder 20 encodes the next prediction unit, such as a P frame or B frame, video encoder 20 is lowest for the reference unit (s) of the previously encoded unit. The value of the sub-integer pixel of the previously encoded unit is calculated using the interpolation filter that would have resulted in the error value. Similarly, for each subsequent inter-coded frame, video encoder 20 can identify which interpolation filter would have resulted in the lowest error value for that reference unit, and most recently encoded. These interpolation filters are applied to the most recently encoded unit to calculate the values of the sub-integer pixels of the units, and then those values will be used as reference frames for motion estimation.

  In some cases, video encoder 20 may identify which interpolation filter has the lowest historical cumulative interpolation error value for each sub-pixel location over a series of coding units. Generate the lowest error value for each sub-pixel position to support motion estimation for the next frame to be coded by tracking the error value generated for each sub-pixel position by each of multiple interpolation filters The interpolated filter can be selected for interpolation. In this way, the video encoder 20 can select an interpolation filter at the sub-pixel position based on the historical interpolation result of the previously coded frame. The historical interpolation results analyzed by the video encoder 20 may take into account several units of interpolation error, or possibly only a single unit of interpolation error. In particular, video encoder 20 analyzes the accumulated error results over multiple frames, or simply analyzes the interpolation error between the current unit N's reference unit (eg, N-1) and the previous reference unit N-2. can do.

  As an example, if there are n subpixel positions to be interpolated and m different interpolation filters that can be selected to interpolate each of the n subpixel positions, video encoder 20 may have n × m Different error values can be calculated. Each error value is a given one of n subpixel position values calculated according to a given one of the m interpolation filters for the actual pixel value in the previous unit to be coded. The error value for one can be expressed. On a block basis, the error value is the pixel value between the pixel value of the block to be coded and the pixel value at the sub-pixel location in the interpolated prediction block generated using the given set of interpolation filters. Can be based on the difference. The sub-pixel location interpolation filter can be applied to all corresponding sub-pixel locations on the frame. Similarly, error values for each interpolation filter and sub-pixel location can be given on the frame. For example, the error value can be an average value, an intermediate value, or a total value of a given frame.

  As described above, the error value of each subpixel position and interpolation filter can be accumulated over a series of units to generate an accumulated error value for each combination of interpolation filter and subpixel position. In some cases, the error values across a series of units can be summed using a weighted sum, and the error value of the more recently coded frame is weighted more heavily than the previously coded frame for the current frame to be coded. The

  As an example, for a unit to be coded, the accumulated error value is the sum of the weighted error values from the first unit in the window to the last unit in the window immediately before the unit to be coded, A sliding window can be used so that the error values of are weighted with a weighting factor that is smaller than the error values of later units. The weighting factor can be incrementally increased from earlier coded units to lower coded units. Furthermore, the accumulated error can be periodically scaled down to prevent overflow. The error values of earlier coded units can eventually be phased out as they go out of the window.

  In order to select a particular interpolation filter at a particular subpixel position for coding the current unit, video encoder 20 performs a weighted sum of error values for that interpolation filter and subpixel position over a range of previously coded units. Can be evaluated. The interpolation filter that produces the lowest cumulative error for a given subpixel position can then be selected as the interpolation filter to be used for interpolation of that subpixel position for the next unit to be coded.

  The interpolation filter selected for a given subpixel location can be used as an interpolation filter for all pixels at the corresponding subpixel location in the block across the coding unit. In particular, the selected filter is used to interpolate its sub-pixel position in the reference unit to support motion search for the next unit to be coded. If there are n subpixel positions in each block, encoder 20 may select a set of n different interpolation filters for each subpixel position. The same set of interpolation filters can be used for subpixel positions in each block of the unit.

  After interpolation and prior to motion estimation, video encoder 20 may also apply an offset value to the pixel value at the interpolated subpixel location. Video encoder 20 may calculate a plurality of offset values for a coded unit of video data, such as a frame or a separately decodable portion of a frame, such as a slice. Different offset values may be associated with a plurality of different integer and sub-integer pixel locations associated with the video block. A sub-integer pixel location defines a generally interpolated data location that is interpolated or extrapolated as described above based on the data at the integer pixel location. Can do. Video encoder 20 applies offset values to integer pixel positions and sub-pixel positions in the predicted video block to generate an offset predicted video block, and encodes the video block of the coding unit based on the offset predicted video block. be able to.

  Video encoder 20 also encodes the offset value as part of a coded bitstream that includes a coded video block of coding units, and transmitter 24 of source device 12 sends the coded bitstream to the receiver of destination device 16. 26 can be transmitted. In particular, video encoder 20 may apply an offset value by adjusting a pixel value of a predictive video block, and generate a residual based on the offset predictive video block to generate a video block based on the offset predictive video block. Can be encoded. For example, the residual can be generated by subtracting the block to be coded from the appropriate offset prediction video block. Since the offset is added to the pixel value of the offset predictive video block based on the location of the predictive video block, coding efficiency can be improved, for example, during flash or background illumination changes.

  When performing a motion search for the unit to be coded, video encoder 20 identifies the prediction block in the reference coding unit. In some cases, the motion search algorithm can be configured to skip some subpixel values. In particular, some predictive motion estimation may reduce the complexity of motion search by predicting locations that are more likely to correspond to blocks to be coded and by structuring the search into patterns. it can. However, the predictive motion estimation process may skip testing several subpixel positions that have been assigned a DC offset value. By imposing the requirement that motion search is to explicitly search for subpixel locations with non-zero offset values, video encoder 20 can be configured to overcome this limitation in the predictive motion estimation process.

  At destination device 16, video decoder 30 receives an identifier for an interpolation filter that is used to encode the video data and a plurality of offset values for each coding unit of the video data. For example, these identifiers can be coded by video encoder 20 as syntax elements in a unit header, such as a frame header or slice header, provided in the coded bitstream. Video decoder 30 may store the same set of interpolation filter parameters as video encoder 20. The video decoder 30 applies the interpolation filter identified by the video encoder 20 to calculate the value of the sub-integer pixel of the predictive video block, applies the offset value to the predictive video block to generate an offset predictive video block, and Decode the video block of the coding unit based on the offset prediction video block.

  In particular, using motion vectors and interpolation filters, video decoder 30 obtains a predicted block for each block in the unit to be decoded and adds residual information to the predicted block to obtain the desired block in the decoding unit. You can play blocks. In this way, the interpolation filter and offset value are defined and applied as part of the encoding process at video encoder 20, and then information identifying the interpolation filter and offset value is encoded from source device 12 to destination device 16. Is communicated as part of a generalized bitstream. The offset value is then applied to the interpolated prediction data as part of the decoding process at video decoder 30 to reconstruct the pixel values of the video unit.

  FIG. 2 is a block diagram illustrating an example of a video encoder 20 that can apply the interpolation and offset techniques described in this disclosure. Video encoder 20 may perform intra-coding and inter-coding of blocks within a video unit, such as a frame or slice. However, for ease of explanation, the intra-coding components of video encoder 20 are not shown in detail in FIG. Intra coding relies on spatial prediction to reduce or remove the spatial redundancy of video within a given video unit. Intercoding relies on temporal prediction to reduce or remove temporal redundancy of video in adjacent units (eg, frames) of the video sequence. Intra mode (I mode) refers to a space-based compression mode, and inter modes such as prediction (P mode) or bidirectional (B mode) refer to time-based compression modes. The techniques of this disclosure can be applied during inter-coding by video encoder 20.

  As shown in FIG. 2, video encoder 20 receives a current video block in a video frame to be encoded. In the example of FIG. 2, the video encoder 20 includes a motion estimation unit 36, a motion compensation unit 35, a reference frame store 34, an adder 48, a transform unit 38, a quantization unit 40, and an entropy coding unit 46. including. For video block reconstruction, video encoder 20 also includes an inverse quantization unit 42, an inverse transform unit 44, and an adder 51. A deblocking filter (not shown) that filters block boundaries to remove blockiness artifacts from the reconstructed video may also be included. If necessary, the deblocking filter can filter the output of the adder 51.

  During the encoding process, video encoder 20 receives the video block to be coded, and motion estimation unit 36 and motion compensation unit 35 perform inter-predictive coding. Motion estimation unit 36 and motion compensation unit 35 can be highly integrated, but are shown separately for conceptual purposes. Motion estimation is generally considered as a process of generating motion vectors that estimates the motion of a video block, resulting in the identification of the corresponding prediction block in the reference unit. The motion vector can indicate, for example, the displacement of the prediction block in the prediction frame (or other coding unit) relative to the current block being coded in the current frame (or other coding unit). Motion compensation is generally regarded as a process of fetching or generating a prediction block based on a motion vector determined by motion estimation. Again, the motion estimation unit 36 and motion compensation unit 35 can be functionally integrated. For purposes of illustration, motion compensation unit 35 will be described as performing the selection of the interpolation filter and offset technique of the present disclosure.

  For illustrative purposes, a coding unit in the form of a frame will be described. However, other coding units such as slices can also be used. Motion estimation unit 36 calculates the motion vector of the video block of the inter-coded frame by comparing the video block of the inter-coded frame with the video block of the reference frame in reference frame store 34. The motion compensation unit 35 is one of a plurality of interpolation filters 37 to be applied to calculate pixel values at each of a plurality of subpixel positions in a previously encoded frame, eg, an I frame or a P frame. Select one. That is, the video encoder 20 can select an interpolation filter at each subpixel position in the block.

  The selected interpolation filter can be different for different sub-pixel positions. The set of interpolation filters applied to different subpixel locations in the block can be the same for all blocks in the reference frame. For example, if the interpolation filter F1 is selected for subpixel positions x1, y1 in a block, the same interpolation filter F1 can be used for that same subpixel position in all blocks in the same frame. A subpixel position may alternatively be referred to as a subinteger pixel, subpixel or subpel.

  Motion compensation unit 35 may select an interpolation filter from interpolation filter 37 based on the interpolation error history of one or more previously encoded frames. In particular, after the frame is encoded by transform unit 38 and quantization unit 40, inverse quantization unit 42 and inverse transform unit 44 decode the previously encoded frame. In one example, motion compensation unit 35 applies selected interpolation filter 37 to a previously encoded frame to calculate a sub-integer pixel value for the frame to form a reference frame that is stored in reference frame store 34. To do.

  For subsequent frames, motion compensation unit 35 may apply a different set of interpolation filters 37 to calculate sub-integer pixel values for that subsequent frame. The selection of the different interpolation filters is such that each of the interpolation filters at a given subpixel position so that motion compensation unit 35 selects the interpolation filter 37 with the lowest cumulative error value for each subpixel position over a series of frames. Based on the history of previously encoded frames, such as The error value can be based on the difference between the actual pixel value in a given frame and the interpolated pixel value in the reference frame.

  The interpolation filter 37 can include a plurality of interpolation filters. Each potential sub-integer pixel location may be mapped to one of a set of interpolation filters 37 for calculation of the corresponding pixel value. For example, motion compensation unit 35 applies a 6-tap Wiener filter with coefficients [1, -5, 20, 20, -5, 1] to calculate half pixel position values based on integer pixel position values. A bilinear filter can then be applied to calculate the quarter pixel position value based on the integer pixel position value. In one example, for a half pixel location, a 6-tap Wiener filter may comprise a default filter for application to the half pixel location, and a bilinear filter may comprise a default filter for application to the quarter pixel location.

  Further, each pixel location can be mapped to an alternative interpolation filter, eg, an interpolation filter with a different coefficient. Other interpolation filters can be constructed using various coefficients. In general, fixed interpolation filters have different coefficients (different coefficients specify different frequency responses) and / or their support is different (this is one-dimensional or two-dimensional, vertical, horizontal, or diagonal) Or the number of taps (and thus pixels) used for interpolation may be different. Greater support generally performs better, but adds more complexity with respect to multiplication and addition that must be done to interpolate the pixels.

  In one example, each sub-integer pixel can be mapped to m possible different interpolation filters, and at least some of the m interpolation filters for one sub-integer pixel position are for different sub-integer pixel positions. May be different from the m interpolation filters. For quarter pixel accuracy, the interpolation filter 37 may comprise m * 15 unique interpolation filters, where m is the number of interpolation filters per subpixel. Alternatively, some sub-integer pixels can be mapped to the same interpolation filter. At least one of the interpolation filters for each sub-integer pixel can be assigned as a default interpolation filter for the corresponding sub-integer pixel. For example, the default interpolation filter may be used initially in the video sequence until enough historical interpolation results are accumulated to allow reliable selection of the interpolation filter.

  Motion estimation unit 36 compares the block of reference frames from reference frame store 34 with the block to be encoded of the current frame, eg, a P frame or a B frame. Since the reference frame in the reference frame store 34 includes inter-integer pixel interpolated values, the motion vector calculated by the motion estimation unit 36 can refer to sub-integer pixel locations. The motion estimation unit 36 transmits the calculated motion vector to the entropy coding unit 46 and the motion compensation unit 35. According to the techniques described in this disclosure, motion estimation unit 36 may calculate one motion vector per block and generate a motion vector based on the interpolation filter and offset selected prior to motion estimation. In this way, video encoder 20 can encode units such as frames, slices and blocks in a single pass, i.e., without performing motion estimation of the video block of the coding unit twice.

  Single pass encoding involves selecting an initial interpolation filter and offset, encoding the video using the initial interpolation filter and offset, and creating a new interpolation filter and offset based on the encoding result. In contrast to a multi-pass encoding process that includes selecting and then re-encoding the frame with the newly selected interpolation filter and offset. Instead, the encoder 20 selects an interpolation filter based on the historical interpolation results of one or more previous frames, selects an offset based on the difference between the current frame and one or more previous frames, Then, motion estimation is performed using the selected interpolation filter and offset to generate a motion vector and residual information for each block. Thus, video encoder 20 does not need to recalculate motion vectors in the second or subsequent pass, thereby improving the efficiency of the encoding process. Alternatively, motion estimation can be performed once. The motion compensation unit 35 calculates an error value between the prediction block of the reference frame and the actual block of the encoded frame, generates a residual block value via an adder 48, and these residuals. The block value is encoded by the transform unit 38 and the quantization unit 40 of the video encoder 20.

  In one example, as described above, motion compensation unit 35 can determine which interpolation filter 37 has produced the lowest error for a previously encoded block of reference frames. For example, after encoding a reference frame using a selected set of interpolation filters for motion estimation, motion compensation unit 35 determines which interpolation filter 37 produced the lowest error value at each of the subpixel positions. Can be identified. To do so, motion compensation unit 35 can recalculate the sub-integer pixels of the predicted video block generated to encode the reference frame using a different set of interpolation filters. In particular, when the motion vector calculated by motion estimation unit 36 refers to a sub-integer pixel location of a block of reference frames in reference frame store 34, motion compensation unit 35 uses other interpolation filters 37 to block Recalculate the values of subinteger pixels. The motion compensation unit 35 then calculates an error value corresponding to each of the interpolation filters 37 for the prediction block.

  Motion compensation unit 35 determines which interpolation filter 37 yields the lowest error value for the encoded frame. Even if a frame has already been encoded using a set of interpolation filters, the interpolation filter that would have produced the lowest interpolation error value can be determined after encoding and the motion of the next frame to be encoded Can be used to perform the estimation. After encoding the frame and identifying the set of interpolation filters 37 that would have produced the lowest error for the frame, motion compensation unit 35 may select the selected interpolation filter before storing the frame in reference frame store 34. It can be applied to reconstructed frames. In this way, the motion compensation unit 35 uses a set of interpolation filters 37 determined to be optimal for the previous frame, and a reference frame that can be used for motion estimation to encode the next frame to be coded. The sub-integer pixel value in can be calculated.

  In another example, as described above, motion compensation unit 35 accumulates each error value of interpolation filter 37 at each sub-pixel position and for each new frame to be stored in reference frame store 34, motion compensation unit 35. 35 selects the interpolation filter 37 with the lowest cumulative error for each subpixel position. In this way, motion compensation unit 35 can accumulate interpolation errors between previously encoded frames and their references for each filter and each subpixel position. As described above, the motion compensation unit 35 periodically scales the accumulated error of the interpolation filter 37 to prevent memory overflow and to weight more heavily the recent use of the interpolation filter than the older history use of the interpolation filter 37. Can be down. In this way, the motion compensation unit can be weighted by the contribution of the most recent frame, resulting in a “window” effect of “forgetting” the oldest frame.

  The interpolation error can be calculated based on the difference between the original pixel value of the frame to be encoded and the pixel value of the predicted frame generated by interpolation. A prediction frame may be generated by interpolation and include prediction blocks corresponding to blocks to be coded in a frame to be coded. The difference can be calculated as the difference in pixel values between the prediction block and the coded block, for example, as determined by SSD, SAD or other difference metrics. Different metrics can be summed or averaged over the blocks in the frame to calculate the overall difference value and the difference value at each subpixel location. An interpolation filter used for motion estimation of the next frame to be coded with respect to the reference frame, the set of interpolation filters that would have produced the lowest difference value at each of the previously coded reference frame sub-pixel positions. Can be selected as a set. Alternatively, an interpolation filter that produces the lowest cumulative difference value at each sub-pixel location over the range of coded video frames can be used for motion estimation of the next frame to be coded relative to the reference frame.

  In general, motion compensation unit 35 may select an interpolation filter for each sub-integer pixel location by calculating decision vector D. D can be a vector of length n, where n corresponds to the number of sub-integer pixel locations for each pixel. In one example, n is equal to 15 if video encoder 20 supports quarter pixel accuracy. Each element of D corresponds to one of the interpolation filters at the corresponding sub-integer pixel position. Thus, if each sub-integer pixel has m different possible interpolation filters, each element of D ranges between 0 and m. For the first reference frame, each element of D can be initialized to “0”, where “0” refers to the default interpolation filter for the corresponding pixel location. Motion compensation unit 35 may calculate a value for each subpixel position of the first reference frame using the corresponding default interpolation filter for each subpixel position.

  The motion estimation unit 36 uses each interpolation unit of a frame to be intercoded later by referring to the reference frame that the motion compensation unit 35 has calculated sub-integer pixel values using the interpolation filter identified by the decision vector D. Can be calculated. Motion compensation unit 35 can also calculate the cost matrix C of the inter-coded frame. In one example, for a predictive coded frame, cost matrix C includes a first dimension F corresponding to the interpolation filter for each sub-integer pixel and a second dimension I corresponding to the sub-integer pixel. Thus, the cost matrix can be defined as C [F] [i], where F = [0, m] and I = [0, n]. Each element of C is initially set to zero.

  The motion compensation unit 35 then C [f] [i] + = | if for each pixel of the inter-coded frame and for each interpolation filter of the pixel, if f is in F and i is in i. actual (i) −predicted (i, f) |, where actual (i) is the actual value of the pixel corresponding to sub-pixel i in the pixel of the frame to be intercoded, and predicted (i, f) Is the value of sub-pixel i calculated using the interpolation filter f referenced by the corresponding motion vector, for example using SAD calculation in one example. Thus, each element of C corresponds to the accumulated error of the corresponding subpixel location using the corresponding interpolation filter of the subpixel.

For bi-predicted frames, a similar cost matrix C ₂ [F1] [F2] [i] [j] can be calculated, where i refers to the subpixels of F1 and j is the subpixel of F2. Refers to the pixel, F1 refers to the interpolation filter used to calculate the subpixel position i in the first reference frame, and F2 to calculate the subpixel position j in the second reference frame Refers to the interpolation filter used. Motion compensation unit 35 can then calculate the error value between the intercoded frame and the reference frame, for example, as a SAD value. In order to calculate the SAD value from the cost matrix C, the motion compensation unit 35 uses the filter used to calculate the subpixel according to the following equation to calculate the accumulated error value for each applied subpixel location: The sum can be calculated.

  The motion compensation unit 35 determines the position and the interpolation filter corresponding to that position so that the combination of position and filter can result in the greatest reduction in the error value between the inter-coded frame and the reference frame. In addition, it is possible to iterate through the cost matrix. That is, the motion compensation unit 35 can identify the value C [f] [p] that results in the greatest reduction in the SAD value calculated above. The motion compensation unit 35 can then change the decision vector so that the determined interpolation filter f is used in the future for the determined sub-integer pixel position p. In particular, for decision vector D, sub-integer pixel position p, and interpolation filter f, motion compensation unit 35 can perform D [p] = f. Motion compensation unit 35 may then recalculate the difference, eg, SAD, as if filter f had been used to calculate the sub-integer pixel value at position p using the equation presented above. it can.

Motion compensation unit 35 can repeat this process until the difference does not change substantially (eg, any difference is below the minimum threshold) or the maximum number of iterations has been performed. The above process is summarized in the following pseudo code.

In some examples, motion compensation unit 35 may accumulate the error of each reference frame, select the most recent reference frame, and “forget” the older reference frame. To do so, motion compensation unit 35 calculates the cost matrix for a particular frame as described above, but also for that cost matrix, the value of the earlier cost matrix of the previously coded frame. Fractional weights can be added. For example, for a previously coded frame cost matrix C and its reference frame cost matrix C _REF , motion compensation unit 35 may calculate C = C + (1 / n) * C _REF . In this case, n can be any integer, preferably greater than 1. In this way, the motion compensation unit 35 avoids overflow, prefers to calculate the cost of the new reference frame when selecting the interpolation filter, and makes the reference farther in time from the older reference frame, ie the frame to be encoded. The frame calculation can be “forgotten” or phased out.

As a further illustration, if frame N is to be coded and frames N-1 through NM are already coded, for example, relative to reference frame N-1, selected for motion estimation of frame N The interpolation filter can be selected according to a cost equation calculated as follows.

Where the coefficients a ₁ , a ₂ ,. . . a _M represents a weighting factor whose value becomes progressively smaller. Based on the accumulated cost C of each interpolation filter at a given subpixel position, encoder 20 selects the interpolation filter that produces the lowest value of C.

  Motion compensation unit 35 may also add an offset value, such as a DC offset, to the interpolated prediction data, ie, a sub-integer pixel value of a reference frame in reference frame store 34. Motion compensation unit 35 may assign a DC offset based on the DC difference between the reference frame and the current frame or the DC difference between the block of the reference frame and the block of the current frame. The motion compensation unit 35 can be assigned a DC offset “a priori”, ie before the motion search of the current frame to be encoded is performed, consistent with the ability to perform coding in a single pass.

  The pixel value of the predictive video block can be offset up or down according to an offset associated with an integer or non-integer pixel location of the predictive video block. The interpolation performed by motion compensation unit 35 can define interpolation data at several possible sub-integer pixel locations for different video blocks. Rather than defining a single offset value for a given coding unit, motion compensation unit 35 defines a different offset value for each possible integer and sub-integer pixel location in some implementations. be able to. In other cases, a single offset value can be applied to all pixel values in the prediction block, slice or frame. Motion compensation unit 35 applies the offset value to only a subset of sub-integer pixels in some examples. The prediction data can then apply a location specific offset based on the pixel location associated with the prediction data.

  In one example, motion compensation unit 35 calculates a first set of metrics (eg, average values) associated with each integer and sub-integer pixel location of the video block of the coded frame to be encoded based on the predicted video block, respectively. can do. Motion compensation unit 35 also calculates a second set of metrics (eg, average values) associated with each integer and sub-integer pixel location of the predicted video block used to encode the block of the current frame, respectively. be able to. That is, the first set of metrics is defined by the data being coded, and the second set of metrics is defined by the prediction data in the reference frame used for predictive coding. Motion compensation unit 35 then calculates a plurality of offset values based on the first and second sets of metrics.

  The first set of metrics comprises a first set of average values respectively associated with each integer and sub-integer pixel location of the video block of the coding unit coded based on the predicted video block, and the second set of metrics The metric may comprise a second set of average values associated with each integer and sub-integer pixel location of the predicted video block, respectively. In this case, the plurality of offset values may comprise a difference between the average value of the first set and the average value of the second set. In this way, several different offset values can be defined for several different integer and sub-integer pixel locations based on the actual data coded in the first coding pass. The difference may represent, for example, a difference in illuminance between frames.

  The first set of metrics may comprise a set of average values corresponding to the average of the pixel values at each integer and sub-integer pixel location of the video block of a given coding unit. The second set of metrics is the average value of the set corresponding to the average of the pixel values at each integer and sub-integer pixel location of the prediction block used to predict the current block being coded in that coding unit. Can be provided. The plurality of offset values may comprise a difference between the average value of the first set and the average value of the second set. Each macroblock location can be defined by a single pixel, eg, each pixel in the upper left corner of each macroblock. However, each macroblock can define 16 pixel values that contribute to a particular average value in the first set of average values. The techniques of this disclosure can of course be applied to video blocks of other sizes.

  The offset techniques of this disclosure can be applied to luma blocks, chroma blocks, or both. Different offsets can be defined for each integer and sub-integer pixel location associated with each type of video block (eg, luma block and chroma block). Further, different offsets may be assigned to each block in each particular size, partition or subpartition of each block. In some examples, two offsets can be calculated for a particular pixel or sub-integer pixel location to inter-encode the bi-predictive block, the first offset being temporally related to the current block of the current frame. Represents the average difference (ie, list 0) between the reference frames closer to the reference frame and the second offset is the average difference between the current frame and the reference frame that is further in time ( That is, it represents list 1). After calculating these two offset values, video encoder 20 can determine which of these two offset values should be applied to the pixel or sub-integer pixel value, based on the list 0 The first offset is applied when encoding, and the second offset is applied when encoding based on list 1.

  Computing a block-based DC offset requires collecting statistics (in the form of a histogram) on the DC difference between collocated blocks of some size (eg, 16 × 16 pixels) in the current frame and the reference frame. . Compare the block in the reference frame with the block in the current frame and calculate the following two quantities for each collocated block in the two frames.

1) squared error of pixel difference (err0 described below with respect to exemplary functions blockDC1 and blockDC2), and 2) squared error of pixel difference after block average DC is subtracted from each pixel value (illustrated) Err1 described below with respect to the typical functions blockDC1 and blockDC2)
By comparing these two quantities (eg, by checking if the error at 1 is greater than twice the error at 2), whether the blocks are sufficiently similar (ie, between collocated blocks) If this is the case, it may be possible to accumulate these statistics into a vector containing a histogram of block-based DC differences. For practical reasons, the DC differences can be quantized, for example, by rounding those DC differences to integer values.

  By examining the histogram bins, it is possible to derive a range of DC differences and calculate DCmin and DCmax. Some possibilities are as follows.

1) Count the number of bins corresponding to positive and negative DC values with more samples than noSamples (method used in the function blockDC2 below)
2) Count the number of bins corresponding to negative and positive DC values with more samples than noSamples, but stop counting when test fails for the first time, or 3) More than noSamples Find the DC value corresponding to the leftmost and rightmost bins containing the samples (the method used in blockDC1 below)
As an example, if noSamples = 20 and the histogram is as shown in Table 2 below, the first method is DCmin = −2, DCmax = 2, second DCmin = −2, DCmax = 1, and Return the third DCmin = −2, DCmax = 4.

  After estimating the value of the block-based DC difference between the current frame and the reference frame as outlined above, an offset is assigned to the subset of subpixel positions. The mechanism used is known to both the encoder and the decoder and is completely determined by the values of DCmin and DCmax. Thus, unless compatibility with other methods need to be maintained, the transmission of the offset can be greatly simplified by transmitting only these two values.

The offsets O (1) to O (15) are subpixel positions in a predetermined order of 5, 15, 13, 7, 9, 6, 11, 14, 1, 4, 3, 12, 10, 2, 8 Can be assigned. Other orders of assignment are possible. Table 3 below shows the sub-pixel position indexing (position 0 is associated with a full pixel) and offset assignment used in 1/4 pixel interpolation. If less than 15 offsets are assigned, the remaining values are set to zero. The method can be extended to different fractional pixel interpolation. An example of applying a DC offset to these subpixel positions will be described with respect to the example of FIG.

  In Table 3 above, the left four columns indicate the subpixel positions, the middle four columns indicate the 15 offset assignments, and the right four columns indicate the nine offset assignments.

Using the (frame-based) DC difference between the current frame and the reference frame, both the threshold noSamples used in the calculation of DCmin and DCmax and the number of offsets for 1/4 pixel interpolation Judgment can be made. The threshold noSamples can also be calculated by taking into account the size of the image (and thus the number of blocks). In that case, DCmin and DCmax can be further improved. A possible way to do this is illustrated in the following pseudo code.

The quantity roundFact in the above pseudo code is used to quantize the DC range and suppress the offset to 15 or less values. Next, offsets in the range [DCmin, DCmax] are assigned to subpixel positions in the order specified above. If the absolute value of the frame-based DC difference is small (eg, less than 2), the DC difference is requantized in a smaller step (eg, 0.1), creating a new quantity called noOffsets1. If noOffsets1 is less than or equal to a threshold (eg, 8), set the first noOffset1 subpixel position (first in the order specified above) to 0 and set the remaining positions (depending on the sign of the DC difference) ) Add +1 or -1. If noOffset1 is greater than the threshold, +1 or -1 is entered at the first noOffsets1 position and the remaining subpixel positions are set to zero.

  Still referring to FIG. 2, motion compensation unit 35 calculates prediction data based on the prediction block. Video encoder 20 forms a residual video block by subtracting the prediction data from the original video block being coded. Adder 48 represents one or more components that perform this subtraction operation. Transform unit 38 applies a transform to the residual block, such as a discrete cosine transform (DCT) or a conceptually similar transform, to generate a video block comprising residual transform block coefficients. The conversion unit 38 is, for example, H.264 conceptually similar to DCT. Other transformations can be performed, such as transformations defined by the H.264 standard. Wavelet transforms, integer transforms, subband transforms or other types of transforms can also be used. In either case, transform unit 38 applies the transform to the residual block to generate a block of residual transform coefficients. The transformation can transform the residual information from the pixel domain to the frequency domain.

  The quantization unit 40 quantizes the residual transform coefficient to further reduce the bit rate. The quantization process can reduce the bit depth associated with some or all of the coefficients. For example, a 16-bit value can be truncated to a 15-bit value during quantization. After quantization, entropy coding unit 46 entropy codes the quantized transform coefficients. For example, entropy coding unit 46 may perform content adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), or another entropy coding method. After entropy coding by entropy coding unit 46, the encoded video can be sent to another device or archived for later transmission or retrieval. The coded bitstream includes an entropy coded residual block, a motion vector for such a block, an identifier of an interpolation filter to be applied to a reference frame to calculate a sub-integer pixel value for a particular frame, Other syntaxes including offset values that identify a plurality of different offsets at different integer and sub-integer pixel locations within the encoding unit.

  Inverse quantization unit 42 and inverse transform unit 44 apply inverse quantization and inverse transformation, respectively, to reconstruct the residual block in the pixel domain, eg, for later use as a reference block. Motion compensation unit 35 can calculate the reference block by adding the residual block to one prediction block of the frames of reference frame store 34. Motion compensation unit 35 can also apply the selected interpolation filter 37 to the reconstructed residual block to calculate sub-integer pixel values. Adder 51 adds the reconstructed residual block to the motion compensated prediction block generated by motion compensation unit 35 to generate a reconstructed video block for storage in reference frame store 34. To do. The reconstructed video block may be used by motion estimation unit 36 and motion compensation unit 35 as a reference block for intercoding blocks in subsequent video frames.

  Motion estimation unit 36 may also change the standard search algorithm used to identify the predictive block for which motion vectors are to be calculated according to the techniques of this disclosure. According to the techniques of this disclosure, motion estimation unit 36 may perform an improved motion search that takes into account a DC offset previously added to the reference frame. In particular, motion estimation unit 36 may improve motion search by forcing the evaluation of sub-pixel locations that have non-zero DC offset values as a result of adding a DC offset. During motion search, motion estimation unit 36 attempts to identify the predicted block of the reference frame that best matches the block to be encoded, eg, based on the block's SAD, SSD or other pixel difference metric.

  In order to improve the encoding resulting from motion search, the standard motion search algorithm directs that motion compensation unit 35 skips blocks starting at one of the integer or sub-integer pixel locations that define a non-zero offset value. Even if so, motion estimation unit 36 may examine the prediction block of the reference frame starting at these pixel locations. A block of the reference frame starts at a pixel or sub-pixel location that has no defined offset value or has an offset value of 0, and the motion search algorithm directs the block at that location to be skipped Sometimes motion estimation unit 36 can skip that location and move to the next pixel location. However, when a block of the reference frame starts at a location where motion compensation unit 35 has defined a non-zero offset value and the motion search algorithm directs skipping the block at that location, motion estimation unit 36 Can nevertheless analyze the block starting at that location, thereby disabling the standard motion search algorithm. In particular, motion estimation unit 36 may compare the block starting at that location in the reference frame with the block to be coded to determine whether the block should be selected as a predictive block for intercoding.

  In one example, video encoder 20 calculates a procedure for selecting an interpolation filter based on the (single or cumulative) history of one or more previously encoded frames and a DC offset for the subpixel position. Combining the procedure for assigning with improved motion search to take into account sub-pixel positions with DC offsets, with a significant speed gain and encoding in a single pass with little or no performance loss Can be executed. In another example, video encoder 20 may: (1) a procedure for selecting an interpolation filter; (2) a procedure for calculating and assigning a DC offset for a sub-pixel position; and (3) a sub with a DC offset. Any combination or replacement with motion search improvements to take into account pixel positions can be performed.

  FIG. 3 is a block diagram illustrating an example of a video decoder 30 that decodes a video sequence encoded by the method described in this disclosure. Video decoder 30 may perform a decoding pass that is the reverse of the encoding described with respect to video encoder 20 (FIG. 2) in some examples. Video decoder 30 includes a motion compensation unit 54 that performs the offset technique of the present disclosure during decoding. In particular, on the decoding side, motion compensation unit 54 may receive syntax elements from entropy decoding unit 52, which define different offsets, for example for integer pixel locations and one or more non-integer pixel locations. , Identifying a plurality of offset values of the coding unit.

  The motion compensation unit 54 can generate prediction data based on the motion vector received from the entropy decoding unit 52. In particular, motion compensation unit 54 uses motion vectors to identify prediction blocks in previously decoded reference frames and to provide appropriate offsets for such prediction data (based on the pixel location of the prediction data). In addition, offset prediction data can be generated. The predicted data can be interpolated data, in which case a corresponding one of the offset values of the non-integer locations can be applied to the interpolated predicted data to generate offset predicted data. Based on this offset prediction data, video data (eg, a reconstructed residual video block) can be decoded. In particular, the decoder can combine the offset prediction data with the residual video block to generate the original encoded video block.

  Entropy decoding unit 52 entropy decodes the received bitstream to generate quantized coefficients and syntax (eg, motion vectors and coding unit offset values). The syntax is transferred from entropy coding unit 52 to motion compensation unit 54. The inverse quantization unit 56 performs inverse quantization on the quantized block coefficient, that is, de-quantize. The inverse quantization process is described in, for example, It can be a conventional process defined by H.264 decoding. Inverse transform unit 58 applies an inverse transform, eg, an inverse DCT or a conceptually similar inverse transform process, to the transform coefficients to generate a residual block in the pixel domain. Motion compensation unit 54 generates motion compensation blocks and optionally performs interpolation using interpolation filter 64. An identifier specifying a particular interpolation filter 64 to be used for a subpixel position in a particular frame may also be included as a syntax element in the bitstream generated by the encoder 20.

  Motion compensation unit 54 may select the same set of interpolation filters 64 that were used by video encoder 20 during the encoding of the video block. In particular, motion compensation unit 54 calculates a sub-integer pixel value using a set of interpolation filters 64 indicated by syntax elements in the bitstream for reference blocks retrieved from reference frame store 62. After motion compensation unit 54 generates a prediction block based on the motion vector, motion compensation unit 54 adds the appropriate offset value to the prediction block and uses the offset prediction block used in the original encoding performed by the encoder. Is generated.

  Adder 64 adds the residual data from the residual block by summing the residual blocks with corresponding offset prediction blocks generated by motion compensation unit 54 to produce a decoded block for the current frame. Form. If desired, a deblocking filter can be applied to filter the decoded block to remove blockiness artifacts. The decoded video block is then stored in a reference frame store 62, which provides a reference block for subsequent motion compensation for subsequent frames to be decoded and for video display (device of FIG. 1). It also generates decoded video for transmission to a display buffer that drives a display device (such as 28).

  Again, the techniques of this disclosure relate to applying offsets to motion compensated prediction data and use different offsets for integers defined by interpolation and different sub-integer pixel locations. The encoder uses the techniques of this disclosure to define and apply various offset values. The decoder also interprets syntax elements sent from the encoder to identify the same offset value that was defined and used by the encoder. Applies an appropriate offset to the pixel value of the prediction data, the appropriate offset being a pixel location defined for such prediction data, eg, an integer pixel location, or some possible non-integer pixel location Is selected based on one of

  FIG. 4 is a conceptual diagram illustrating integer pixel locations associated with prediction data and sub-integer pixel locations associated with interpolated prediction data. In the conceptual diagram of FIG. 4, the various boxes represent pixels. Uppercase letters (in the solid box) represent integer pixel locations and lowercase letters (in the dotted box) represent subinteger interpolated pixel locations. Pixel locations “aa”, “bb”, “cc”, “dd”, “ee”, “ff”, “gg”, “hh”, “ii”, and “jj” are assigned to pixel location “C3”. A half-pixel location used in fractional interpolation of various associated fractional locations. Each pixel can correspond to a pixel on the upper right side of the video block such that the pixel defines a video block. For interpolation or extrapolation, each of the pixels of the video block can be similarly interpolated or extrapolated with respect to various integer pixels having the same spatial distance from the respective sub-integer pixels.

  Every integer pixel location has 15 different fractional ("sub-integer") locations associated with it. In the example of FIG. 4, these 15 different fractional locations associated with pixel “C3” are sub-integer pixel locations “a”, “b”, “c”, “d”, “e”, “f”. , “G”, “h”, “i”, “j”, “k”, “l”, “m”, “n”, and “o”. Similarly, the 15 different fractional locations associated with pixel “E5” are the sub-integer pixel locations “a ′”, “b ′”, “c ′”, “d ′”, “e ′”, “f ′”. ”,“ G ′ ”,“ h ′ ”,“ i ′ ”,“ j ′ ”,“ k ′ ”,“ l ′ ”,“ m ′ ”,“ n ′ ”, and“ o ′ ” ing. For simplicity, most of the other fractional locations (other than the fractional location described above used to generate one or more of the 15 different fractional locations associated with pixel “C3”) are illustrated. Not shown.

  For example, ITU H. The H.264 / AVC standard generally uses a 6-tap Wiener filter with coefficients [1, -5, 20, 20, -5, 1] to obtain a luma signal at a half pixel position. A bilinear filter is then used to obtain the luma signal at the quarter pixel location. The bilinear filter is an H.264 filter. It can also be used in fractional pixel interpolation of chroma components, which may have up to 1/8 pixel accuracy in H.264 / AVC.

  After motion estimation, in order to balance coding rate and video quality, for example, a rate distortion model can be used to identify the best motion vector for a given video block. The best motion vector is used to form a predictive video block during motion compensation. As outlined above, a residual video block is formed by subtracting the predicted video block from the original video block. A transform is then applied to the residual block and the transform coefficients are quantized and entropy coded to further reduce the bit rate.

  The techniques of this disclosure include adding an offset to the predicted video block. The value of the offset can be location specific in that different offsets are defined for different integer and sub-integer pixel locations. Since pixels “b” and “b ′” define the same sub-integer pixel location with respect to integer pixels C3 and E5, the offsets associated with the video blocks identified by these two pixels can be the same. . However, since pixels “c” and “d ′” define different sub-integer pixel locations for integer pixels C3 and E5, the offset associated with the video block identified by pixels “c” and “d ′” is Can be different. 16 different pixel locations “C3”, “a”, “b”, “c”, “d”, “e”, “f”, “g”, “h”, “i”, “j”, Each of “k”, “l”, “m”, “n”, and “o” can define a different offset. Further, these different offsets are the 16 different pixel locations “E5”, “a ′”, “b ′”, “c ′”, “d ′”, “e ′”, “f ′”, “g” Apply to each of '"," h' "," i '"," j' "," k '"," l' "," m '"," n' ", and" o '" You can also. The offset can define a signed value that essentially biases every pixel value of the prediction block up or down to produce an offset prediction block.

  The offset may be referred to as a DC offset because it may comprise the average difference of all pixels of the coding unit that have the same sample location (integer or specific sub-integer location) relative to the average of all corresponding predicted pixels. . That is, since each of the pixels of the block is similarly biased to the same extent, the offset results in a change in the DC value of the transformed data resulting from, for example, the DCT calculation, where the DC value is the upper left of the matrix resulting from the transformed calculation. Is the value of This is because the DC value represents the average of the pixel values of the block. Thus, by biasing the pixel by a certain value, the DC value resulting from the conversion calculation will be similarly biased. Thus, in some examples, rather than applying a DC offset to each pixel of the block, a DC offset can be applied to the DC value resulting from the transform calculation.

  In some examples, a different offset value can be assigned to each of the integer and sub-integer pixel locations. An offset value corresponding to the pixel or subpixel referenced by the motion vector is applied to each pixel of the block. As an example, pixel locations "e", "f", "g", "i", "k", "m", "m", " Offset values {1, 6, 4, 5, 7, 3, 8, 2} can be assigned to “n” and “o”, respectively. That is, the offset value mapping is: {C3-> null, a-> null, b-> null, c-> null, d-> null, e-> 1, f-> 6, g-> 4, h- > Null, i-> 5, j-> null, k-> 7, l-> null, m-> 3, n-> 8, o-> 2}. In another example, the offset value may comprise a difference between an average value of pixels in the reference frame and an average value of pixels in the current frame. When such a difference is called DCframe, in one example, when 0 <DCframe <1.5, an offset with a magnitude of 1 is assigned to 10 * DCframe pixel positions (rounded to the nearest integer). It is done. For example, if DCframe is equal to 0.83, an offset with magnitude 1 is assigned to 8 of the 16 pixel locations.

  In accordance with the techniques of this disclosure, motion estimation unit 36 may be instructed by the standard motion search algorithm to skip one or more of the blocks whose locations have defined offset values. These blocks can also be evaluated. Motion estimation unit 36 may skip evaluating those blocks when the motion search algorithm directs that blocks starting at pixel locations that do not have a defined offset value are skipped. Offset values {1, 6, 4, 5, 7, 3 for pixel locations “e”, “f”, “g”, “i”, “k”, “m”, “n”, and “o”, respectively. , 8, 2} in the above example, motion estimation unit 36 is associated with pixel locations “C3”, “a”, “b”, “c”, “d”, “h”, “j”, and When the motion search algorithm directs to skip “l”, it can skip evaluating blocks starting at these pixel locations. However, motion estimation unit 36 skips blocks starting at pixel locations “e”, “f”, “g”, “i”, “k”, “m”, “n”, and “o”. The pixel location “e”, “f”, “g”, “i”, “k”, “m”, “n”, and “o” are defined offset values even when the motion search algorithm directs Will evaluate the blocks starting at these locations.

  As another example, let DCmb represent the difference between the average value of the pixels in the reference block and the average value of the pixels in the current block. Further, the minimum value of DCmb allocated to at least the threshold number of macroblocks is defined as DCmin, and the maximum value of DCmb allocated to at least the threshold number of macroblocks is defined as DCmax. In one example, when DCframe> 1.5, an offset value spanning between DCmin and DCmax is assigned to each of the pixel values.

  Again, FIG. 4 shows integer pixel samples (also called full pixels) in a solid line box with the characters in the above case. For any given integer pixel sample, there can be fifteen sub-pixel locations, shown in FIG. 4 for integer pixel sample “C3” and labeled “a”-“o”. H. In accordance with H.264 / AVC, motion compensation unit 35 may first calculate half pixel positions “b”, “h”, and “j” using a one-dimensional 6-tap Wiener filter. The interpolation filter can be applied first in the horizontal direction and then in the vertical direction, or vice versa. The motion compensation unit 35 can then filter the remaining quarter pixel positions using the bilinear filter and the already calculated half pixel samples.

  The actual filter applied to generate the interpolation data can have a wide variety of implementations. As an example, motion compensation unit 35 may use adaptive interpolation filtering to define the interpolation value. In another example, several sets of interpolation filters can be applied to select the set that yields the best prediction data. In some examples, as described in this disclosure, an interpolation filter can be selected based on historical interpolation data of one or more reference frames. According to the present disclosure, the offset is added after generating some interpolated prediction data from the reference frame to be used in coding, but before motion estimation is performed on the current frame to be coded. .

  As described with respect to FIG. 2, the motion compensation unit 35 may use a switched interpolation filter using an offset (SIFO) scheme. The motion compensation unit 35 can select among a plurality of fixed interpolation filters 37, each of which can be defined, for example, by a plurality of different sets of predefined interpolation filter coefficients. The selection of the interpolation filter can be performed at each coding unit level (eg, frame level or slice level) or at each subpixel location (sample level). Further, according to the present disclosure, a DC offset can also be added after prediction, and the DC offset can also be defined for each possible integer or fractional pixel location, as described in the present disclosure.

  That is, motion compensation unit 35 can use a different set of fixed interpolation filters 37 to define some possible alternative interpolation data. As an example, the motion compensation unit 35 is a standard ITU-T H.264 standard. H.264 filter set, H.264 A set of filters based on H.264 / AVC but with higher accuracy (no intermediate rounding for 1/2 pixel position and no bias rounding for 1/4 pixel position), or a set of customized interpolation filters Can be used. A customized set of interpolation filters can be pre-defined by using a set of training video sequences.

  The filter set that provides the best prediction (ie, smaller prediction error energy) is selected by the motion compensation unit 35, as indicated by the historical interpolation error of the previous reference frame or multiple reference frames on a cumulative basis, It can be applied to generate interpolation data. When using multiple reference frames, different filter sets can be selected for different reference frames. In one example, motion compensation unit 35 applies a standard filter for ½ pixel positions (positions b, h and l), and motion compensation unit 35 is customized for other ¼ pixel positions. A filter set can be applied.

  Once the prediction data is generated or interpolated by the motion compensation unit 35, a DC offset can be applied to the prediction data based on the sample location associated with the interpolated (or non-interpolated) data. In particular, this disclosure provides for the use of different DC offsets for different integer or sub-integer pixel locations. Again, in the exemplary data shown in FIG. 4, this includes 16 different pixel locations “C3”, “a”, “b”, “c”, “d”, “e”, “f”, Each of “g”, “h”, “i”, “j”, “k”, “l”, “m”, “n”, and “o” may define its own different offset Means you can. Thus, for the 16 possible integer and sub-integer locations, there can be 16 different offsets. Further, these different offsets are the 16 different pixel locations “E5”, “a ′”, “b ′”, “c ′”, “d ′”, “e ′”, “f ′”, “g” Apply to each of '"," h' "," i '"," j' "," k '"," l' "," m '"," n' ", and" o '" You can also.

  Adding a DC offset to the prediction data pixels can help capture the effects of illumination changes between different video frames. Illuminance changes can be caused by flashes in the video sequence or by the skies. H. H.264 / AVC uses weighted prediction that may allow an offset to be added to the predicted pixel value. However, H.C. The DC offset defined by H.264 / AVC weighted prediction is only possible on the frame level, i.e. only one offset value for a given frame, regardless of whether the prediction data of the video block is interpolated. Is defined. In other words, at a frame level offset, all pixels in the same frame have the same DC offset.

  In accordance with this disclosure, in some cases, the DC offset can be defined differently for different sample positions associated with interpolated and non-interpolated data. Therefore, the fifteen subpixel positions (“a”, “b”, “c”, “d”, “e”, “f”, “g”, “h”, “i”, “ j ”,“ k ”,“ l ”,“ m ”,“ n ”, and“ o ”), and different DC offsets for integer pixel positions. it can. When using sample-based DC offsets, a total of 16 DC offsets can be coded and transmitted as syntax elements in the video bitstream to the decoder. By applying a sample-based DC offset, motion compensation unit 35 can provide a simple but effective motion segmentation tool.

  As an example, a video frame may include an object that moves as a foreground through the sky that settles as a static background. In this case, the background and foreground can have different degrees of illumination change, and by using a location specific DC offset value defined by the location of the pixel that identifies a given video block, the motion compensation unit 35 may be able to capture different degrees of illumination change in the foreground and background more efficiently than can be achieved without such location-specific DC offset values.

  Further, when the video block of the coding unit uses, for example, multiple different reference frames for bi-directional prediction, it calculates and transmits a different DC offset for each integer and sub-integer location associated with the different reference frames. be able to. In some cases, some reference frames may have a single frame-based DC offset and other reference frames may have several location-specific DC offsets. Some schemes use only a location-specific DC offset for the reference frame that is temporally closest to the current frame being coded, and a single frame-based DC for all of the other reference frames. An offset can be used.

  One bit per frame can be used to indicate whether to use a single frame-based DC offset or several location-specific DC offsets to code the DC offset . If the DC offset has only integer precision, the values of these offsets can be coded using a signed exponential-Golomb code. If the DC offset has fractional precision, the integer offset value is coded using a signed exponent Golomb code, and the non-integer offset value is the residual difference for the integer offset defined by the signed exponent Golomb code. Can be coded using

  On the decoder side, the decoder can simply apply an appropriate DC offset value to the pixels of any generated predicted video block. The DC offset value used by the decoder can be defined in a specific integer or sub-integer pixel location syntax element associated with each predicted video block. For example, a syntax element in the header of a coding unit can include a flag or value for specifying each of a plurality of offset values for that coding unit. The terms “offset” and “DC offset” are used interchangeably in this disclosure. As long as an overall offset is defined for each same pixel location, for example, one offset is defined for integer locations and as many different offsets are defined for each possible sub-pixel location Is called a DC offset.

  According to the techniques of this disclosure, offsets can be applied to integer and sub-integer pixel locations in a single coding pass. In one example, a method for assigning a DC offset value in a single pass may include the following operations.

a. DCDiff [idx], ie, calculate the DC difference between the reference frame “idx” and the current frame b. Set sgn = sign (DCDiff [idx]), but sign () returns +1 or -1 c. If the reference frame has idx> 0, set frmOffset [idx] = min (round (DCDiff [idx]), 1) and return, otherwise
d. Determine numOffsets as min (round (DCDiff [idx]), 8) e. If numOffsets ≧ numOffsetsMin (eg, if numOffsetsMin = 2), then several offsets that are equal to numOffsets and increase in value such as {1, 2, 3, 4, 5, 6, 7, 8} , In order, added to sub-pixel positions {5, 15, 13, 7, 9, 6, 11, 14} f. Otherwise, calculate numOffsets1 as min (round (DCDiff [idx] /0.1), 15) g. If numOffsets1> 0 and numOffsets1 <= thFullPel0, then sub-pixels according to the order {5,15,13,7,9,6,11,14} are offset by several offsets equal to numOffsets1 and the value sgn. Add to position h. Otherwise, if numOffsets1> 0, add 8 offsets with the value sgn to the subpixel position {0, 1, 2, 3, 4, 8, 10, 12}, equal to numOffsets1, and the value is Add some offsets that are sgn to sub-pixel positions according to the order {14,11,6,9,7,13,15,5} In some examples, use the process as outlined above Allows the motion compensation unit 35 to assign offset values to integer and sub-integer pixel locations in a single pass. The motion compensation unit 35 can then apply a specific integer or sub-integer pixel location offset value identified by the motion vector calculated by the motion estimation unit 36 to the prediction block.

  In some examples, motion compensation unit 35 calculates a DC difference between two frames to determine an offset value to assign to a particular sub-integer pixel. The motion compensation unit 35 can collect statistics in the form of a histogram on the DC difference between the current frame block and the reference frame collocated block relative to the current frame. The motion compensation unit 35 may first calculate the square error of the pixel difference between the current frame block and the reference frame collocated block as a first error value. Motion compensation unit 35 also calculates the average DC difference between the two blocks, subtracts the calculated DC difference from the pixel value of the block of the current frame, and then subtracts the average DC difference of the current frame. The square error of the difference between the pixel of the block and the pixel of the collocated block of the reference frame can be compared as a second error value.

  The motion compensation unit 35 determines which of the first error value or the second error value is better, for example, by checking whether the first error value is greater than twice the second error value. Can be judged. Based on this comparison, the motion compensation unit 35 can determine whether the block of the current frame and the collocated block of the reference frame are sufficiently similar. If motion compensation unit 35 determines that the two blocks are sufficiently similar, motion compensation unit 35 may accumulate the error values in a vector that includes a block-based DC difference histogram.

Motion compensation unit 35 may analyze the histogram bins in various ways to calculate values such as DC _min and DC _max values based on a number of samples collected for the histogram. it can. In one example, for a minimum number of samples “noSamples”, the motion compensation unit 35 has a DC _min value equal to −1 times the count of several bins corresponding to negative DCs having more samples than noSamples, and DC _It can be determined that the _max value is equal to a count of several bins corresponding to a positive DC having more samples than noSamples. In another example, for the minimum number of samples “noSamples”, motion compensation unit 35 counts the number of bins whose DC _min value corresponds to a negative DC having more samples than noSamples (the count is DC difference). The motion compensation unit 35 is configured to stop counting after the test fails for the first time, and the motion compensation unit 35 _{Although it} can be determined that the _max value is equal to the count of the number of bins corresponding to a positive DC having more samples than noSamples, motion compensation unit 35 counts after the test fails for the first time. Configured to stop. In another example, for a minimum number of samples “noSamples”, motion compensation unit 35 has a DC _min value equal to the DC corresponding to the leftmost bin containing more samples than noSamples, and the DC _max value is , NoSamples can be determined to be equal to the DC corresponding to the rightmost bin containing more samples. Table 4 below is the same as Table 2 above, but is reprinted below for ease of explanation.

To further illustrate these exemplary methods, assuming noSamples = 20 and the histogram is as shown in Table 4 above, motion compensation unit 35 uses DC _{min in} the first exemplary method. = −2 and DC _max = 2, in the second exemplary method, DC _min = −2 and DC _max = 1, and in the third exemplary method, DC _min = −2 and DC _max = 4 Can be determined.

After calculating DC _min and DC _max , motion compensation unit 35 may assign a DC offset value to the sub-integer pixels. In one example, the motion compensation unit 35 adds offsets O (1) to O (15) to the sub-integer pixels 15, 13, 7, 9, 6, 11, 14, 1, 4, 3, 12, 10, 2, The assignments are in the order of 8 (ie, subpixel indices 15, 13, 7, etc.), but other orders of assignment are possible. In some examples, motion compensation unit 35 may not be able to calculate the total number of possible offset values, in which case motion compensation unit 35 assigns the calculated offsets according to the order presented above. Can do.

Motion compensation unit 35 can use the frame-based DC difference between the current frame and the reference frame to determine threshold noSamples used in the calculation of DC _min and DC _max . Motion compensation unit 35 can also calculate a threshold noSamples value according to the size of the image based on the number of blocks in the image. In one example, motion compensation unit 35 determines that the number of offset values to be assigned is equal to the DC difference between two frames +0.5.

In some examples, motion compensation unit 35, when the calculated number of offset values is greater than the calculated value DC _max is whether to use the number calculated offset value as DC _max, or the calculation of the offset value If the calculated number is smaller than the calculated value DC _min , the assigned DC offset value can be further improved by determining whether the calculated number of offset values should be used as DC _min . In other cases, motion compensation unit 35 may use the first calculated value of DC _max and / or DC _min .

The motion compensation unit 35 has a value of “roundFact” having a minimum value of 1 and equaling the difference between the value of DC _{max and} the value of DC _min divided by the number of sub-integer pixel positions, eg, 15. And the difference is rounded to the next highest integer value. The motion compensation unit 35 can further improve the value of DC _max and / or DC _min by calculating:

The following pseudo code represents an exemplary method for improving the above-described DC _max and DC _min , where blockDC also uses one of the exemplary methods described above to use DC _min and DC _{min. The} pseudo code that calculates the initial value of _max and follows blockDC improves the values of DC _min and DC _max .

  FIG. 5 is a conceptual diagram illustrating integer pixel locations with offset values assigned according to the exemplary method described with respect to FIG. In particular, in the example of FIG. 5, it is assumed that the difference between the reference frame and the current frame, round (DCDiff [idx]), is 6 rounded to an integer value. Accordingly, motion compensation unit 35 assigns offset values 1-6 to pixel locations {5, 15, 13, 7, 9, 6}, respectively. Thus, in the example of FIG. 5, the pixels “e”, “f”, “g”, “i”, “m”, and “o” have offset values 1, 6, 4, 5, 3, and 2 respectively. Is assigned. According to the techniques of this disclosure, when motion estimation unit 36 performs a motion search, one of pixels “e”, “f”, “g”, “i”, “m”, and “o”. Even if the normal search algorithm would dictate skipping blocks starting in one or more, such pixels have a defined offset value, so pixels “e”, “f”, “g "," I "," m ", and" o "starting blocks can be analyzed.

  FIG. 6 is another conceptual diagram illustrating integer pixel locations with offset values assigned according to the exemplary method described with respect to FIG. In particular, in the example of FIG. 6, it is assumed that the difference between the reference frame and the current frame, ie, DCDiff [idx] is 0.7. Thus, assuming that numOffsetMin is equal to 2 according to the example of FIG. 4, DCDiff [idx] is less than numOffsetMin when rounded to an integer. Dividing DCDiff [idx] by 0.1 gives 7 and thus numOffset1 is equal to 7. According to the exemplary method of FIG. 4, several offsets equal to numOffsets1 and of magnitude sgn are added to the pixel location in the order {5,15,13,7,9,6,11,14}. Is done. Here, sgn is equal to sign (DCDiff [idx]), and is 1 in this example. Accordingly, the motion compensation unit 35 has a value of 1 for pixels “e”, “f”, “g”, “i”, “k”, “m”, and “o”, as shown in the example of FIG. Assign an offset. According to the techniques of this disclosure, when motion estimation unit 36 performs a motion search, pixel positions “e”, “f”, “g”, “i”, “k”, “m”, and “o” Pixels “e”, “f”, “g”, “i”, “k” even if the normal motion search algorithm would direct skipping blocks starting at one or more of ”,“ M ”, and“ o ”starting blocks can be analyzed.

  FIG. 7 is another conceptual diagram illustrating integer pixel locations with offset values assigned according to the exemplary method described with respect to FIG. In particular, in the example of FIG. 7, it is assumed that the difference between the reference frame and the current frame, ie, DCDiff [idx] is 1.3. Therefore, assuming that numOffsetMin is equal to 2 according to the example of FIG. 4, DCDiff [idx] is smaller than numOffsetMin when rounded to an integer. Dividing DCDiff [idx] by 0.1 gives 13 and thus numOffset1 is equal to 13. According to the exemplary method of FIG. 4, eight offsets of size sgn are added to the pixel location in the order {0, 1, 2, 3, 4, 8, 10, 12}. Here, sgn is equal to sign (DCDiff [idx]), which is 1 in this example, and equal to numOffsets 1-8 (5 in this example), and some offsets of size sgn are pixel locations. {14, 11, 6, 9, 7, 13, 15, 5}. Accordingly, as shown in the example of FIG. 6, the motion compensation unit 35 includes pixels “C3”, “a”, “b”, “c”, “d”, “f”, “g”, “h”, Assign an offset of value 1 to “i”, “j”, “k”, “l”, and “n”. According to the techniques of this disclosure, when motion estimation is performed, the pixels “C3”, “a”, “b”, “c”, “d”, “f”, “g”, “ When a normal motion search algorithm specifies to skip a block starting at one or more of “h”, “i”, “j”, “k”, “l”, and “n” However, the pixels “C3”, “a”, “b”, “c”, “d”, “f”, “g”, “h”, “i”, “j”, “k”, “l” , And a block starting at “n”.

  FIG. 8 is a flowchart illustrating an exemplary operation of video encoder 20 for encoding video data in a single pass using a switched interpolation filter and offset. The example method of FIG. 8 illustrates one example method for selecting an interpolation filter based on a history of previously encoded frames. Although described with respect to video encoder 20 (FIG. 2) for purposes of explanation, it should be understood that other video encoders may employ the method described with respect to FIG. Initially, video encoder 20 receives digital video from a video source, such as video source 18 (FIG. 1) (80). The digital video can comprise newly recorded video when the video source 18 comprises a video capture device, or in other cases can comprise pre-recorded uncoded digital video. Digital video generally comprises a sequence of frames, each frame comprising a plurality of slices and macroblocks. The video encoder 20 encodes the frames in the video sequence as I frames, P frames, or B frames. Within the P and B frames, some macroblocks may be encoded as I macroblocks.

  Video encoder 20 encodes the first frame in the picture group (GOP) as an intra-coded I frame (82). Then, to reconstruct the reference frame, after the encoded I frame is decoded, eg, by inverse transform unit 44, video encoder 20 may use interpolation filter 37 to calculate the value of the sub-integer pixels of the I frame. A default set of is selected (84). Motion compensation unit 35 receives the reconstructed version of the I frame and uses the selected set of interpolation filters 37 to calculate the values of the subpixels of the I frame (86). For the first intercoding unit, the selected set of filters may be the default set of filters. In particular, for each sub-pixel of the I frame, motion compensation unit 35 applies the corresponding default interpolation filter to calculate the value of that sub-pixel. In some examples, the default interpolation filter for one subpixel may be different from the default interpolation filter for another subpixel in the same I frame. The designer may specify one of those interpolation filters as the default for each of the subpixel positions. The video encoder 20 may select one of the interpolation filters 37 that historically gave the lowest error at the sub-pixel location as the default interpolation filter, or the default interpolation filter may be selected in other ways. May be. In any case, motion compensation unit 35 calculates the value of the subpixel of the I frame using the default interpolation filter of interpolation filter 37 and stores the I frame in reference frame store 34.

  Motion compensation unit 35 also calculates an offset value for one or more of the reference frame pixels and sub-pixel locations (88). In some examples, the offset value may represent an average pixel value difference between the reference frame and the next frame to be encoded, eg, a B frame or a P frame. Motion compensation unit 35 may calculate and assign an offset value for a pixel in the reference frame using any of the methods described with respect to FIG. 4 or elsewhere in this disclosure. After calculating the offset value, the motion compensation unit 35 changes the pixel value of the reference frame according to the offset value.

  Video encoder 20 then encodes the next frame, eg, a P frame or a B frame, according to the reference frame (90). When the next frame to be encoded is a P frame, video encoder 20 may encode that frame based on the most recent I or P frame in reference frame store 34. When the frame is a B frame, video encoder 20 may encode the frame based on one or more frames in reference frame store 34. Based on one or more frames of the reference frame store 34, motion estimation unit 36 calculates a motion vector for the coding unit, eg, block, of that frame. The motion vector for that block points to the corresponding prediction block in the reference frame. A motion vector may refer to an integer or sub-integer pixel location within a reference frame. For sub-integer precision motion vectors, as described above, the reference frame is interpolated to generate values at the sub-pixel positions (86).

During motion search to calculate motion vectors, motion estimation unit 36 may skip evaluation of some integer or sub-integer pixels according to a motion search algorithm. However, the motion estimation unit 36, according to the techniques of this disclosure, is an integer and sub-integer to which the motion compensation unit 35 has applied the offset value, regardless of whether the motion search algorithm specifies that the pixel be skipped. It can be configured to evaluate blocks starting from pixels. This is the opposite of some conventional motion estimation units that will skip these positions according to a motion search algorithm. In one example, even when a conventional motion search algorithm directs that motion compensation unit 35 skips any or all of the sub-integer pixel positions that have been assigned an offset value, motion estimation unit 36 Can be configured to explicitly inspect each of the positions. For example, if the motion compensation unit 35 assigns the following offset values in Table 5 to the following sub-integer pixel positions, Table 5 is the same as Table 1 above, but is repeated below for ease of explanation.

In that case, each time one of the “virtual” offsets {−2, −1,0,1,2,3} is added to the reference block, the SAD calculation of the two blocks is calculated six times. Can do. The motion vector with the smallest SAD can be selected. An offset value that appears in more than one location (eg, offset “2” that appears in both subpixel location 1 and subpixel location 11) can only be attempted once. That is, the motion estimation unit can search only one of a plurality of subpixels each having the same calculated offset value.

  Motion compensation unit 35 calculates a residual value for each block based on the difference between the current block in the frame being encoded and the predicted block in the reference frame. Video encoder 20 encodes the motion vectors, residuals, and applicable interpolation filter identifiers, and offset values using, for example, DCT, quantization and entropy coding, and decoders this data into a coded bitstream. Send to.

  The motion compensation unit 35 also determines the error value of the frame encoded using the reference frame with sub-pixels calculated using the selected interpolation filter applied for motion estimation ( 92). Motion compensation unit 35 also determines an error value for each of the other potential interpolation filters that may be used for each sub-pixel location for motion estimation (94). In this way, motion compensation unit 35 can recalculate the error value based on the various remaining interpolation filters. For example, the motion compensation unit 35 can calculate a cost matrix C [F] [I]. For each (f, i), where i refers to the interpolation filter corresponding to the sub-pixel position i and f corresponds to the sub-pixel position i, the motion compensation unit 35 may, for example, as described with respect to FIG. For each subpixel position i having a value calculated using, compute the cumulative error on the frame.

  Motion compensation unit 35 then determines which set of interpolation filters yielded the lowest error value (96). For example, motion compensation unit 35 may iterate over each sub-pixel location and each untrial filter to determine a location that reduces the total frame error by a maximum amount and its interpolation filter. Motion compensation unit 35 then recalculates the total error of the frame and corresponds to a position that reduces the total error of the frame until the maximum number of iterations is reached or until the total error is reduced below a threshold amount. The interpolation filter can continue to be identified. An example of this method is described with respect to FIG.

  Motion compensation unit 35 then determines whether the current frame is the last frame in a video sequence, eg, a picture group (GOP), a frame group, or a fully recorded video sequence (98). If not (98 “NO” branch), motion compensation unit 35 applies the selected set of interpolation filters to the previously encoded frame to calculate a sub-pixel value and the subsequent frame for that frame. If the subsequent frame is a bidirectional encoded frame, it potentially encodes another reference frame. This process is used to determine that the interpolation filter produces the lowest error of the previous coding unit or series of coding units, and so as to support motion estimation of the current coding unit, Can continue repeatedly. In this way, the selection of an interpolation filter to support motion estimation for the current coding unit is based on historical interpolation results for one or more previously coded units. As explained above, the interpolation filter may be selected based on a cumulative interpolation result that gives a higher weight to the result for the more recent coding unit.

  FIG. 9 is a flowchart illustrating an exemplary method for identifying an optimal set of interpolation filters based on historical interpolation results for previously coded frames to calculate sub-pixel interpolation values for the current frame. It is. Initially, motion compensation unit 35 receives a history decision vector D having an element index ranging from 0 to n-1 for n sub-integer pixel positions (110). Each element of the decision vector D corresponds to an interpolation filter that is used to calculate the value of the corresponding subpixel.

Motion compensation unit 35 then calculates a cost matrix C [F] [I], where each i in I corresponds to a sub-integer pixel position, and each f in F is a sub-integer pixel i. Corresponding to the interpolation filter used to calculate the value of, where C [f] [i] is each sub-integer pixel in the current frame with the value calculated using the interpolation filter f It is equal to the sum of errors at position i. Thus, in the following equation, error (x, i) refers to the error value associated with the i th sub-integer pixel for pixel x in the frame for the encoded frame, where each frame has M pixels. Have.

  Motion compensation unit 35 calculates an error value for each interpolation filter at each sub-integer pixel location. In one example, motion compensation unit 35 further includes the cost of the previous frame in the cost matrix of the current frame to perform an interpolation filter selection based on, for example, cumulative interpolation error results for multiple historical frames, not just the reference frame. Matrix fractional values can be added. After calculating the cost matrix, motion compensation unit 35 calculates the error value of the historical frame (s) by adding the values corresponding to the subpixels calculated by the interpolation filter of the previous frame ( 114). That is, motion compensation unit 35 for each i in I, each C [f] [i] where i corresponds to the subpixel position and f corresponds to the interpolation filter used to calculate the value of i. ] Can be accumulated. In one example, this value corresponds to SAD.

  Next, the motion compensation unit 35 determines the position i that most reduces the SAD value and the interpolation filter f of i (116). For example, motion compensation unit 35 iterates over each value in the cost matrix for a combination of subpixels and interpolation filters that were not used to calculate the subpixels of the reference frame, and these values are Can be compared with the corresponding values for the combination of sub-pixel and interpolation filter used. The largest difference between the combination of unused subpixel positions and interpolation filters, the corresponding subpixel position and the interpolation filter used is this combination. When this difference is greater than the minimum threshold (“YES” branch of 118), motion compensation unit 35 has newly determined the used interpolation filter for that subpixel position in the decision vector at that position. Replace with interpolation filter (120) and use this new combination to recalculate error (eg, SAD) (122). The motion compensation unit 35 does this until the maximum number of iterations is reached (124 “YES” branch) or until the difference described above is less than a predetermined threshold (118 “NO” branch). The process can be repeated. The motion compensation unit 35 then outputs the updated decision vector (126). The updated decision vector can then be used to select a set of interpolation filters for the current unit to be coded.

  The following pseudo code provides an example for implementing the DC offset technique of this disclosure. In that pseudo code, “blockDC1” is a procedure for calculating the DC offset values DCmin and DCmax for the 16 × 16 pixel block identified by “input” based on the reference frame identified by “list”. is there. “NoSamples” is an integer value that defines the number of samples for the procedure that are available to calculate DCmin and DCmax. “List” is an integer value that identifies a reference frame (referred to as “list” according to ITU H.264 terminology). In general, BlockDC1 and BlockDC2 are functions that estimate the difference between the visibility of the reference frame and the visibility of the frame being encoded (or the current frame). The estimated offset can be applied to the reference frame and can be used to improve motion compensated prediction.

  One approach would be to consider the average visibility of the reference frame and compare it to the average visibility of the current frame. This solution has the disadvantage that the average visibility does not represent local variations. It is conceivable that two frames having extremely different average visibility become almost the same macroblock after motion compensation. This is the case, for example, when a bright object is not obstructed. The second possibility uses motion vectors to estimate the DC difference between motion compensation blocks. While this method is accurate, it may have the disadvantage that motion vectors must be available, so that the current frame must be encoded multiple times (at least one for estimating motion vectors). And one time to apply the DC offset to the prediction, an operation that is known to affect motion information that needs to be recalculated to take into account the newly calculated offset). Implied.

  The block-based DC difference calculation can overcome the above-described problems and the like without using a motion vector. The calculation of block-based DC differences may be based on collecting statistics on DC differences between collocated blocks of some size (eg, 16 × 16 pixels). The statistics can be collected in the form of a histogram. In the pseudo code presented below, BlockDC1 and BlockDC2 represent techniques for setting DCmin and DCmax values to define a range of offset values. Compare the block in the reference frame with the block in the current frame and calculate the following two quantities for each block of pixels:

1) Square error of pixel difference (err0 in the function BlockDC1 or BlockDC2)
2) The square error of the pixel difference after the average DC of the block is subtracted to the value of each pixel (err1 in the function BlockDC1 or BlockDC2)
An estimate of whether the blocks are sufficiently similar (ie, whether the differences between collocated blocks are mostly caused by DC differences) may be based on a comparison of the above quantities 1) and 2). For example, if the error (err0) in 1) is greater than twice the error (err1) in 2), the blocks are not sufficiently similar and the difference is largely not caused by a DC difference It can be judged. If the collocated blocks are sufficiently similar, their statistics can be accumulated into a vector containing a block-based DC difference histogram. In some cases, the DC differences can be quantized by rounding those DC differences to integer values.

  By examining the histogram bins, it is possible to derive a range of DC differences and calculate DCmin and DCmax. BlockDC1 calculates DCmin and DCmax values by determining the DC values corresponding to the leftmost and rightmost histogram bins that contain more than noSamples samples. BlockDC2 calculates the DCmin and DCmax values by counting the number of bins corresponding to positive and negative DC values having more than noSamples samples. Another approach, as a variant of BlockDC2, counts the number of bins corresponding to negative and positive DC values with more samples than noSamples, but stops counting when the test fails for the first time It can happen. In the following pseudocode, setSubpelOffset1 and setSubpelOffset2 are the sub-integer pixel offset values (eg, DCmin, DCmax, and / or DCmin) of the frame according to the calculations performed during the execution of the corresponding function of BlockDC1 and BlockDC2. A value between DCmax).

The “BlockDC1” procedure is an example of a procedure for setting DCmin and DCmax values for the 16 × 16 pixel block identified by “input”, as shown by the following pseudo code.

“SetSubpelOffset1” is a procedure for setting the DC offset value of the block identified by “SubpelOffset [16]” in the range between DCmin and DCmax. “List” is an integer value for identifying a reference frame. “ImgOffset” identifies the frame offset used on the reference frame having an index greater than zero. In this example, setSubpelOffset1 calls blockDC, thereby calling the procedure of blockDC1 presented above.

The “BlockDC2” procedure is an example of another procedure for setting the DCmin and DCmax values for the 16 × 16 pixel block identified by “input”, as shown by the following pseudo code. DCmin and DCmax are pointers to the values calculated by BlockDC2, and another procedure may use it as a range to set the block's DC value between DCmin and DCmax. “NoSamples” is an integer value that defines the number of samples for the procedure that are available to calculate the DC offset value.

“SetSubpelOffset2” is a procedure for setting an offset value of a sub-integer pixel position of a block identified by “input” in a range between DCmin and DCmax. In this example, setSubsetOffset1 calls blockDC, thereby calling either the blockDC1 or blockDC2 procedure presented above.

  The techniques of this disclosure may be implemented in a wide variety of devices or apparatuses, including a wireless handset, an integrated circuit (IC) or a set of ICs (ie, a chip set). Although any given component, module or unit has been described to emphasize functional aspects, implementation with different hardware units is not necessarily required.

  The techniques described in this disclosure may be implemented, at least in part, in hardware, software, firmware, or any combination thereof. For example, various aspects of the described techniques include one or more processors including one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs). , Or other equivalent integrated or discrete logic circuits, as well as any combination of such components. The terms “processor” or “processing circuit” may generally refer to any of the above logic circuits, alone or in combination with other logic circuits, or other equivalent circuits.

  Such hardware, software, and firmware can be implemented in the same device or in separate devices to support the various operations and functions described in this disclosure. Further, any of the units, modules or components described can be implemented together or separately as separate but interoperable logic devices. Diagrams of various functions as modules or units highlight various functional aspects and necessarily imply that such modules or units must be implemented by separate hardware or software components. Do not mean. Rather, functionality associated with one or more modules or units may be performed by separate hardware or software components, or incorporated within common or separate hardware or software components.

  Also, the techniques described herein may be implemented or encoded on a computer readable medium, such as a computer readable storage medium that includes instructions. An instruction embedded or encoded in a computer readable medium may cause a programmable processor or other processor to perform the method, for example, when the instruction is executed. Computer readable storage media include random access memory (RAM), read only memory (ROM), programmable read only memory (PROM), erasable programmable read only memory (EPROM), electrically erasable programmable read only memory (EEPROM), It may include flash memory, hard disk, CD-ROM, floppy disk, cassette, magnetic media, optical media, or other computer readable media.

Various examples have been described. These and other examples are within the scope of the following claims.
The invention described in the scope of the claims at the beginning of the present application is added below.
[1] interpolating sub-integer pixels of a reference video unit using a selected interpolation filter before performing motion estimation of the current video unit;
Applying an offset to at least some of the sub-integer pixels of the reference video unit before performing motion estimation of the current video unit;
Encoding a block of the current video unit using motion estimation based on the reference video unit;
A video encoding method comprising:
[2] The method of [1], wherein encoding the block comprises performing motion estimation of the block only once such that encoding is performed in a single pass.
[3] The method of [1], further comprising selecting the set of interpolation filters based on historical interpolation results for one or more previously encoded video units.
[4] selecting the interpolation filter;
Determining a first interpolation error value of a first set of interpolation filters used to encode a previously encoded video unit;
Determining a second interpolation error value of a second set of interpolation filters;
Comparing the first error value and the second error value;
Selecting the interpolation filter based on the comparison;
The method according to [3], comprising:
[5] selecting the interpolation filter;
Accumulating interpolation error values of different interpolation filters used to encode a plurality of previously encoded video units;
Selecting the interpolation filter based on the accumulated interpolation error value;
The method according to [3], comprising:
[6] The method according to [3], wherein the interpolation filter corresponds to each sub-integer pixel, and the interpolation filter includes different interpolation filters for at least some of the sub-integer pixels.
[7] The method of [3], further comprising encoding a syntax element indicating the selected interpolation filter and the offset of the encoded block.
[8] encoding the video block;
Performing motion estimation to identify a predictive block in the reference video unit relative to the block in the current video unit;
Determining a motion vector that identifies the prediction block in the reference video unit;
Determining a residual error between the block in the current video unit and the prediction block in the reference video unit;
The method according to [1], comprising:
[9] Performing motion estimation
Executing a motion search algorithm specifying that the evaluation of blocks associated with sub-integer pixels to which an offset is applied is skipped;
Forcing the evaluation of the block relative to the sub-integer pixels to which the offset is applied;
The method according to [8], comprising:
[10] The method of [1], wherein each of the video units comprises one of a video frame or a video slice.
[11] interpolating sub-integer pixels of the reference video unit using the selected interpolation filter before performing motion estimation of the current video unit;
Applying an offset to at least some of the sub-integer pixels of the reference video unit before performing motion estimation of the current video unit;
Encoding a block of the current video unit using motion estimation based on the reference video unit;
A video encoding device comprising a video encoder configured to perform:
[12] The apparatus of [11], wherein the video encoder is configured to perform motion estimation of the block only once such that encoding is performed in a single pass.
[13] The apparatus of [11], wherein the video encoder is configured to select the set of interpolation filters based on historical interpolation results for one or more previously encoded video units. .
[14] The video encoder
Determining a first interpolation error value of a first set of interpolation filters used to encode a previously encoded video unit;
Determining a second interpolation error value of a second set of interpolation filters;
Comparing the first error value and the second error value;
Selecting the interpolation filter based on the comparison;
The apparatus according to [13], which is configured to perform:
[15] The video encoder is
Accumulating interpolation error values of different interpolation filters used to encode a plurality of previously encoded video units;
Selecting the interpolation filter based on the accumulated interpolation error value;
The apparatus according to [13], which is configured to perform:
[16] The apparatus of [13], wherein the interpolation filter corresponds to a respective sub-integer pixel, and the interpolation filter includes a different interpolation filter for at least some of the sub-integer pixels.
[17] The apparatus of [13], wherein the video encoder is configured to encode a syntax element indicating the selected interpolation filter and the offset of the encoded block.
[18] The video encoder is
Performing motion estimation to identify a predictive block in the reference video unit relative to the block in the current video unit;
Determining a motion vector that identifies the prediction block in the reference video unit;
Determining a residual error between the block in the current video unit and the prediction block in the reference video unit;
The apparatus according to [11], configured to perform:
[19] The video encoder
Executing a motion search algorithm specifying that the evaluation of blocks associated with sub-integer pixels to which an offset is applied is skipped;
Forcing the evaluation of the block relative to the sub-integer pixels to which the offset is applied;
The apparatus of [18], configured to perform:
[20] The apparatus of [11], wherein each of the video units comprises one of a video frame or a video slice.
[21] The apparatus of [11], wherein the video encoder forms part of a wireless communication device.
[22] The apparatus according to [11], wherein the video encoder includes an integrated circuit device.
[23] means for interpolating sub-integer pixels of the reference video unit using the selected interpolation filter before performing motion estimation of the current video unit;
Means for applying an offset to at least some of the sub-integer pixels of the reference video unit before performing motion estimation of the current video unit;
Means for encoding a block of the current video unit using motion estimation based on the reference video unit;
A video encoding device comprising:
[24] The apparatus of [23], wherein the means for encoding the block comprises means for performing motion estimation of the block only once such that encoding is performed in a single pass. .
[25] The apparatus of [23], further comprising means for selecting the set of interpolation filters based on historical interpolation results for one or more previously encoded video units.
[26] The means for selecting the interpolation filter comprises:
Means for determining a first interpolation error value of a first set of interpolation filters used to encode a previously encoded video unit;
Means for determining a second interpolation error value of the second set of interpolation filters;
Means for comparing the first error value and the second error value;
Means for selecting the interpolation filter based on the comparison;
The apparatus according to [25], comprising:
[27] The means for selecting the interpolation filter comprises:
Means for accumulating interpolation error values of different interpolation filters used to encode a plurality of previously encoded video units;
Means for selecting the interpolation filter based on the accumulated interpolation error value;
The apparatus according to [25], comprising:
[28] The apparatus of [25], wherein the interpolation filter corresponds to a respective sub-integer pixel, and the interpolation filter includes a different interpolation filter for at least some of the sub-integer pixels.
[29] The apparatus of [25], further comprising means for encoding a syntax element indicating the selected interpolation filter and the offset of the encoded block.
[30] The means for encoding the video block comprises:
Means for performing motion estimation to identify a predictive block in the reference video unit relative to the block in the current video unit;
Means for determining a motion vector identifying the prediction block in the reference video unit;
Means for determining a residual error between the block in the current video unit and the prediction block in the reference video unit;
The apparatus according to [23], comprising:
[31] The means for performing motion estimation comprises:
Means for executing a motion search algorithm specifying to skip evaluation of blocks associated with sub-integer pixels to which an offset is applied;
Means for forcing an evaluation of the block associated with sub-integer pixels to which an offset has been applied;
The apparatus according to [30], comprising:
[32] The apparatus of [23], wherein each of the video units comprises one of a video frame or a video slice.
[33] interpolating sub-integer pixels of a reference video unit using a selected interpolation filter before performing motion estimation of the current video unit;
Applying an offset to at least some of the sub-integer pixels of the reference video unit before performing motion estimation of the current video unit;
Encoding a block of the current video unit using motion estimation based on the reference video unit;
A computer-readable storage medium encoded with instructions for causing a processor to perform.
[34] The storage medium of [33], wherein encoding the block comprises performing motion estimation of the block only once such that encoding is performed in a single pass.
[35] The apparatus according to [33], further comprising instructions for causing the processor to select the set of interpolation filters based on historical interpolation results for one or more previously encoded video units. The storage medium described.
[36] determining a first interpolation error value of a first set of interpolation filters used to encode a previously encoded video unit;
Determining a second interpolation error value of a second set of interpolation filters;
Comparing the first error value and the second error value;
Selecting the interpolation filter based on the comparison;
The storage medium according to [35], further comprising an instruction for causing the processor to perform the following.
[37] accumulating interpolation error values of different interpolation filters used to encode a plurality of previously encoded video units;
Selecting the interpolation filter based on the accumulated interpolation error value;
The storage medium according to [35], further comprising an instruction for causing the processor to perform the following.
[38] The storage medium of [35], wherein the interpolation filter corresponds to a respective sub-integer pixel, and the interpolation filter includes a different interpolation filter for at least some of the sub-integer pixels.
[39] The storage of [35], further comprising instructions for causing the processor to encode a syntax element indicating the selected interpolation filter and the offset of the encoded block. Medium.
[40] performing motion estimation to identify a predictive block in the reference video unit relative to the block in the current video unit;
Determining a motion vector that identifies the prediction block in the reference video unit;
Determining a residual error between the block in the current video unit and the prediction block in the reference video unit;
The storage medium according to [35], further comprising an instruction for causing the processor to perform the following.
[41] performing a motion search algorithm that specifies to skip evaluation of blocks associated with sub-integer pixels to which an offset has been applied;
Forcing the evaluation of the block relative to the sub-integer pixels to which the offset is applied;
The storage medium according to [40], further comprising: an instruction for causing the processor to execute.
[42] The storage medium of [33], wherein each of the video units comprises one of a video frame or a video slice.

Claims (16)

Determining a first interpolation error value of a first set of interpolation filters used to encode a previously encoded video unit;
Determining a second interpolation error value of a second set of interpolation filters;
Comparing the first interpolation error value and the second interpolation error value;
Selecting the interpolation filter based on the comparison;
Interpolating the sub-integer pixels of the reference video unit using a selected interpolation filter before performing motion estimation of the current video unit;
Applying an offset to at least some of the sub-integer pixels of the reference video unit before performing motion estimation of the current video unit;
Encoding a block of the current video unit using motion estimation based on the reference video unit;
Encoding the block includes
Calculating one motion vector per block to generate a motion vector based on an interpolation filter and offset selected prior to motion estimation, and motion estimation of the block such that encoding is performed in a single pass A video encoding method comprising: performing the method only once. Encoding the video block;
Performing motion estimation to identify a predictive block in the reference video unit relative to the block in the current video unit;
Determining a motion vector that identifies the prediction block in the reference video unit;
The method of claim 1, comprising determining a residual error between the block in the current video unit and the prediction block in the reference video unit. Performing motion estimation is
Executing a motion search algorithm specifying that the evaluation of blocks associated with sub-integer pixels to which an offset is applied is skipped;
3. Forcing the evaluation of the block associated with sub-integer pixels to which an offset has been applied.   The method of claim 1, wherein each of the video units comprises one of a video frame or a video slice. Determining a first interpolation error value of a first set of interpolation filters used to encode a previously encoded video unit;
Determining a second interpolation error value of a second set of interpolation filters;
Comparing the first interpolation error value and the second interpolation error value;
Selecting the interpolation filter based on the comparison;
Interpolating the sub-integer pixels of the reference video unit using a selected interpolation filter before performing motion estimation of the current video unit;
Applying an offset to at least some of the sub-integer pixels of the reference video unit before performing motion estimation of the current video unit;
A video encoder configured to encode a block of the current video unit using motion estimation based on the reference video unit;
The video encoder configured to perform encoding of the block;
Calculating one motion vector per block to generate a motion vector based on an interpolation filter and offset selected prior to motion estimation, and motion estimation of the block such that encoding is performed in a single pass The video encoding device is configured to perform only once. The video encoder is
Performing motion estimation to identify a predictive block in the reference video unit relative to the block in the current video unit;
Determining a motion vector that identifies the prediction block in the reference video unit;
6. The apparatus of claim 5, wherein the apparatus is configured to determine a residual error between the block in the current video unit and the prediction block in the reference video unit. The video encoder is
Executing a motion search algorithm specifying that the evaluation of blocks associated with sub-integer pixels to which an offset is applied is skipped;
7. The apparatus of claim 6, wherein the apparatus is configured to enforce evaluation of the block associated with sub-integer pixels to which an offset is applied.   The apparatus of claim 5, wherein each of the video units comprises one of a video frame or a video slice.   The apparatus of claim 5, wherein the video encoder forms part of a wireless communication device.   The apparatus of claim 5, wherein the video encoder comprises an integrated circuit device. Means for determining a first interpolation error value of a first set of interpolation filters used to encode a previously encoded video unit;
Means for determining a second interpolation error value of the second set of interpolation filters;
Means for comparing the first interpolation error value and the second interpolation error value;
Means for selecting the interpolation filter based on the comparison;
Means for interpolating the sub-integer pixels of the reference video unit using the selected interpolation filter before performing motion estimation of the current video unit;
Means for applying an offset to at least some of the sub-integer pixels of the reference video unit before performing motion estimation of the current video unit;
Means for encoding a block of the current video unit using motion estimation based on the reference video unit;
The means for encoding the block comprises:
Means for calculating one motion vector per block to generate a motion vector based on an interpolation filter and offset selected prior to motion estimation, and motion estimation of said block so that encoding is performed in a single pass A video encoding device comprising: means for executing only once. The means for encoding the video block comprises:
Means for performing motion estimation to identify a predictive block in the reference video unit relative to the block in the current video unit;
Means for determining a motion vector identifying the prediction block in the reference video unit;
The apparatus of claim 11, comprising means for determining a residual error between the block in the current video unit and the prediction block in the reference video unit. Said means for performing motion estimation comprises:
Means for executing a motion search algorithm specifying to skip evaluation of blocks associated with sub-integer pixels to which an offset is applied;
13. The apparatus of claim 12, comprising: means for forcing an evaluation of the block associated with sub-integer pixels to which an offset has been applied.   The apparatus of claim 11, wherein each of the video units comprises one of a video frame or a video slice. Determining a first interpolation error value of a first set of interpolation filters used to encode a previously encoded video unit;
Determining a second interpolation error value of a second set of interpolation filters;
Comparing the first interpolation error value and the second interpolation error value;
Selecting the interpolation filter based on the comparison;
Interpolating the sub-integer pixels of the reference video unit using a selected interpolation filter before performing motion estimation of the current video unit;
Applying an offset to at least some of the sub-integer pixels of the reference video unit before performing motion estimation of the current video unit;
Encoded with instructions for causing a processor to encode a block of the current video unit using motion estimation based on the reference video unit;
Encoding the block includes
Calculating one motion vector per block to generate a motion vector based on an interpolation filter and offset selected prior to motion estimation, and motion estimation of the block such that encoding is performed in a single pass A computer-readable storage medium comprising: executing the program only once.   The storage medium of claim 15, wherein each of the video units comprises one of a video frame or a video slice.

Images