[go: up one dir, main page]

Page MenuHomePhabricator

High dynamic range (HDR) videos aren't consistently supported or tone-mapped to SDR
Open, Needs TriagePublic

Description

Videos with high dynamic range (HDR) or wide color gamut (WICG) or otherwise not standard color types are not consistently supported in MediaWiki at this time.

  • thumbnails usually appear gray/desaturated due to failure to convert the colorspace in ffmpeg creating a JPEG thumb
  • transcoded video output may be forced to 8-bit, damaging 10-bit content
  • transcoded video output may still retain the colorspace and hdr mastering markers but browser support is hit or miss and there's no attempt to check for support

Firefox, Chrome, and Safari all have different capabilities with HDR video playback, and may differ depending on operating system, hardware, etc.

Ideally, we should detect HDR and wide-gamut input and convert it to a bt709 SDR output, as well as a bt2020/pq HDR output; ffmpeg's zscale filter can map colors and tonemap and perform clipping or tone mapping.

There are some complications; current versions of ffmpeg don't handle smpte2084 (pq) transfer function in the colorspace filter, hence recommendation to use zscale. Additionally neither filter overrides HDR light level mastering metadata, so some files may require additional manipulation before they display correctly in some playback systems that take the light levels into account.

Example of an HDR video on Commons:

  • Stream #0:0(eng): Video: vp9 (Profile 2), yuv420p10le(tv, unknown/bt2020/smpte2084), 3840x2160, SAR 1:1 DAR 16:9, 30 fps, 30 tbr, 1k tbn (default)

See also T19505 which asks for still image HDR support -- AVIF would be our primary target these days over on that end, and we should produce matching thumbnails if/when needed.

Event Timeline

Is this issue being prioritized at all? As more public domain works are getting released with HDR, this is becoming a pain point.

Is this issue being prioritized at all?

No. The entire area of videosupport runs as a best effort kinda thing, mostly on spare time and/or volunteers.

I have a question about your video file used as demonstration: why is the "colorspace" (yuv2rgb matrix) unknown? Keep in mind that bt709 matrix is slightly different from bt2020nc matrix, and video players may use heuristics to decide to use bt709 to do yuv2rgb.

The MaxCLL MaxFALL and Mastering Display Metadata in metadata (implemented with SEI) are usually filled with boilerplate data in videos I have seen, so there really is no big deal. We can insert some boilerplate data back in the stream if really needed.

ffmpeg notoriously does not take colorspace/transfer into account when extracting JPEG from video stream. It just copies the yuv over and marks it as bt470bg sRGB... ALL major video hosting website's video thumbnails (and keyframe previews on the timeline) have these awful bt709 frames marked as bt601 problem, making the green go oversaturated etc. It really is just a matter of adding a vf to the vf chain.

This video may be helpful: https://thebroadcastknowledge.com/2019/07/24/video-user-generated-hdr-is-still-too-hard/

I spent several months painfully trying to understand ffmpeg and all HDR technologies, even tried to develop a HDR debugger for macOS. I am happy to help. But am wondering how I can get involved in the commons infra development.