Qt Contributors Summit 2022 - Program/Color space in Qt - issues and possibilities

From Qt Wiki
Jump to navigation Jump to search


Session Summary

  • What are color spaces
    • Wide gamut color spaces
    • HDR color spaces
    • "Gamma" curves
      • Why is gamma correction difficult, especially with text
  • What can Qt currently do
    • QColorSpace and image transforms
    • Linear color spaces
    • Floating point image formats
  • What can't Qt currently do
    • Gamma correct blending
      • Non fast-path text rendering
      • Real instead of platform faked gamma-correction on text fast path
      • Other gamma corrected text painting without linear color spaces
    • Automatic color correction
      • From QImage inputs
      • To QPA outputs (for instance DP3 macs)
  • What APIs can we add to help
    • QPainter API
    • QScreen/QSurface API
      • QPA integration
    • QFont API?
    • QColorSpace API
      • XYZ,XYy
      • Luminosity
      • RT2100
      • Double table ICC formats

Session Owners

  • Allan Sandfeld Jensen

Notes

(Taken by CristianMaureiraFredes)

The session was based on slides that will be shared afterwards. These are notes on the discussion.

  • Comments & Comments
    • Friedemann: Note Qt was mentioned as one library implementing color spaces correctly at Meeting C++ https://meetingcpp.com/mcpp/online/talkview.php?tid=2
    • Nicolas Fella: For colors on Wayland there is https://gitlab.freedesktop.org/wayland/wayland-protocols/-/merge_requests/14
    • Lars Knoll: There is some stuff available there that Laszlo has been working on. I'm using that for Qt MM.
    • Lars Knoll: But it's preview and doesn't work on all platforms.
    • Andy: There is a huge performance impact for software rendering for doing this. And image formats "lie" about the color space as well. It's hard to just expect it to work given a random image file. In Qt Quick 3D we do all the rendering in linear color space and tonemap back out to sRGB space. But all user input via textures colors needs to be converted based on context. Qt Declarative just mixes everything
    • Max: So we could implement some of that on the RHI level?
      • Sure, but we need to check if that would have an impact on performance
    • Lars: comments on the topic of multimedia

we have to be very careful with QPAinter (rendering on sw) things can become too expensive. What you want to do is to render images, but assume linear blending? Alan: yeah, that's considered. Maybe we need a setting to correct that... maybe a setting tool to correct the images on load? other thing...some work on the output side. Some of the rendered colors, you need built-in tone mapping support. We can't get into some colorspaces, but there are limits with HDR colors on screen. without the mapping is saturated. Alan: ack it's expensive, and we need to do it in hw I think Alan: for videos..yeah, in hw. videos go up in luminosity like 5, 10, or 15. Modern macbook HDR 700 nits, if the display is at max bright, then luminosity of 3 you can render properly. for a low surrounding light setting, maybe a factor of 8. You need to get that factor out of the OS, and tone it to that. Alan: we need also the default value for SDR values (windows) I'm using the same in all platforms... Alan: most screens have 200 nits, maximize makes that blurry... for macos they tell you SDR value, then the range that they support on top of that. So a luminosity on 1 is what the max SDR gives you, then you need to cut it off. (Things will be followed up with the changes in multimedia, to gather more insights)

    • Alexandru Croitor: Are there commercial users asking for better color support? It feels like all improvements are done because the developers feel passionate about it, rather than paying users needing it? Which is surprising to me
      • Andy: I think thats correct. Currently customers just make it look good regardless of what we do. Doing it perfectly everywhere will be painful (esspecially in Widgets and 2D Quick, and both painful for us as well as the users in breakages of existing code + performance penalties).
      • Fabian: Well, at least for font rendering, we had bug reports
      • Andy: Ideally we should do all (2D) rendering in linear color space as well and output to whatever is the ideal color space for the output, but there is a cost.
    • Shawn: sounds a bit like hidpi: let's plan for getting it really proper eventually, make sure it can be hardware-accelerated, and try not to have it take 10 years this time
    • Albert Astals Cid: Related to colorspaces but unrelated to this presentation, in KDE we'd super like if QColorSpace had CMYK support. We're trying to support loading CMYK PSD files and it's not great we have to solved it in our side :D https://invent.kde.org/frameworks/kimageformats/-/merge_requests/72wn Rutledge: sounds a bit like hidpi: let's plan for getting it really proper eventually, make sure it can be hardware-accelerated, and try not to have it take 10 years this time
    • Lars: I think correct colorspace handling is needed for our users at least for photos and videos.


Follow up conversation on chat

  • Lars: UI elements can I believe mostly be handled in linear color space and UX designers will tune the color values until it looks correct.
  • Allan: yes, but it might need to be optional.
  • Allan: though I would also prefer if we could handle srgb colors in linear space.
  • Giuseppe: yeah, we've had N requests in the last couple of years for handling CMYK in QImage / QPainter. "it's complicated"
  • Laszlo: @Allan What I was missing here was a bit more inclusive look at the full stack as things are in Qt 6. QPainter/QImage/QColorSpace is one small piece in the puzzle. Qt Quick has its own story. Then Qt Quick 3D for instance does implement lots of stuff mentioned here (work in linear, tonemap, floating point images without QImage, HDR ready, etc. because it is simply more essential for 3D content). Also there's the HDR work in Multimedia Lars mentioned, built on the experimental Windows/macOS HDR swapchain support in QRhi. In the end the full story is a lot more complicated, esp. with overlapping rendering technologies (e.g. think an image in Qt Quick scene embedded within a Qt Quick 3D scene in a Qt Quick scene in a QQuickWidget composited by the widget stack to a window..), with many challenges along the way. Anyway, there's definitely room for a lot of experimenting in this area.
  • Allan: right, I knew RHI had a new model for the backing store, which is what I hoped would be ready for the qtgui side of things as well
  • Lars: Agree with Laszlo. It's certainly not easy to make this consistent, esp. given that we don't want to break existing uses. Ideally, I believe we should do all our rendering in linear, and tonemap the output to whatever the underlying surface uses.
  • Allan: and part of the wish for the talk was to bridge some of these separate efforts together.
  • Lars: I think the work on an RHI enabled backing store might help here. That should at least make tonemapping possible without completely killing performance.
  • Laszlo: yes one new thing in 6.4 for composited backingstores (if there's a QQuickWidget/QOpenGLWidget in the top-level) is that we use QRhi for that, not OpenGL directly. Which in turn enables some other interesting things using the platforms' best suported APIs (e.g. the high dpi scaling experiments), and can be useful for future color space stuff as well later on.
  • Andy: You shouldn't forget that every color value and texture/image needs to be converted to linear color space (when needed) which also has a cost. And that ignores the fact that it's dificult to know when to do so without the user being explicit about the color space of each image since image formats don't always speceify their color space correctly. Also don't ignore the fact that many exising applications depend on the current "wrong" behavior.
  • Lars: Btw, you can find the stuff I'm using in https://code.qt.io/cgit/qt/qtmultimedia.git/tree/src/multimedia/shaders. hdrtonemapper.glsl is the tonemapper that limits luminosity to the range the screen can render (to avoid hard clipping of values on the screen)
  • Allan: though theoretical HDR format should be using absolute luminosity, but I am happy to ignore that, as not being able to adjust brigthness is not good in my book
  • Morten: Should we add a new experimental "color correct" application mode? Then we would be free to make changes without breaking existing apps (which may have implemented custom color correction on top if Qt)
  • Lars: @andy right now, we know that our default 2D rendering basically assumes the output surface is sRGB with a gamma of ~2 - 2.2, and we do alpha composition linearly in that space. Should be possible to map that so the default rendering looks the same.
  • Andy: @Morten S +1
  • Lars: @allan from all I could see, that is only theory. All TVs and screens to a brightness adjustment.
  • Allan: mine does not :D and I hate it!
  • Lars: otherwise you could never properly watch an HDR movie unless you perfectly control the surrounding light.
  • Lars: mine does :P
  • Allan: yeah I have to draw the curtains to watch some movies..
  • Andy: I like how HDR content on my TV has stuff that is so bright its like looking at the sun :-p
  • Lars: Sure, that's the purpose of HDR... it should blind you :)
  • Andy: lol
  • Lars: but macOS does adjust things according to the surrounding light (or the brightness setting of the display). So a luminosity of 1 is full SDR brightness. And the max luminosity that can be rendered depends on the display brightness (ie. you get a larger range with low display brightness settings)
  • Allan: so how does it do tone mapping of things that ends up too bright? A clamp, or late curve?
  • Lars: If my display is at full brightness I can render luminosities from ~0 - 2.5. If it's at a very low brightness the range goes from ~0 - 16.
  • Lars: @allan https://code.qt.io/cgit/qt/qtmultimedia.git/tree/src/multimedia/shaders/hdrtonemapper.glsl
  • Lars: That's basically doing the mapping of the Y component in linear YUV space.
  • Allan: right, so that is what you based it on
  • Lars: Yes. RHI gives me the maxLum value.
  • Lars: see QRhiSwapChainHdrInfo::maxLuminance.
  • Allan: that makes it output specific though.. For QColorSpace, I would prefer something general, but maybe that should be kept out of it then
  • Lars: (and qvideowindow.cpp:425ff)
  • Lars: If you want to render things on screen you need some tonemapping. Ideally I think the backing store could do that.
  • Lars: Of course you don't want any tone mapping to happen if you want to process images or videos.
  • Allan: right, but if a user requests a conversion from sRGB to RT.2100 on an image, it would need some display independent conversion too
  • Lars: Of course. Simply leave out the tonemapping.
  • Lars: I've got shaders for quite a bit of that as well in Qt MM by now.