Open Source Color Management is broken
Since I am now in the business of photography and image processing (see my travel photography blog here), I thought it was time to finally get proper monitors and calibrate them. I wanted to do this with Open Source tools and use the calibration data for my Linux desktop, so I ordered a ColorHug2 colorimeter, which is Open Hardware compliant and all the tools are FOSS licensed. And from then on everything just went downhill.
The ColorHug2 comes with a small USB storage key which is supposed to contain a Fedora Live Session environment with all the tools to calibrate the display. The created ICC color profiles can then also be used on other devices (even on Windows and MacOS). Except that both my laptop and PC refused to even offer the key as a bootable device (which seems to be a known issue). Well, it uses the DisplayCAL GUI frontend for ArgyllCMS anyways, and both these packages are in the repositories of any modern Linux distribution. I had already used them two years ago to calibrate some devices at work, so I put the USB key aside, installed DisplayCAL on my devices, set up the ColorHug2, and started calibration runs. Only to see ArgyllCMS fail every time time because the USB connection to the ColorHug2 kept failing after a while, which also seems to be a known issue. Grunt. This is absolutely not what I expect when I invest more than 100 € in an Open Hardware device.
The known workaround for the problem seems to be an additional USB reset after the device is released (pointing to a firmware bug in the ColorHug2), which is contained in the very latest ArgyllCMS 2.0.1 beta development soure code. So I downloaded, built and installed it, and the calibration finally ran through. But before the calibration starts, DisplayCAL shows an interactive display adjustment screen to have you prepare the display as good as possible before the actual calibration starts. A work colleague lent me his Pantone huey colorimeter, which also works with ArgyllCMS. My photo editing monitor at home is a BenQ BL2711U, which is sold as a professional 4K CAD Monitor, has 100% sRGB color space coverage and is known to have very good color representation even on factory defaults. So I expected both the huey and the ColorHug2 to see just minor deviations, and the measurements to be consistent.
Which was absolutely not the case. The dialogue shows the distribution between the three color channels, and if the display has the necessary controls (laptops don’t), you correct until all three channels meet in the middle. With the huey, ArgyllCMS detected a very slight green tint with the factory settings, which I corrected by lowering the Green channel from 100 to 99 in the monitor settings. The ColorHug2 measured a non-existing very heavy green tint, and I had to lower the Green channel from 100 to 80 to make the bars meet. At that point the whole display had become a quite heavy purple (the Blue and Red channels being way overblown in comparison to Green), which definitely wasn’t right. So I gave up on the ColorHug2 for the moment and continued with the huey, but there are some indications that this might be another ColorHug2 issue. (I later found out that there is a firmware update to version 2.0.7 for the ColorHug2, which only seems to be available via the
fwupd tool, but not via the
colorhug-tools or from the repository where all the other releases can be found. I haven’t tried this update yet.)
DisplayCAL/ArgyllCMS gave me a good profile using the huey and confirmed that the BL2711U actually exceeds the advertised 100% sRGB coverage. The difference between using the calibration data and not using it was barely noticeable. Now I wanted to have the system and every application use my generated profile. Turns out the XFCE desktop I use simply doesn’t have any color management, like most other “smaller” FOSS desktop environments. You can use a tool like
dispwin to manually write the correction values into the video look-up table (VideoLUT) of the graphics card, but the applications also have to know about the profile so they can account for it, e.g. when converting between color spaces (every browser has to do this). And this is where it gets really complicated: This is Linux, so there are no rules or standards.
There are multiple ways to tell an application about the current profile. There is
colord, a daemon which can create, store and manage calibration profiles and sensors and offers its services via D-Bus. And there is the
_ICC_PROFILE X11 atom, which allows applications to attach profiles to and query profiles from X.Org display outputs. Ideally
colord would do everything, but it runs in the background and doesn’t have access to the running X session, so it needs helpers. GNOME and KDE Plasma have GUI frontends for this, and both
gnome-settings-daemon have plugins for setting the
_ICC_PROFILE X11 atom. Except that they only set it for the primary output. Turns out if you have multiple displays, the profile for the first one is put into the
_ICC_PROFILE X11 atom, but the profile for the second one in the
_ICC_PROFILE_1 X11 atom, for the third display in the
_ICC_PROFILE_2 X11 atom, and so on. It’s just that nobody seems to do this (KDE bug here, GNOME bug here).
That means your profiles might actually be written into the VideoLUTs of the respective outputs, but every application which relies on the
_ICC_PROFILE X11 atoms will only be able to get the profile for the primary display. darktable pretty much seems to be the only clever one out there, because it supports both
colord and also supplies the excellent
darktable-cmstest tool which can be used to check which setting is currently set to which value.
But it gets even worse. GNOME (and Ubuntu Unity, which borrows most of its tools) at least has color management built into the desktop environment by default. KDE Plasma offers
kde-colord, but it’s an optional package, and for example in Ubuntu 18.04 it is four years old, two releases behind and doesn’t work because it seems to still assume we live in a KDE 4 world. If you use KDE Plasma on Ubuntu today, you can’t even have color management. All in all this is probably why many developers, if they even care about color management, just give up and let the user set the profile manually. This at least seems to be true for GIMP, PhotoFlow and RawTherapee. Firefox by default only does color management for tagged images. Chrome/Chromium, Gwenview and digiKam seem to use
_ICC_PROFILE, but probably only for the first display as well.
So these are my current suggestions for color management on Linux:
- Make sure you really need and want it. As I’ve written the factory settings of a good monitor can be quite good already.
- Make sure the applications you need color management for know what color management is. Most don’t.
- Only use one display, or make sure you can live with having color management on just the primary one.
- Use GNOME Shell or Ubuntu Unity with X.Org. I don’t think Wayland even has a specification on how to do color management.
- If you really want KDE Plasma, make sure
colord-kdeworks. It is broken on Ubuntu 18.04, but works fine on Arch.
- Find someone who lends you a working, professional-grade calibration device supported by ArgyllCMS (it supports many) before you spend more than 100 € on something just because it has an Open Hardware sticker.
9 thoughts on “Open Source Color Management is broken”
Pingback: New top story on Hacker News: Open Source Color Management Is Broken – Tech + Hckr News
I am also in the quest of proper color management for linux. I use XFCE4 and openbox.
It looks like just debian/ubuntu is stuck with kde4-colord:
cmake.lists of colord-kde from git shows:
A good source regarding the ICC Linux Jungle is
arch resource also states:
“Before using a particular ICC loader, you should understand that some tools set only the calibration curves (e.g. xcalib), other tools set only the display profile to X.org _ICC_PROFILE atom (e.g. xicc) and other tools do both tasks at once (e.g. dispwin, dispcalGUI-apply-profiles).”
DisplayCal also installs an Autostart item for dispcal-apply-profiles, so thats what I use.
Questions still unanswered:
– which one harmonizes with multiple monitors?
What I have found so far the most promising, is using Compiz with the compICC plugin, this will color-manage the whole desktop – and covers all Displays.
Then there xiccd, which does the same as gnome-settings-daemon or colord-kde, but is desktop-independent…
Xicc states to Enumerates displays and register them in colord; So maybe you could tell it to do so with more than just one display..?
Another great find is this repository, which is actually a dev repo from the color.org people
(which designed the ICC profiles in first place.)
maybe you find something useful there for your distro (compicc builds are also there), as opensuse build services support a lot of different ones…
thanks for pointing me to the darktable-cmstest tool! very nice one.
Thanks for all the info! Today I’ve had the time to update/open bug reports on all sides (see the other comment), so I’m hoping that both KDE and GNOME fix the multi-monitor issues. I’m reluctant to invest any work in Compiz since there has been no stable release for more than two and a half years and none of the desktop environments I would use (GNOME, XFCE, KDE) plays nice with Compiz.
xiccd sounded like a good solution, but it always segfaults on my machine, even if I did a manual checkout from GitHub and built it manually :/
Pingback: Links 24/2/2018: Npm Bug, Mycroft AI on Plasma | Techrights
Hi. Great write up! Did you report any of your findings as bugs?
I finally managed to pull together all the info today.
– The ColorHug2 USB drive not booting was reported here: https://groups.google.com/forum/#!topic/colorhug-users/gJZ2UbQTmc4
– ArgyllCMS failing to read from the ColorHug2 was reported and “solved” here: https://groups.google.com/forum/#!topic/colorhug-users/Dssu1xKTYvM
– The ColorHug2 putting too much red into everything was reported here: https://groups.google.com/forum/#!topic/colorhug-users/PFycMgsKCr0
– Ubuntu shipping an outdated and broken package even in Bionic Beaver was already reported at https://bugs.launchpad.net/ubuntu/+source/colord-kde/+bug/1706383 , I’ve added more info.
– colord-kde not assigning X11 profile atoms in multi-monitor setups was already reported at https://bugs.kde.org/show_bug.cgi?id=328248 many years ago. I tried to port the proposed patch to the current code base, but there was a mixup with the _ICC_PROFILE atoms being set in the wrong order, and I don’t know enough about XRandR and friends to fix it.
– gnome-color-manager assigning X11 profile atoms in multi-monitor setups didn’t seem to have been reported before, so I reported https://bugzilla.gnome.org/show_bug.cgi?id=794486 .
I’ve also updated the post with the new links.
Pingback: Simon Raffeiner: Open Source Color Management is broken | Linux Hub
I had the “X atom” problem when using colord to install the profiles. However, if I use Dispcal (which uses dispwin) to install, it was able to properly set the X atom for my two displays, the laptop and the external LCD. The darktable-cmstest binary is great.
great article. so are you guys saying that color management with kde and arch (manjaro) works with multiple displays?
today I tried to configure color management on the command line with colormgr. DE is Mate. partial success: colord seems to use the right profiles. but not x atom. however, firefox and chrome both use x atom, so wrong profile.
should I really try manjaro with kde? as far as I remember, kde always got a wrong profile form somewhere, the “default” profile just could not be deleted.