When Good Profiles Turn Bad

By Ctein

Last week I wrote about a problem I was having getting good prints of my Apollo/Soyuz in Floodlights photograph. You can read that column here; the condensed version is that I couldn't get the flood light beams to print out with smooth and even gradients unless I turned off color management and handed over the rendering to the Epson 3880 printer driver.

Several readers suggested that this was a problem with out-of-gamut colors, an idea I rejected because the gamut preview in Photoshop told me that everything in that photograph was well within my printer's color range.

Well, I'm un-rejecting it. And therein lies a curious and important tale.

Mike Chaney, of QImage fame, kindly agreed to look at the file and see if he could figure out what was going wrong. He observed that when he previewed the gamut for a different printer that he happened to have, he did see out-of-gamut colors, something like the preview above. Their locations aligned pretty well with the harsh breaks that I was seeing in my print's rendering. I brought Dave Polaschek (the printing engineer on Photoshop) in on the discussion, and the three of us batted it around for a bit, for, you see, I still wasn't seeing any evidence of out-of-gamut colors in my previews.

I still don't know for certain that there are out-of-gamut colors in that photograph for my printer. There's no test that I or Dave or Mike can run to determine that for certain. But it looks like a duck. And it quacks like a duck. And there are webbed footprints in just the right (or wrong) places. So we think it's a duck.

This is important because it means I can't entirely count on gamut previews to tell me if all my colors are properly printable. Neither can you. Most of the time it works pretty well. For this photograph, it failed very badly.

Does it squish?
Why doesn't gamut preview work like it's supposed to? It lies in the problem of rendering out-of-gamut colors in a pleasing way on a printer. Color films and decent digital cameras can record colors that no printer can print. Furthermore, an RGB color space like your display uses isn't a one-for-one match with the CMYK color space a printer works in. There are colors your monitor can display that can't be printed. What do you do about those colors? Well, it's that problem of squeezing a pint into a 12-ounce can, again. The only way everything is going to fit is if something squishes. Colors in your photograph that fall outside the ones the printer can print get mapped into colors that it can print.

Without getting technical about it, the software that creates the profile that color management uses has to make some assumptions about how that mapping occurs. For the sake of computation, it also has to treat some out-of-gamut colors as if they were in gamut so that it can deal with them at all. Consequently, there is not really a clear and accurate boundary in the profile or in the gamut preview between colors that are in gamut and colors that aren't. The boundary is fuzzy.

The printer, though, doesn't think so. If it gets thrown a color to print that's outside its comfort range, it will print something, but it may not do it very well. ICC profiles don't understand a particular printer's idiosyncrasies very well. The printer driver is more likely to be written to take into account said printer's foibles.

Most of the time this kind of situation won't jump up and bite you. It bit me good in this photograph. I can think of a few others I have that I've yet to print successfully where I think the same thing may be going on. I'm going to try printing them with color management off, letting the printer control color, and see if it works better.

If profiles can cause problems like this and give misleading results, why not just avoid them all the time? Because 99+% of the time, profiles won't cause problems like this, and they will render in-gamut colors much more faithfully than printer-managed color will. Printer-managed color will not produce superior color most of the time.

As I wrote previously, if you consistently get better results from printer-controlled color then full color management, something's wrong with your system. Either with your profiles, your software, or how you're using them. I can't help you fix that, I'm afraid. All I can do is give you the heads up. On occasion, though, printer-controlled color can save you from a nasty boundary condition failure.

Ctein

Ctein's regular weekly column appears on TOP on Wednesdays.

Comments

[Michael Farrell]

I can't help you fix that, I'm afraid.

So who can?

Serious question. I've just spent 15 minutes Googling around looking for hands-on inkjet printing courses, both locally (for me, Vancouver) and globally. The results weren't obvious. I'm looking for something more comprehensive than a 2-day workshop.

Not knocking workshops, but apparently the process of mastery takes far longer than two days. Even with practice, practice, practice after learning the basics, one is left with oneself as the judge of whether mastery is being achieved. Be nice to find actual courses of some length and density with a certified instructor somewhere.

Any ideas?

[David Bostedo]

"ICC profiles don't understand a particular printer's idiosyncrasies very well. The printer driver is more likely to be written to take into account said printer's foibles."

"Printer-managed color will not produce superior color most of the time."

If I didn't know anything about digital printing, and I read the first statement, I'm not sure I'd believe the second statement. I suppose it's because there are different idiosyncrasies that each handle.

[Tom Judd]

I don't recall that rendering intent was mentioned. Relative colorimetric would 'squish' out-of-gamut colors only, while perceptual would compress the entire gamut so might be better in this case.

[Mike]

This may be a silly question but how, if at all, would this apply to printing black and white?

[Robert Roaldi]

I have a more general question. Will printing issues EVER be resolved in such a way that we'll be able to press a button, or whatever, and reliably get something that is more or less right?

Is this an inherent problem or is that the makers of the equipment cannot (will not) agree on a standard?

Is it a technology maturation issue, because I, for one, really detest being at the bleeding edge. Give me an appliance that works or give me a printer in town who can make it work, but it seems to me sometimes that we don't have either one. Why is that?

[Benjamin Marks]

This whole issue and the related discussion reminds me that chemical/wet photography was always (for me), more like cooking, when compared to inkjet printing. In that older world, I knew what carbonate did, or benzotriazole (not how they worked at a molecular level, but what their effects were) and could add or subtract for the desired effect. Interpolating 01010110 for a printer is a whole other level of magic/abstraction and when something goes wrong, fixing things an understanding why the fix works really pushes you into another realm.

Ben Marks

[Mike Johnston]

"Will printing issues EVER be resolved in such a way that we'll be able to press a button, or whatever, and reliably get something that is more or less right?"

We've got that now. The problem lies in your phrase "more or less." What Ctein is talking about is getting it really right as opposed to more or less right.

"Is this an inherent problem or is that the makers of the equipment cannot (will not) agree on a standard?"

It's an inherent problem, in my opinion. Color management is technical field that can only be simplified so far. About the only way it could give you what you're asking for is if a manufacturer were to make and sell a completely closed "suite" of products--including monitor, camera, RAW converter, printer, printer software, inks, and paper. Then you'd stand a decent chance of getting plug'n'play. But once you add in the consumer's desire to pick his own camera, use a third-party RAW converter, pick from a variety of non-manufacturer printers, and choose his own paper, then you just have to deal with the complications yourself. I don't think there's any way around it.

Mike

P.S. Anyone in publishing has struggled with color management more or less continuously. Of course, there's an awful lot of "more or less" in publishing.

[MBS]

Thanks. This may explain some problems I've had. However, I can't entirely follow this and all of my problems are, unfortunately, in the paragraph that begins, "Without getting technical about it…."

Why would the software creating the profile have to treat "some out-of-gamut colors as if they were in gamut"? My understanding is that a profile is doing a couple of things, one is accounting for the eccentricities of the device (to print a pure blue, we have to add 4 points of green to the number we send to the printer). The other is mapping colors to in-gamut colors for the paper and printer given the rendering intent specified.

So given a highly saturated and out-of-gamut yellow with a relative rendering intend, the software would pick a less saturated, in-gamut yellow that it thought was closest to the original, and adjust the color numbers so the printer would print the less saturated yellow correctly. Is this not true? Does the profiling software assume the printer is still doing the mapping, or some mapping? That is, is the profile adjusting one out-of-gamut color to another out-of-gamut color that it thinks the printer will then map to the correct color choice for the rendering intent? Seems unlikely. In fact, I think I've gone off the mark, here. Clearly, I don't see why it treats an out-of-gamut color as if it were in gamut. That certainly would likely create a profile that would fail at certain color values.

I also don't understand the fuzzy boundary. (I understand a fuzziness that results from the imprecision, both of mechanical devices and an inability to calculate out to infinity, but this sounds like something more.) Probably though, I have to understand the above to understand this.

Thanks again,
Michael

[Eric Chan]

Note that different printer profile-building software (and software options) can lead to very different results, especially when using the Perceptual rendering intent. This is true even when feeding the exact same profile target measurement data to the various software.

The reason is that the mapping of out-of-gamut colors (and nearly out-of-gamut colors) to in-gamut colors is part art, part science. There's no single correct way to do it. Each method has tradeoffs. It's rather like making a flat map of the earth: there are many ways to do it (called "projections" -- e.g., Mercator projection), with very different results. The "best" choice of map projection depends on what the map is being used for.

[ctein]

Dear David,

Indeed, if you didn't know anything about digital printing… and you read both of those phrases WITHOUT reading the 1500 words of surrounding material I've written in the past 2 weeks that give them context… they would appear to be contradictory.

As I regularly remind people, I don't write just to hear myself talk (well, not ENTIRELY [s]). If people choose to lift material out of context, I can pretty much guarantee they will come away misinformed.

~~~~~~

Dear Tom,

I mentioned rendering intent only in passing in the 1st column. The problem shows up regardless of rendering intent. It's more an effect of the way the printer behavior goes extremely nonlinear when you're printing very near maximum densities than a specific way the out of gamut colors get squished.

The squishing is somewhat more complicated than what you describe. Stuff within the gamut needs to be squished to make room for the out of gamut colors to come within printing range in both intents. Very hard to explain the precise differences in less than a few thousand words with pictures. (Maybe Mike Chaney wants to give a stab at it.) A weak and vague description: relative colorimetric tries to preserve the relative accuracy of colors within the gamut while still squishing them in a bit to make room for the out of gamut colors. Perceptual tries to produce a smoother look where there is less heavy squishing right at the boundary.

Please don't take that to the bank.

~~~~~~

Dear Mike,

Not a clue. Good question. I've thought about it a little, in particular how printing “normally” with a custom profile might compare to using the printer driver's Advanced B&W printing mode, but I've never gotten around to doing the comparison (it's harder than just making two prints, because the different modes have different “characteristic curves”). Maybe one of these days…

~~~~~~

Dear Robert,

I already told you how to do that, in both articles. Ignore the color management settings, just let the OS and the printer manage color. Ignore all the settings that relate to color control and allow the software to use the factory defaults. You will get decent color. The technology is more than sufficiently mature for that.

If you want exceptional color, it gets harder. It may always be. Master-Printer-in-a-Box hasn't even reached version 0.1 yet.

pax \ Ctein
[ Please excuse any word-salad. MacSpeech in training! ]
======================================
-- Ctein's Online Gallery http://ctein.com 
-- Digital Restorations http://photo-repair.com 
======================================

[Stephen Best]

Photoshop's Gamut Warning is, at best, indicative. It doesn't take much experience to realize that it's inaccurate. There's also a bug, still in CS5 I believe, in that it picks up the wrong monitor profile in multi-monitor setups when OpenGL is enabled. I suspect Adobe would very much like to excise Gamut Warning (same as they did for No Color Management). Despite its potential usefulness if something can't be made to work properly in all cases, it's better not being there.

While overkill for gamut mapping issues, Chromix's ColorThink Pro can help visualize what's going on. But in practice, Photoshop's soft-proofing facilities and a good monitor are adequate and will allow selection between output profiles/rendering and show if further action is required. It all comes down to learning the limitations of the tools, how to work with them.

Personally I don't think wider understanding of colour management is helped by imprecise definitions of what's going on. It's complicated enough without resorting to voodoo.

[ctein]

Dear Eric,

What you say is absolutely true. Each profiling package has its own idiosyncrasies. In fact, if I print out a color gradient test target with a profile, I can usually figure out which software package created it by looking at the “wrinkles” in the rendering. They are like fingerprints.

What put me on the path of trying my vexatious photo without color management was that profiles produced 3 different ways gave me more or less the same results.

Mike C. thinks he can generate a custom profile that will finesse its way around this problem. If he does, I'm going to give it a try out of curiosity. It will probably involve compromises elsewhere in the color space.

~~~~~~

Dear MBS,

Sorry, I can't help you. I don't know how to explain this without resorting to complicated algebra and topology.

It's 'cause I don't know enough. There is a level of knowledge at which one can do something. Then there's a higher level of knowledge at which one can explain it in technical terms to somebody else. Then there's a higher still level of knowledge where one can successfully explain the same thing in lay terms. (Like some of my physics columns… I hope).

I'm only at level II when it comes to color management. If Dave or Mike C. want to take up the challenge, more power to them. Me, I am not worthy.

pax \ Ctein
[ Please excuse any word-salad. MacSpeech in training! ]
======================================
-- Ctein's Online Gallery http://ctein.com 
-- Digital Restorations http://photo-repair.com 
======================================

[Jan]

Isn't it just Rayleigh scattering?

Well, let's assume that the beams of light shown in the photograph were photographed at around 90 degrees. Then the light taken should be due to Rayleigh scattering, mostly. This pushes the spectrum on the blue side to the UV. Especially for artificial light sources, this should produce a very weird distribution of colors on any gamut.

[Royce Howland]

I wonder if this situation could be debugged a bit further by working through some analysis similar to what Norman Koren describes here:
http://www.gamutvision.com/docs/smudged_pines.html

Using Koren's Gamutvision tool, I took a look at the generic profile for Ilford Galerie Gold Fibre Silk (as I read it, Ctein's paper for larger printers), specifically the version for my Epson 4880 printer. There are some things about this profile that look odd to me. For example the B&W density response visualization shows a noticeable flattening of L* in the bottom few tones which could reduce contrast down there. The same visualization also shows some very sharp discontinuities in the a*, b* and c* curves, again in the bottom tones where L* < 6 or so. This is true for both perceptual and relative colorimetric rendering intents, though RC is marginally less skewed.

In comparison, the generic 4880 profile for one of my favorite Harman papers shows nothing similar to these effects.

Now, Ctein certainly isn't using the generic Ilford profile for a 4880. But potentially there is something funky going on in the bottom tones / greatest densities with this paper, even with custom profiles. Gamutvision may be able to show it, if so. Besides the density response curves display, there are many other profile visualizations. Plus there's a useful false-color soft-proofing display, more comprehensive than what soft-proofing in Photoshop shows...

[David Bostedo]

Hi Ctein - I didn't mean to imply that I was actually thinking that those statements ARE contradictory. I just thought it was interesting that they sound that way, and a comment on the odd state of affairs here.

I've certainly read just about everything you've ever written for this site and I do understand digital printing to a decent extent. Sorry to seem like one of those folks who asks a question that's already answered in the text!

[Mark Roberts]

Congratulations on cracking that nut! I'm a little disappointed that the problem was less mysterious and dramatic than had been speculated. But when you hear hoofbeats it's usually more likely to be horses than zebras, as they say.

For those really interested in this color management stuff, "Real World Color Management" by Fraser, Murphy and Bunting is still the book I recommend first.

[Bob Rosinsky]

Inkjet printing involves art, science, and a bit of luck. Sometimes, when I run into a problem like Ctein's, I'll do a tweak in LAB color space to settle out of gamut issues. Funny thing, my monitor is a top-of-the-line Eizo sRGB monitor. Viewing a LAB color space file on an sRGB monitors require the operator to look at the numbers, look at the screen for color vector trends, and cross a couple of fingers for good measure. The LAB technique is powerful, especially in critical color matching for fine art repro.

[Michael Bearman]

Sorry Ctein late to the discussion. I had a print on a R2400 that did exactly the same thing - I got as far as figuring it was an out of gamut problem - but decided I didn't like the pic enough to waste more paper on it- then ran one last one using Epson's standard settings and voila! I'll see if I can find it and post a link.

[Ed Hawco]

*Sigh* The more I learn about color the more I like black & white.

[AW]

Finally an answer to why my excellent pictures of cats and garden flowers look so bad when I print them(I already ruled out problems with the camera since i use a Canon 5d mk2). I print directly from iPhoto and get the prints in the mail a couple of days later and they always look terrible! I'll email Apple about this issue right away.
Thank you!

Best regards
//cameraguy81

[Gijs Langelaan]

Ctein,

Is the problem perhaps related to the color gamut of the profiling device itself? In particular, with the darker tones (as in your current problem) is sensor noise an issue? Can the profilometers reliably distinguish the darker colors? As a related thought, if the internal light source of the print profilometer is strong enough to reliably measure the dark shades, is it then so strong that some light returns from underneath the paper, so that you would get a different result if you placed your print on (say) a red table cloth instead of a blue one?

I would like to echo Ctein and Mike's comments about the various questions surrounding "plug and play" printing to get good colors. As noted, this already exists, but if you're looking for excellence, it will NEVER exist. It's like asking the camera to always take the right picture even if the photographer pushes the button wrong! (N.B. not just the wrong button, but using the correct button in the wrong way)

regards

Gijs

[Hans Muus]

[quote] I've thought about it a little, in particular how printing “normally” with a custom profile might compare to using the printer driver's Advanced B&W printing mode, but I've never gotten around to doing the comparison (it's harder than just making two prints, because the different modes have different “characteristic curves”). Maybe one of these days…[unquote]

Ctein, one thing I know (pointed out to me by Martin Evening in a personal email) is that in Epson's own Advanced B&W Photo printing mode, almost no colour is used. It is practically only carbon, coming from the Black, Light Black and LL Black cartridges. This not only makes for easy fine tuning/tweaking of the 'color mode' (yes, I am aware of the contradiction here), but also for an extremely stable print. Photoshop would use much more colour pigments to arrive at the same greys (again according to ME, whose source was a specialist at Epson's).

[Dave Polaschek]

Royce,

You've hit on exactly what I think is the problem.

When I was working on printers back in the day, one of the toughest nuts for us to crack was the way the response curve of the ink got highly non-linear near the ends of the curve. It's almost the inverse of the typical film response curve, getting steeper at the ends until it slams into either paper white or ink running off the print and suddenly makes a right-angle-turn and goes flat.

The typical ICC profile has a bunch of curve points, and the CMM interpolates between those points. That's all well and good when you're in the smooth part of the response curve for each ink, but when the curve changes slope abruptly (as it does when you hit the ends), where your points are relative to that abrupt change in the curve becomes critically important. Pick the wrong points, and the (usually linear) interpolation will miss the abrupt change, and you start getting problems.

The software inside the printer (or driver) knows better the shape of the response curve for the printer, sometimes even knowing about that specific printer in printers that monitor themselves, and it can shape the curve to play tricks to extend the apparent gamut a little outside the actual gamut of the printer. It also can speak to the printer in its native color-space of CCYMMMKKKRGBASDF or whatever inks the printer has, rather than being forced to express the colors in an RGB space (all inkjet drivers I'm aware of today take RGB input).

Having seen the software inside the various pieces involved, it doesn't surprise me at all that the printer can sometimes do a better job than any application software does. If anything, I'm more surprised by how frequently application-managed color can "beat" the printer's internal color-matching which has something of an unfair advantage.

tl;dr: This stuff is hard, and the physical world is messy.

[Mark Gunion]

I can't help but wonder whether something like Colorbyte's ImagePrint 8 RIP ( www.colorbytesoftware.com ) would be beneficial here. Since Imageprint 8 drives the printheads directly, Photoshop is not involved in the color management of the print. By report ( www.luminous-landscape.com/reviews/software/imageprint8 ; www.luminous-landscape.com/essays/in_search_of_the_ultimate_inkjet_print ), paper profiles are first rate and perhaps better than custom. Reading between the lines of said reports, I get the impression that the overall "finesse" of ink handling is superior to that possible through Photoshop, which seems to be the need of the particular Ctein print in question (to be very nontechnical about it). Colorbyte provides a 30 day moneyback guarantee in case you are not satisfied with the product. Not an inexpensive solution, but perhaps a more consistent one than switching back and forth between Photoshop's driver and the printer's driver, hoping one will fix whatever problems are perceived in the other. (Disclaimer: I have no connection to Colorbyte or it's retailers, although I do admit to occasionally thinking that I should give Imageprint 8 a trial myself.)

[ctein]

Dear Mark,

I tried ImagePrint out on this photograph, and it did a much better job than printing via Photoshop color management, but it was still not as good as the plain old Epson printer driver.

That's not to badmouth ImagePrint. I have a friend who owns it and it's an excellent program. It is, unfortunately quite pricey, as you commented. More than half the cost of the printer itself.

pax / Ctein

[Marvin]

color management/combination is one thing I like to do but find it hard to do hehehe! :(