[jbills]

Hi all - thought I would put this out to the gurus out there. I'm a compositor by trade but having to do some matte painting work for set extensions, and I'm just wondering how you all deal with working with overbright (>1.0) values in source plates within photoshop.

My plates are EXR from cam raw images the values regularly spike up over 1, be it for cloud highlights, street lamps, whatever. But as we all know, Photoshop is essentially crippled while working in 32 bit mode. Ya can't do much of anything.

But the second you drop to 16 bit, you clamp your highlights at 1, which is taboo in the comp world. never clip your highlights!

So for my situation and sanity, I came up with a workflow where I compress the superwhites back into the 0-1 range, work on them in photoshop 16 bit, and then expand them back out.

the long winded version of this is - it's as simple as running the curve tool in Nuke on the sequence or source image, and finding the brightest pixel. Let's say it comes back as 1.214410, for example. I then do a white point remap (grade node) where I set 1.21441 to white (1). This has the effect of darkening the image.

I render out that compressed (for lack of a better term) image as 16 bit and bring it into photoshop as such. Work with it with an adjustment layer over all my layers that "undoes" the darkening, with an inverted levels that goes from 0-.823445. (you know, inverting the darken is just 1/x). that expands things back out to normal for painting.

So then the matte painting is saved out 16 bit with the adjustment layer turned off, and brought back into nuke where I apply the white point of .823445 with a grade node again to expand everything back out. superwhites saved!

It's a little bit of a hack but seems to be the path of least resistance. I'm waiting for the day when I need something that's not even 16 bit capable in photoshop and I have to drop down to (gasp) 8 bit and then copy/paste it back into the 16 bit comp. The pixel nerd in me is going to be grinding my teeth when that day comes.

I just came up with this on my own and it seems to be about the only logical way of doing it, kind of a hack of what After Effects does with it's HDR Compander effect. Haven't hit up any matte painter friends to see if this is basically what you all do. Is it? or am I missing a better way of doing it?

thanks in advance
-j

[metalrahul]

I am a compositor by trade too. Anyways, have read something on Steve Wright's book on this issue and he suggested to lower your Display Gamma while doing Log to Linear conversation. For eg: 1.7 to 1.0 and then compensate this by adjusting Gamma of your workstation monitor.

[Alex Jenyon]

Matte painters (at least for film) paint in log space for exactly this reason. Log colour space is basically a hack that allows you to save detail in both the blacks and the overbrights, when you only have a 0-1 range to work with. You have to give up something in return - log space heavily compresses your midtones - but it works.

Your workaround is basically doing the same thing, just using an arbitrary whitepoint. Try painting on a log image, and see if you get any better results.

Being a pixel nerd is great - but 8-bit stuff ends up on cinema screens ALL the time, and no-one notices.

[jbills]

yes, that's a good idea. I suppose any time you drop from float to 16 you should try to keep all that detail where it counts, and log is the way to do it.

should probably even intelligently set up the log conversion to max out the usable colors in our plates (could probably remap the black and white points to juice a bit more than the standard 95-685 would give us).

I guess on the photoshop side I'll just need to get some sort of log/lin lookup or colorspace going. i think our matte painters will eventually be working with some stock images and that sort of thing that will originate in sRGB alongside this log stuff. should be possible to keep it all under the same log roof.

will research it, thanks for the tip.

[metalrahul]

I am just resurrecting this thread again, so Jbills, have you managed to get more information on the workflow? I end up having more than 80% of my references pic coming from 8 bit srgb though my workspace is 16 bit (I know it doesn't make it any better since my sources has only 8 bit color information).

I think I am going to see some shots screened for review soon and I am wondering whether to expect any clamping since I was not painting it in Log space?

[jbills]

well, if your source imagery doesn't ever dip into the superwhite 1.0< region (or superblacks below 0 for that matter) then you shouldn't be introducing any clipping that wasn't already in the sources.

see, my problem was that my source images are camera raws, which capture data above and below 0. not as much latitude as a film scan, but I've been getting regular top end values up around 1.3 and regular bottoms around -.05, so those are definitely worth saving.

i ended up not setting up a log workflow since I'm only saving just a little bit of data. it's not worth it. maybe if I had more the type of information in a dpx scan, but just for .3 and -0.05 above and below, log is too much trouble as it's a pain to work with and none of photoshop's tools are geared for it.

So we simply compress the above and below float information and remap to 0-1. then, in photoshop (16 bit) I'll add an adjustment layer and remap the white point to .7, for example, to expand these values back out. Matte painter works with that turned on but exports without it and we invert the compression in nuke. pretty simple.

in your case, with 8 bit sources it doesn't really matter unless you have some extended range stuff in that last 20% of your sources that you didn't mention.

it is, however, good to still be working in 16bit, because every operation you do, be it a blur or color correct, will take advantage of the extended headroom.

you would think float (32bit) would be even better but in float, photoshop is essentially crippled. in fact, many operations are still 8 bit! So you'll find yourself making a copy of your file, switching to 8 bit, doing that effect that is only 8 bit, and then copying and pasting that layer back into your 16bit master file. what a pain, hopefully adobe will catch up to the rest of the world and prioritize this soon.

[metalrahul]

Thanks for the reply jbills, it is really good to have discussion on this issue as there isn't much info anywhere else on web. Btw what you think about Steve Wright's option of adjusting display gamma of a calibrated monitor?

[jbills]

Originally Posted by metalrahul

Thanks for the reply jbills, it is really good to have discussion on this issue as there isn't much info anywhere else on web. Btw what you think about Steve Wright's option of adjusting display gamma of a calibrated monitor?

I assume you mean to paint on log images?

sure, that's about the only way you can and still remain in 16 bit.

that's what we have to do to paint on film images in 16bit, and it might even be a valid workflow for working on CG renders or other images that have a similar amount of data as a film scan. I don't claim to be quite as big of a pixel nerd as Wright and some of those guys - :) - but I think the general consensus off the top of my head is that there's about 13.5 stops of theoretical data able to be stored in a 10bit log film scan, with a good 4 or so (usually) being above the mapped white point - and a half float EXR can hold what most agree is a similar amount of light referenced data with the handy advantage of storing the data as float values. So when your data amount kicks up into that range with a fair amount outside of 0-1, yes, better paint in log and keep as much data where it matters most to the human eye.

but also be prepared for some whacky behavior from most of the tools in photoshop that are expecting linear images and not log. or in the case of some of the legacy plugins it's actually expecting sRGB, but regardless things get weird when you try to apply something simple like a levels or a brightness. the sliders get touchy and unpredictable. Clone tool still works, though!

There is a filter out there that fnord put out I think that mimics the after effects "cineon convertor" effect, and I think when Adobe finally gets it together and updates it's dinosauric filters to be fully 32 bit compatible, then things get much much easier. You'll just pull in any source and work truly linear float like we've grown to love in Nuke and other programs, and there will be not workaround or compromises. And you can convert a dpx into linear with the fnord plugin and be rocking your way to a fantastic float image.

but the sad fact is adobe doesn't really prioritize our little industry - we're just a blip on their map. Even though joe photographer should really see the advantage of working in 32 bit as well...

I think it's safe to say most matte painters have been working this way for a while and will continue to.

Maybe Mari or one of the up and coming texture paint apps will get enough brushes and effects built up to knock photoshop off the top spot. I know painting directly on geo has to appeal to more than a few of the more technical matte painters. If the Foundry could beat photoshop to the punch and catch the crowd before adobe fully gets on the 32bit bus, I'm sure you'd have quite a few people willing to take the pepsi challenge and possibly make the switch! I'm just a comper but I'd certainly not have any loyalty to old man photoshop if something made my occasional matte painting life that much easier!

[metalrahul]

Actually I was referring to paint on 16 bit linear image. From what I understand from the book, he was suggesting to decrease the display gamma of the log image from 1.7 to 1 during LOG to LIN conversation and compensate the image darkness by changing gamma of matte painting workstation monitor. This might disrupts normal look of GUI but the image should work fine as the gamma correction applied to the file is not clipping any values. Once the paint work is done we can inverse the same settings for LIN to LOG conversation.

Yeah, it would be great if Adobe take care of this issue...

[jbills]

oh, yeah... right, yeah I hear ya - good times, let me tell ya. so, think of how washed out a cineon image is, now think about inverting that so you can view it normal! ha - makes staring at the rest of your GUI quite a challenge.

but yes, totally valid. we used to use MonLut at weta I think it was called. a little kodak logo, must've been from the Cineon days. was good for gamma slamming, too.

Whatever gets the job done. I have a feeling we're going to look back on this period as the "stone ages" of matte painting. we might as well be cutting out pieces of film with an x-acto knife and rubber cement. :)