XSIBlog

Each release, we have a list of major projects we focus on. There is also some planned time for developers to do maintenance, and to enhance their areas, even if it isn’t necessarily part of the bigger picture. I think this constant progress in all areas is very important to the product. There is a moment when each member of the team looks for “low hanging fruits”, these little things we can do make the product better by staying a few extra hours in the evening or in the weekend.

There are three main new items in the Fx Tree for version 6.0.

The first one is the integration of the warp kernel from Elastic Reality, as a new “morpher” and “warper” transform effect.

The second one is a new image compare feature in the Fx Viewer which is more like the one Shake. The existing image compare function is still there, but it is more useful to compare changes while editing an image, while the new one is more useful when comparing two full screen image sequences.

The third one is basic OpenEXR 16-bit float native support and requires a bit of explanation.

First, the XSI FX Tree is Windows port of Avid Media Illusion, which had very limited floating point support. It supported the 32-bit float images, but only to gain extra precision and not to process color values that are outside of the 0.0 and 1.0 range. That sounds silly nowadays, but most Media Illusion users at the time (1996) would be working in 16-bit, with cineon files, and wouldn’t immediately use this, so there would have been time to gradually add support for HDR when it became relevant. Illusion also had video support, which definitely did not support HDR imaging, and that uses 8-bit YUV storage in order to be real-time. In other words, floating point images wasn’t a feature that was in use, and therefore it wasn’t really implemented when the development of the product ended.

One additional problem with 32-bit float images is that they are slower to process, and take four times as much RAM. The RAM part is an especially critical issue. Floating point is so tempting to use, there is no doubt that if it were enabled in the Fx Tree people would use it and then run into huge memory allocation issues. It’s not that much of a problem in Shake because it is a tile-based architecture, but Illusion is not made this way, it needs to hold complete images in RAM.

During some down time after Siggraph 2006 in Boston, I began implementing part one of my plan to move the Fx Tree forward, and this is what you get in XSI 6.0

The first thing is to focus on ILM’s 16-bit float image format. This format trades some runtime performance to get lower memory consumption. The image format is also directly supported by the GPUs.

The images in RAM are the same size as “normal” 16-bit images, which I will refer to as “16-bit unsigned short images”, and we know the Fx Tree can handle those very well. And just like any 16-bit image format, it is possible to do very fast look-up-table processing with them.

Look up table operations on 32-bit float images are not directly possible, and doing the equivalent require a lot more processing, or using the GPU. Look up tables on 8 and 16-bit images requires only a few CPU instructions and a 256 byte and 65535*2 bytes LUT.

Finally, it is generally accepted that the 16-bit float OpenEXR image format is “good enough” for most images. Meaning, it does trade some precision, and the range is not infinite like 32-bit float, but it is very good. For example, between 0.0 and 1.0, there is still more precision in a normal 16-bit unsigned image, where there are 65535 discreet values instead of 15361 like 16-bit float, but generally the human cannot see them, and preserving the values that are out of that range is more important to the quality of the image processing.

So in the FxTree the first thing I did was introduce a new image format I called “HDR”, which is 16-bit floating point. It was named this way, and put after the existing “float” in order to make it obviously that this is where the effort is focused for HDR support (i.e. high dynamic range pixel values), and that it is better.

Then, the core tools were updated, in order of importance for compositing passes, until time ran out on 6.0.

“Supporting HDR” means that an effect supports processing the 16-bit float image without clamping the values. Many of the 32-bit float code path were updated, but generally it should be assumed that there were not and you shouldn’t use them. The HDR code path are always going to execute much faster because the code is new, optimized and painstakingly verified.

The 6.0 Fx Tree now loads OpenEXR images as “HDR” 16-bit float images, and can save them back to disk without loss of data, so File Input and File Output both support HDR. All effects in the “filters” group, plus Over, Transform, crop, Math Composite, support HDR.

However, it stops there. The PainterlyFx, which are an 8-bit photoshop plug-in library, do not support HDR. Keyer, Paint, Warper, Morph, shapes, Distort and none of the color correction tools support HDR or float without clamping, except for the HSV Adjust effect in RGB mode.

To color correct HDR images, you’ll need to use the HSV Adjust’s RGB controls, which are in the “Basic” and “RGB Graph” group. The Fx Viewer uses it internally to apply display LUT. Anything that uses HSV or HLS will clamp the HDR values.

So as you can see, it’s partial support. It’ll get you up and running for doing simple compositing and blurs of mental ray passes. I expect that in the future more effects will support HDR, but I can’t promise a roadmap for this at this time. I think it’s worth putting this in the hands of users even if it isn’t completed. Ideally I’d like in the future to drop the support for non-HDR image formats in order to cut down on the development time, testing, of each code path.

The new image format has also been added to the UFO SDK, users simply need to use the “half” c++ class in the OpenEXR SDK to process the pixels.

This entry was posted on Thursday, December 21st, 2006 at 6:55 pm and is filed under FXTree. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.