Camera Projection Rendering
June 12th, 2007 by Helge Mathee - Viewed 17235 times - Popularity: 77% [?]When dealing with camera projections, which are very helpful for image based modeling, scene layout etc… you sometimes experience some limitations. The camera projection in XSI is not available as a sub-projection, neither can you limit the effect of the camera projection based on surface angle or distance. Another problem are quite weird looking stretches, if the angle of the surface becomes too steep.
After working on this for some time, I realized that those stretches cannot be avoided. I reimplemented the camera projection operator to get rid of them, but ended up with the same stretches and itches, but I found a workaround, which can be applied to image based modeling workflows, as well as mental ray rendering.
Also, of course, I will provide the result as a plugin.
Warning! A lot of images!
The image above shows a default cube with a camera projection applied. As said before, you cannot limit the effect based on angle or distance. The method I propose supports both, as well as multiple camera.
In this example I am using one camera, and limit the camera projection effect to an angle, so that only polygons facing the camera will get the camera space UVs.
Another way of working with this projection, is to use multiple cameras driving one uv projection. This can be extremely helpful for image based modeling, as well as projection of multiple images when dealing with panoramas. You can limit the output of each camera in U and V, so that you can use stripe images, or cross images for example.
To apply the projection, just select the mesh and at least one camera (you can select up to eight cameras), and click the “camera projection” button on the ‘mt_camera_proj” toolbar provided with the addon.
This image shows the distance based limiter in action. Basically, the angle isn’t limited, so all polygon receive UVs, but the distance of the projection is set to 8, so that only polygons in that distance are projected.
So far so good. We still end up with the strechtes though. This is due to triangulation and UV interpolation. There is also no difference in mental ray vs. real time rendering. The problems always show up.
To get around this, the only way to improve accuracy while maintaining the same projection, is to add a shader, which will do the calculations all over again, without using the existing UVs. What’s really nice about this is, that the shader doesn’t actually require UVs, which makes it possible it to apply to groups, layers or even partitions for pass rendering.
Here’s a rendertree showing the new shader in action. It requires an image, and outputs color.
To enable multiple cameras for this use, the shader has all of the camera settings necessary, including field of view, aspect ratio, a position and rotation. Moreover, you can link all of the eight parameters to an existing camera by clicking the “Connect To Camera” button, and then pick a camera in the explorer.
Furthermore there are settings for limits based on angle and distance, this time with a min and max, because, as we are looking at a shader, we can do smooth blends now. The parameter “layer_below” is used for the areas not covered by the projection, respectively for blending. This enables you to use multiple camera projections in a row, by plugging the previous one into the next one’s “layer_below” port.
This image shows a camera projection render with angle based blending turned on.
As we have all parameters of the shader as pluggable ports, you can use other shaders to drive parameters such as the angle limits for example. (pretty wild.. huh!)
Also, if you don’t want to camera-project an image, but a procedural shader, you can turn the “only_use_color” parameter on. In this mode the shader never pulls the texture, but outputs the calculated UV coordinates as a RGB encoded color.
You can then use this color (after beeing converted to a vector) to drive any other texture generator (a ripple in this case). This makes it possible to camera project procedural textures at render time, without any UVs.
The plugin can be found here:
MT_Camera_Proj_V1.4
The plugin contains a toolbar, the camera UV projection, as well as the projection shader (which is a texture shader).
Version 1.4 is motion blur capable.
This is applicable as well to panorama images, by overlaying multiple images….
so if you use a couple of cameras:
a shader setup like this:
you can turn a spherical geometry like this:
into a panorama like this (using 4 photographs):
Let me know of all of the cool stuff that comes out of this… I am pretty sure there are enough crazy people out there to come up with completely different uses for this! ;)
cheers.
Popularity: 77% [?]
Nice one Helge really useful thx.
Oh forgot mention: You can use this shader as a depth blend + incidence as well, if you set it to “only_use_color” and convert the result to scalar.
This is fantastic, Helge! Basic stuff that goes a really long way.
-T
Added version 1.3. Includes a texture repeat control as well as U and V based blending.
thanks a lot…
that will come in very handy in many situations…
does it work in 64 bit version?
Nice Helge! You always have the best toys.
For some reason i cant see the last 4 images. Would it be a problem with my sys?
thanks for sharing helge…
i am missing some images also, bullit
Images are fixed. Something weird with my server, uploaded the images to XSIBLOG now, seems to work better.
Btw: If anyone wants to compile for x64, I can share the sourcecode no problem.
By pressing Camera Projection in MT_Camera_Proj toolbars I receive the message
// ERROR : Object expected – [line 1]
// ERROR : (Microsoft JScript runtime error)
// ERROR : >[1] mt_apply_cameraProj();
// ERROR : Object expected
Any ideas?
Great Helge!
I ve done something close to what you ve done but more messy… It seems really useful and handy.. where can i download it to try?
my method:
(http://www.xsibase.com/forum/index.php?board=29;action=display;threadid=27515)
NSK IBMod_V0.1_ image-based modelling for XSI
the download link is right on this page. watch out! ;)
// ERROR : Object expected – [line 1]
// ERROR : (Microsoft JScript runtime error)
// ERROR : >[1] mt_apply_cameraProj();
// ERROR : Object expected
…is what I get when I try to press Camera Projection in the new toolbar on a mesh (have tried different meshes, 2 different computers, same error).
You are a genius, Helge– this addon looks like a total blast! I can’t wait to play with it!!
Many many thanks!
Brian.
: )
Very nice, Helge! :)
If I understand what you’re doing, a person could set up multiple cameras with
images of different angles of the face, for instance, and surround a head-model
with appropriate textures…then run a rendermap (supposing the head had proper
UVs already) of the model?
Thanks for that! It´s great to have this functionality as a shader
and I can imagine this generally would be a great addition to the default
rendertree nodes, especially when not limited to camera projections
but instead allowing to pick any geometry (using it´s facenormal) as input.
Now that stuff like roadkill makes seamless UV sets easy to create
it´s probably just a matter of time that people are asking why there
aren´t more elaborate ways to project UV´s onto surfaces and bake
the result to another, clean UV set for further use. It´d be great not
to be limited to a fixed set of shapes/texturesupports, e.g. planes, cubes,
spheres, cylinders, cameras and so forth but instead just model what fits
the shot and have a shader that makes it easy to transfer those UVs,
the resulting texture or whatever else like gradients for anglecalculations
to other objects sucessively added to a shot or swapped due to changes
without having to redo any projection linking at all but just assigning a
shader for further use.
Cheers
tim
hey helge,
no time no see. U still in the states? nice plugin will help me to with my matte paintings :D
Ismini
x
Hi Helge–
I’m gettting the following error message when trying to run the script from the “MT_Camera_Proj” toolbar…
// ERROR : Object expected – [line 1]
// ERROR : (Microsoft JScript runtime error)
// ERROR : >[1] mt_apply_cameraProj();
// ERROR : Object expected
Here’s a simple setup JScript which gets me to the point of pushing the “Camera Projection” button from your toolbar:
NewScene(null, null);
CreatePrim(“Torus”, “MeshSurface”, null, null);
ApplyShader(“Lambert”, “”, null, “”, siLetLocalMaterialsOverlap);
ToggleSelection(“Camera”, null, null);
I’m certain I’m missing something very simple, but I don’t know what it is. Both the camera and object are selected, but I stilll error out.
Any help you could point my way would be terrific!
Thanks!
Brian.
: )
UPDATE: although the “Camera Projection” button doesn’t work for me, dropping your “Camera_proj” nodes directly into the render tree works flawlessly.
This is an incredibly fun plugin, Helge! Thanks again for putting it out to the public!
Brian.
: )
Anyone compiled for x64?
Thanks
Hi Helge, amazing tool man!
I’ve working 2 years on DonkeyXote feature film, doing almost all camera projections (and their mattepaintings) of the film, mainly for grounds and sets, and I know very well how tricky It can turn, using default xsi’s camera projection. And now you come with this piece of heaven!! Damn, we’ve already finished the film :D
I mean, I love it, incredible stuff, with an incredible set of options and parameters. A must for next version of XSI.
people, respect this guy.
Thank you,
Diego
If only I had this tool before. Great work. VERY useful.
i’d love to give this a spin, but i’m getting the same error in xsi 6.5 as some other people here when i use the toolbar button provided:
// ERROR : Object expected – [line 1]
// ERROR : (Microsoft JScript runtime error)
// ERROR : >[1] mt_apply_cameraProj();
// ERROR : Object expected
any suggestions?
tia,
-j
Just to revive these comments: has anyone already compiled a x64 version of this great shader?
I am sure someone has done x64 version already. Uncover yourself! :)
im in the same situation, anyone built a 64 bit version?
looks very handy