XSIBlog

An aggressive crash course to XSI architecture

Not a DAG?

One day, I made the comment that XSI is not an operator graph. That technically true, but on the surface it’s not something one wants to hear, because of this simple formula :

Operators = good
not operators =  bad.

That’s over-simplifying things. The technical truth underneath it all is that XSI is a data graph with operator relations, and not an Operator Graph. The Fx Tree, however, is a true operator graph.

And operator graph is often implemented as a DAG, or Directed Acyclic Graph which means it is a graph that has no cycles. (You may be aware that there are cycles in XSI that can happen in a number of senarios, including IK and constraints, so at least that Acyclic part can’t be true.)

An operator graph is a graph where each operator (drawn as circles) in the graph takes a number of inputs, does some work, and creates a new output it passes to the next step. If it takes no input, it’s considered a “generator”. In the image to the left, the graph is evaluated from the bottom to the top, as it is in compositing trees. The data is born at the leaves, and eventually “falls out of the pipe” at the end of the operator graph.

In the classic implementation, an operator creates new data. So if you had an operator that moved a point on a mesh, it would take the mesh as an input, and make a copy of the entier mesh with one point modified. It has to do this because the data flows only one way : the inputs are “Read Only”. This is required because the data of the input could be, for example, a cache that the system is keeping, or maybe that input is going to multiple operators. So don”t screw with that input data.

In some implementations an operator graph may optimize memory and processing by allowing an operator to change the input and give that directly as its output instead of a copy. This is called “pass by reference”, as oposed to “pass by value”, and it can be used if caching an input is not necessary and it isn’t re-used anywhere else in the graph, or cheap to re-generate. But I disgress.

In a pure operator graph architecture, all your data is expressed with Operators, and their parameters, and operators are usually connected directly to other operators.

For example, in the Fx Tree, there is no “image” object, there are only operators that generate images. When you save the scene, no pixel data is saved in the scene, only operator settings are saved

The Operator Stack..

XSI is a data graph. That means the fundamental builting blocks is an object that contains data. In XSI, it is possible to have no operators at all in your scene. This happens for example when you “Freeze” everything, i.e. delete the construction history. The data structure (minus the actual geometry data structures) is pretty much what you see in the Scene Explorer. It consists of property sets arranged in a hierarchy, what we call nested data.

A data object constains zero or more parameters, and may have internal data as well that is not parameters. For example, a weight map object contains arrays of values.

Each object has what we call an operator stack, which represents the construction history of that specific object. It is an array of connections, which we read from bottom-to-top. Operators may be connected to the entier object, or only to some of its parameters.

Operators may read from an object, or write to it.
The first operator to write to an object is called the Generator.

Generators

When you animate a parameter, it creates an operator that contains an animation curve, and this operator is connected to the parameter of the object. On the connection stack, a “write” entry is created, with specification that it is for a specific parameter, instead of the entier object.

That animation curve operator is said to “generate”that parameter, because it overwrites whatever value was there previously. When you create a weight map property, a “Weight Map Generator” operator is created, and with its parameters it “generates” the weight map by giving it its initial values.

Modifiers On Bicycles And Operators In Relationships

We’ll call any operator that processes an existing object a Modifier. For example, the Move Point or the Paint Weight operator are modifiers.

In XSI, the operators modify data “in place”, in order words, an operator like the Paint Weight tool will take the weight map, and modify only the values that you touched with your brush. It will not create a new copy of the Weight Map, plus modifications, which is what a pure operator graph would do.

This implies that the operator reads from the same data object that it writes to. If this were expressed in terms of an operator graph, instead of a stack, it would mean that it creates cycles for almost all operations.

When an operator reads from another object that it writes to, it establishes a dependency between two objects.

Things are Getting Lazy and Dirty
In this sample image, we have three data objects. A, B and C. B is nested under A. C is somewhere else in the scene.

Object A has three operators : (1) is a generator, and (2), and (3) modify the object subsequently in that order. The Scene Explorer also shows you the operator stack bottom-up like this.

Object B and C are in relation with an operator, (4). This could for example be an operator that reads the camera position, and moves the object in relation to it.

If a parameter changes in operator (2), the operator stack is said to be “dirty” starting at the output of (2).

When an object becomes dirty like this, a notification is sent upwards. It travels through the nested data, going up the tree and telling anything who cares that something will need to be re-evaluated. Generally in XSI, the pieces that care are the Viewports, the render region, or a property page.

This is a called Lazy Evaluation and it has two fundamental parts. First, when parameters are changed, the data is only set to be “dirty” and no precessing is done. That’s pretty lazy. Secondly, when other pieces of the software find out that things are dirty, they will only ask processing for the part that are essential to what they need.

For example, an expression operator that is reading from a parameter on an object will only ever cause the evaluation of that parameter, and not the entire object.

Let’s say we only have a viewport on screen. The viewport will hear the news that data is dirty from the Scene Root, and figure the scene needs to be redrawn.

The scene will begin to redraw, and each time an object is about to be drawn, the operator stack will be evaluated as necessary to “clean” the dirty object. In this example, this means that all the operators of A need to be re-evaluated.

Now, XSI is smarter than this, it in many case it will have made a cached copy of the object just prior to giving it to operator (2). In that optimized case, that means that only operator (2) and (3) will re-evaluated. The minimal set of “dirty” operators.

Invisible objects will remain dirty until their data is necessary for someone.

In the third box of the figure, a parameter has changed on operator (4). This will make object B dirty. As an object nested under object A, it will also indirectly cause A to become dirty. Object C is completely unaffected by this, as operator (4) is reading from it, and Object C has not itself changed in any way.

When a parameter in a custom property nested under a 3D object changes, for example, it does not cause the geometry to be re-generated because even though the custom property may appear to be “under” the object in the Scene Explorer, it in fact isn’t. In addition to this, XSI uses a separate data structure for geometry data such as triangles with additional caching technology that minimize unnecessary evaluation.

Screwing the Architecture
If we say that object A and B are visible 3D objects, when the viewport need to redraw it will evaluate B, followed by object A.

The evaluation will look like this

1. Object C - nothing to do
2. Operator 4
3. We can draw object B now, it’s clean
4. Operator 1
5. Operator 2
6. Operator 3
7. We can draw object A now, it’s clean

The best way you can break the architecture is dirtying the graph while the graph is trying to clean it, or reading and writing to objects without being connected to them.

For example, what would happen if operator (4) decided to access (i.e. Read) the parent of B to get some info from it, without using an operator connection? It would get access to a copy of “A” that is “dirty”. This would work some of the time, but would break during playback or render. Operator (4) would be getting the values of the previous frame evaluated, or some random, uninitialized value. This is why it’s extremely important to explicitly have an connection to everything an operator will use.

What would happen if operator (4) decided to modify object C? First of all, no one would be notified about this, so the scene wouldn’t refresh. Secondly, since “C” has no reason to ever ask for an evaluation of operator (4), the result of the scene would be random. Sometimes it would work, sometimes it would not, it depends if the rest of the scene has somehow caused operator (4) to evaluate.

What would happen if operator (4) decided to modify object A? Again, random results. The only operators that are declared to modify A are (1), (2) and (3). This means that the modification that (4) could do to A could be overwritten by other operators at any moment.

And what is the worse way to screw the architecture? Operators that create new object. As you can see here from this explanation, operators are attached to objects at creation time and stay with them until their death. Data objects are the fundamental objects, not operator. And operators must be connected to the data they create or modify. Therefore, it is not possible from an operator to create new objects because it is being called to validate an existing object.

When a scripted operator calls a command to create new objects, it is adding new elements in the scene and causing “dirty” notifications to be sent. Which means that the scene can never be clean after an evaluation of the operator stacks, which in turns breaks the fundamental rule of the architecture.

Out Of The Beaten Path

Sometimes, you will read from object, but not write to it. In the previous pictures, operator (4) is only reading from C.

What happens when more operators are added to object C? Let’s make a new picture for this.

Here we have object “H” and an operator numbered (10). Operator (10) only reads from object H, but we have now added a new operator, (11).

There is a problem with this.

At the first evaluation, you have object H, Version 1, which operator (10) reads.
Then operator (11) modifies H into Version 2.
We have now lost version 1, so we can never call operator (10) again.

This problem occurs because H did not have a generator. In other words, there is no operator that exists that can create a bran new object H, which is what is needed to re-evaluate (10).

In this case, the operator architecture must create a “safe copy” of the object and it will add an internal copy operator to simulate that missing generator.

The problem addressed here is the answer to the question “Why can’t I freeze only part of the operator stack”?

You can’t freeze only part of the operator stack because an operator stack (or graph) architecture needs to be able to re-generate the data at any moment. If you’ve lost some step of the construction history, then you can’t do that. It’s as good as having lost all of them.
Like making a cake, you can’t skip steps.

When an object is completely generated by an operator stack, XSI only saves the operators and not the data. At load time, it can re-generate everything by evaluating the operator stack, just as the Fx Tree re-generates the images from the fx operators.

I hope this helps, or confuses in new and interesting ways.

This entry was posted on Sunday, October 30th, 2005 at 9:07 pm and is filed under General. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.