Qt 3D Synchronisation Revisited

As mentioned in the previous article in this series, Qt 3D 5.14 is bringing a number of changes aimed at improving performance.

Most people familiar with Qt 3D will know that the API is designed around the construction of a scene graph, creating a hierarchy of Entities, each of them having having any number of Components (the frame graph is similar). Entities and Components are only data. Behaviour, such as rendering or animating, is provided by a number of aspects.

Since Qt 3D was designed to be a general simulation engine, different aspects will care about different things when accessing the scene graph. They will also need to store different state data for each object. On top of that, aspects do much of their work using jobs which are parallelised when possible using a thread pool. So, in order to keep data related to each aspect separate and to avoid locking when accessing shared resources, each aspect maintains, when appropriate, a backend object used to store the state data matching each frontend object.

In this article, we examine how state is synchronised between frontend and backend and how that process was changed in 5.14 to improve performance and memory usage.

Frontend and Backend Nodes

Qt 3D scenes are created by building a tree of Entities, and assigning Components to them. Qt3DCore::QNode is the base class for those types.

Entities and Components have properties that control how that will be handled by the aspects. For example, they all have an enabled flag. The Transform component will have translation, rotation and scale properties, etc.

In order to perform its tasks, each aspect will need to create backend version of most nodes to store its own information. For example, the backend Entity node will store the local bounding volume of the geometry component assigned to it, as well as the world space bounding volume of own geometry and that of all of its children. The backend Transform node will have a copy of the homogenous transformation matrix. And so on…

Obviously, if the data in the frontend changes, say some qml-driven animation changes the translation property of a Transform component, then the backend needs to be notified so it can get a copy of the data and trigger updates in the aspects (like recomputing the transformed bounding volumes for culling).

This process of synchronising frontend changes to backend node was implemented using change messages. Every time a property changed (as determined by tracking signals), the name of the property and the new value would be stored in a message object, which would be put on a queue. On the next frame, all the message in the queue would be delivered to the backend objects in every aspect (if they existed). Each backend node would look at the name of the affected property and copy out the updated value, triggering some updating if necessary.

This messaging system was very useful for Qt 3D:

• It clearly isolated frontend and backend nodes. In particular, the frontend nodes know nothing about the backend nodes which is vital for the extendability of the aspect engine.
• Since the aspects (and thus the backend nodes) lived in a separate thread than the frontend nodes, this prevented having any shared properties that would require locking.
• As we will see below, messaging was a useful pattern that could be extended for communication usage within Qt 3D.

However it had a number of drawbacks, in particular:

• Every change generated a message, resulting in a potentially large number of allocations (messages are not QObjects but have a private pimple).
• Backend nodes needed to perform string-based comparisons to find out which property was affected by message.

On some platforms, these drawbacks could have serious performance implications. So 5.14 introduces a new way of propagating changes to address these issues.

Change propagation in Qt 3D 5.14

As described in the previous article by Paul, one of the major changes in 5.14 is the removal of the aspect thread. This means frontend nodes and the matching backend nodes for each aspect now all live in the main thread. So it was now safe to introduce a non-locking synchronisation mechanism to copy the changed data from the frontend to the backend.

The process now works like this:

• Changes to properties are tracked by listening to signals. When one is emitted, the corresponding node is added to a list of dirty nodes.
• Once every frame, the aspect engine will take the list of dirty frontend nodes and, for each aspect, will lookup the matching backend node and call a virtual method passing the pointer to the frontend node that was changed.
• Inside the virtual method, the backend node will copy the data directly from the frontend node instance using its public (and sometimes private) API.

So, no more memory allocations, changes to multiple properties get batched in a single update call, and string-based comparisons are no longer needed to find out what has changed.

In Qt 5.14, all the nodes from the 4 aspects that Qt 3D included by default have been updated to use this new synchronisation mechanism. In an ideal world, the virtual method would have been added to the base class of all backend nodes, Qt3DCore::QBackendNode. However this would have broken binary compatibility. That class has a pimple, the virtual method could have been added there. However, very few of the several dozens of backend node types actually implement derived pimples so it would have required adding a rather large amount of new classes. As you will see looking at the code, each aspect uses an intermediate private class derived from QBackendNode for all the common code for the aspect, so the virtual method was added there. This will be cleaned up in Qt 6 to avoid the duplication of the dispatch logic.

It’s not just about properties

As mentioned earlier, messages were not only used to dispatch changes in properties. They were used in a number of other places also.

Backend node creation and deletion

When frontend nodes get created or deleted, the aspects need to manage the life cycle of the matching backend nodes (if appropriate). This was done previously by, you guessed it, sending messages.

This process has also been changed in favour of a more direct approach. The aspect engine keeps track of created and deleted nodes and will inform the aspects once every frame. The newly created backend nodes will go through the same synching process that is used to update properties.

Hierarchy changes

Other crucial bits of information that need to be synchronised are hierarchy changes and component list changes. If the frontend scene graph is changed in any way (objects added or removed, reparenting, etc), then the backends of every aspect also need to update their internal representation. And, yes, those changes used to be notified using messages.

Now in 5.14, the pimple attached to each QBackendNode has virtual methods that will be called when a node is reparented or a component is added or removed (i.e., it was done properly :) ). Up to now, only Entity in the render aspect cared about those details, so adding a derived private class for that was the way to go.

Spying

Message dispatch is controlled by a subscription mechanism. The backend node would subscribe to message from the frontend node (and vice-versa, see below). But nodes could also subscribe to change messages from other nodes!

For example, the Scene2D backend node subscribed to messages from the ObjectPicked backend node to know when mouse events occurred and forward them to the rendered QtQuick scene.

This has been changed in 5.14 to use the more traditional signal and slot mechanism. This is of course implemented in the frontend nodes (as backend nodes are not QObjects) but it has little overhead as everything now lives in the main thread.

What about the other way?

We saw how property changes in the frontend get propagated to the backend. But what about changes in the backend?

Lots of things happen in the backend jobs: loading of meshes and textures, computation of bounding volumes, updating of animations, etc. Some of the resulting information needs to be propagated to the frontend.

Actually some of it needs to be propagated to the backend of other aspects! For example, as the animation aspect updates the translation value of a transform over time, the changes need to be sent to the frontend but also to the backend in other aspects, in particular the render aspect so it could update the transformation matrices.

Now the actual changes are usually computed in jobs running on a thread pool. Even though the main thread is stopped waiting for all the jobs to complete, it is not safe for these to access frontend nodes directly. So up to now, changes were propagated using messages, flowing backend to frontend and backend to backend, depending on the type of update. This meant, again, potentially lots of allocation on many threads running concurrently.

In order to remove use of messages for this purpose, jobs now get notified, on the main thread, that the processing has completed, so they get a chance to safely update nodes. The process works like this, on every frame:

• Jobs are run as before in the thread pool.
• Each job is responsible for keeping track of data that needs to be propagated to the nodes.
• When all jobs have completed on the thread pool, each job is notified using a virtual method on the job’s private pimple. Jobs can then look up nodes and deliver the changes using public and/or private API.

For example, the animation aspect will interpolate values of properties over time. It knows nothing about which node the properties belong to. This abstraction previously relied on messages: here’s a new value for property “translation”. Now it’s driven by Qt’s very own property system, just calling setProperty on the frontend node. That node will emit a change signal, which will cause the node to be marked dirty and, in the next frame, the new value will be synchronised to the backend node in other aspects. All will animate properly as before.

But, as explained above, this will happen with very little memory allocations and a lot fewer function calls.

This has been implemented for all jobs in Qt 3D’s default aspects. Thankfully, not all of them need to propagate data in this way :)

A note to creators of custom nodes and aspects

As you will have no doubt noticed by now, the message mechanism was central to a lot of Qt 3D’s processes. So now that we’ve changed the way all the data flows around, have we broken all your code?

We hope not!

In particular, the new direct syncing mechanism is opt-in. When backend nodes are registered with the aspect, they need to be flagged as supporting the new type of syncing. If they do so, then no messages will be created, neither at creation time, nor at update time.

If they do not, then the old system remains. With one change: since Qt 3D no longer tracks which properties change (it only flags the node as dirty), when it comes to process dirty nodes at each frame, it needs to create a change message for every property of the object. If the node doesn’t support direct syncing, the default implementation will delivery property change messages for every property defined on the object. So the message handling method will be called much more often than before. The good thing though is that it uses a stack allocated message rather than a heap allocated one. So it’s much lighter on the memory.

But we would encourage developers who have created their own nodes and aspects to update their code and use the new synchronisation mechanisms.

Conclusion

Hopefully, this post will have explained how these changes clarify some of the changes going on in the upcoming release of Qt 3D. Performance wise, these should be very beneficial. We have seen property update times improved by 300-500% on highly dynamic scenes (with lots of objects animated from QtQuick). This also vastly reduces the number of allocations, which should be particularly interesting on embedded platforms where fragmented memory and threaded allocations can be problematic.

For example, Kuesa 1.1 contains a demo called manyducks, which renders and animates, well, many ducks, 2000 of them. Animation is driven from the main thread using a timer. Every duck slowly spins. Up to Qt 3d 5.13, a 30 second run of this demo would generate produce 1.7 million(!) instances of the property update message (which allocating a pimple, contains a QVariant, etc). This number was reduced to zero in 5.14.

Of course, this is an atypical example. Rotation is performed around the 3 axis so updating each duck produced 3 messages. And of course, 2000 identical ducks should really be rendered using instance rendering and all transformation matrices stored in a buffer. And no need to tune your screens, if the screenshot above appear blurry, it's because depth of field effect is enabled :)