Overview of Qt3D 2.0 - Part 1

Introduction

Back in the days when Qt was owned by Nokia, a development team in Brisbane had the idea of making it easy to incorporate 3D content into Qt applications. This happened around the time of the introduction of the QML language and technology stack, and so it was only natural that Qt3D should also have a QML based API in addition to the more traditional C++ interface like other frameworks within Qt.

Qt3D was released alongside Qt 4 and saw only relatively little use before Nokia decided to divest Qt to Digia. During this transition, the Qt development office in Brisbane was closed and unfortunately Qt3D never saw a release alongside Qt 5. This chain of events left the Qt3D code base without a maintainer and left to slowly bit rot.

With OpenGL taking a much more prominent position in Qt 5's graphical stack -- OpenGL is the underpinning of Qt Quick 2's rendering power -- and with OpenGL becoming a much more common part of customer projects, KDAB decided that it would be good for us and for the Qt community at large if we took over maintainership and development of the Qt3D module. To this end, several KDAB engineers have been working hard to bring Qt3D back to life and moreover to make it competitive to other modern 3D frameworks.

This article is the first in a series that will cover the capabilities, APIs, and implementation of Qt3D in detail. Future articles will cover how to use the API in various ways from basic to advanced with a series of walked examples. For now, we will begin in this article with a high-level overview of the design goals of Qt3D; some of the challenges we faced; how we have solved them; what remains to be done before we can release Qt3D 2.0; and what the future may bring beyond Qt3D 2.0.

What Should Qt3D Do?

When asked what a 3D framework such as Qt3D should actually do, most people unfamiliar with 3D rendering simply say something along the lines of "I want to be able to draw 3D shapes and move them around and move the camera". This is, of course, a sensible baseline, but when pressed further you get back wishes that typically include the following kinds of things:

• 2D and 3D
• Meshes
• Materials
• Shadows

Then, when you move on and ask the next target group, those who already know about the intricacies of 3D rendering, you get back some more technical terms such as:

• Shaders
• Ambient occlusion
• High dynamic range
• Deferred rendering
• Multitexturing
• Instancing
• Uniform Buffer Objects
• That cool technique I saw at SIGGRAPH...

That is already a fairly complex set of feature requests, but the real killer is that last entry which translates into 'I want to be able to configure the renderer in ways you haven't thought of'. Given that Qt3D 1.0 offered both C++ and QML APIs this is something that we wished to continue to support, but when taken together with wanting to have a fully configurable renderer this led to quite a challenge. In the end, this has resulted in something called the framegraph.

Framegraph vs Scenegraph

A scenegraph is a data-driven description of what to render.
The framegraph is a data-driven description of how to render.

Using a data-driven description in the framegraph allows us to choose between a simple forward renderer; or including a z-fill pass; or using a deferred renderer; when to render any transparent objects etc. Also, since this is all configured purely from data, it is very easy to modify even dynamically at runtime. All without touching any C++ code at all!

Once you move beyond the essentials of getting some 3D content on to the screen, it becomes apparent that people also want to do a lot of other things related to the 3D objects. The list is extensive and wide ranging but very often includes requests like:

• Physics simulation
• Collision detection
• 3D positional audio
• Animation: rigid body, skeletal, morph target
• Path finding and other AI
• Picking
• Particles
• Object spawning
• ...

This is obviously a tall order, and one that we couldn't possibly hope to satisfy out of the box with the limited resources available. However, it is clear, that in order to support these features in the future, we needed to do some ground work now to architect Qt3D 2.0 to be extensible and flexible enough to act as a host for such extensions. The work around this topic took a lot of effort and several aborted prototypes before we settled on the current design. We will introduce the resulting architecture later and then cover it in more detail in an upcoming article.

Beyond the above short and long term feature goals, we also wanted to make Qt3D perform well and scale up with the number of available CPU cores. This is important given how modern hardware is improving performance -- by increasing the numbers of cores rather than base clock speed. Also, when analysing the above features we can intuitively hope that utilising multiple cores will work quite naturally since many tasks are independent of each other. For example, the operations performed by a path finding module will not overlap strongly with the tasks performed by a renderer (except maybe for rendering some debug info or statistics).

Overview of the Qt3D 2.0 Architecture

The above set of requirements turned out to be quite a thorny problem, or rather a whole set of them. Fortunately, we think we have found solutions to most of them and the remaining challenges look achievable.

For the purposes of discussion, let's start at the high-level and consider how to implement a framework that is extensible enough to deal with not just rendering but also all of the other features plus more that we haven't though of.

At its heart, Qt3D is all about simulating objects in near-realtime, and then very likely then rendering the state of those objects onto the screen somehow. Let's break that down and start with asking the question: 'What do we mean by an object?'

Of course in such a simulation system there are likely to be numerous types of object. If we consider a concrete example this will help to shed some light on the kinds of objects we may see. Let's consider something simple, like a game of Space Invaders. Of course, real-world systems are likely to be much more complex but this will suffice to highlight some issues. Let's begin by enumerating some typical object types that might be found in an implementation of Space Invaders:

• The player's ground cannon
• The ground
• The defensive blocks
• The enemy space invader ships
• The enemy boss flying saucer
• Bullets shot from enemies and the player

In a traditional C++ design these types of object would very likely end up implemented as classes arranged in some kind of inheritance tree. Various branches in the inheritance tree may add additional functionality to the root class for features such as: "accepts user input"; "plays a sound"; "can be animated"; "collides with other objects"; "needs to be drawn on screen".

I'm sure you can classify the types in our Space Invaders example against these pieces of functionality. However, designing an elegant inheritance tree for even such a simple example is not easy.

This approach and other variations on inheritance have a number of problems as we will discuss in a future article but includes:

• Deep and wide inheritance hierarchies are difficult to understand, maintain and extend.
• The inheritance taxonomy is set in stone at compile time.
• Each level in the class inheritance tree can only classify upon a single criteria or axis.
• Shared functionality tends to 'bubble up' the class hierarchy over time.
• As library designers we can't ever know all the things our users will want to do.

Anybody that has worked with deep and wide inheritance trees is likely to have found that unless you understand, and agree with, the taxonomy used by the original author, it can be difficult to extend them without having to resort to some ugly hacks to bend classes to our will.

For Qt3D, we have decided to largely forego inheritance and instead focus on aggregation as the means of imparting functionality onto an instance of an object. Specifically, for Qt3D we are using an Entity Component System (ECS). There are several possible implementation approaches for ECSs and we will discuss Qt3D's implementation in detail in a later article but here's a very brief overview to give you a flavour.

An Entity represents a simulated object but by itself is devoid of any specific behaviour or characteristics. Additional behaviour can be grafted on to an entity by having the entity aggregate one or more Components. A component is a vertical slice of behaviour of an object type.

What does that mean? Well, it means that a component is some piece of behaviour or functionality in the vein of those we described for the objects in our Space Invaders example. The ground in that example would be an Entity with a Component attached that tells the system that it needs rendering and how to render it; An enemy space invader would be an Entity with Components attached that cause them to be rendered (like the ground), but also that they emit sounds, can be collided with, are animated and are controlled by a simple AI; The player object would have mostly similar components to the enemy space invader, except that it would not have the AI component and in its place would have an input component to allow the player to move the object around and to fire bullets.

On the back-end of Qt3D we implement the System part of the ECS paradigm in the form of so-called Aspects. An aspect implements the particular vertical slice of functionality imbued to entities by a combination of one or more of their aggregated components. As a concrete example of this, the renderer aspect, looks for entities that have mesh, material and optionally transformation components. If it finds such an entity, the renderer knows how to take that data and draw something nice from it. If an entity doesn't have those components then the renderer aspect ignores it.

Qt3D is an Entity-Component-System

Qt3D builds custom Entities by aggregating Components that impart additional capabilities. The Qt3D engine uses Aspects to process and update entities with specific components.

Similarly, a physics aspect would look for entities that have some kind of collision volume component and another component that specifies other properties needed by such simulations like mass, coefficient of friction etc. An entity that emits sound would have a component that says it is a sound emitter along with when and which sounds to play.

A very nice feature of the ECS is that because they use aggregation rather than inheritance, we can dynamically change how an object behaves at runtime simply by adding or removing components. Want your player to suddenly be able to run through walls after gobbling a power-up? No problem. Just temporarily remove that entity's collision volume component. Then when the power-up times out, add the collision volume back in again. There is no need to make a special one-off subclass for PlayerThatCanSometimesWalkThroughWalls.

Hopefully that gives enough of an indication of the flexibility of Entity Component Systems to let you see why we chose it as the basis of the architecture in Qt3D. Within Qt3D the ECS is implemented according to the following simple class hierarchy.

Qt3D's 'base class' is QNode which is a very simple subclass of QObject. QNode adds to QObject the ability to automatically communicate property changes through to aspects and also an ID that is unique throughout the application. As we will see in a future article, the aspects live and work in additional threads and QNode massively simplifies the tasks of getting data between the user-facing objects and the aspects. Typically, subclasses of QNode provide additional supporting data that is then referenced by components. For example a QShaderProgram specifies the GLSL code to be used when rendering a set of entities.

Components in Qt3D are implemented by subclassing QComponent and adding in any data necessary for the corresponding aspect to do its work. For example, the Mesh component is used by the renderer aspect to retrieve the per-vertex data that should be sent down the OpenGL pipeline.

Finally, QEntity is simply an object that can aggregate zero or more QComponent's as described above.

To add a brand new piece of functionality to Qt3D, either as part of Qt or specific to your own applications, and which can take advantage of the multi-threaded back-end consists of:

• Identify and implement any needed components and supporting data
• Register those components with the QML engine (only if you wish to use the QML API)
• Subclass QAbstractAspect and implement your subsystems functionality.

Of course anything sounds easy when you say it fast enough, but after implementing the renderer aspect and also doing some investigations into additional aspects we're pretty confident that this makes for a flexible and extensible API that, so far, satisfies the requirements of Qt3D.

Qt3D has a Task-Based Engine

Aspects in Qt3D get asked each frame for a set of tasks to execute along with dependencies between them. The tasks are distributed across all configured cores by a scheduler for improved performance.

Summary

We have seen that the needs of Qt3D extend far beyond implementing a simple forward-renderer exposed to QML. Rather, what is needed is a fully configurable renderer that allows to quickly implement any rendering pipeline that you need. Furthermore, Qt3D also provides a generic framework for near-realtime simulations beyond rendering. Qt3D is cleanly separated into a core and any number of aspects that can implement any functionality they wish. The aspects interact with components and entities to provide some slice of functionality. Examples of future possible aspects include: physics, audio, collision, AI, path finding.

In the next part of this series, we shall demonstrate how to use Qt3D and the renderer aspect to produce a custom shaded object and how to make it animate all from within QML.