Find what you need - explore our website and developer resources
The 5.11 release of Qt 3D is mostly about speed and stability but it also introduces a number of new features.
One of them is generalized ray casting which can be used to find objects intersecting a 3d ray.
Available since 5.6, QObjectPicker can be used to detect mouse interactions with an entity. As the user moves the mouse and presses the buttons, events will be emitted by the component. This can be used to change properties of a entity based on the events, highlighting it when the mouse enters, scaling it up when a button is pressed, moving it when the mouse is dragged, etc.
By default, only mouse buttons will trigger events (if the mouse overlaps the geometry). This is done for performance reasons since computations can be expensive (see below). However, this can be changed by setting properties hoverEnabled, to detect when the mouse enters or leaves the entity, and dragEnabled, to detect when the mouse moves across the surface of the geometry.
When the mouse is pressed on an entity with a QObjectPicker component, that component will "grab" the mouse and only that object will be considered for picking events, until the mouse button is released.
When a hit is found, the generated event will contain information about the point of the geometry under the mouse. In particular, it will contain the coordinates of the mouse (in screen space) where the hit was made, the state of the buttons. But it will also include the coordinates of the point on the surface of the geometry both in local and in world coordinates.
By now it should be obvious that QObjectPicker, like MouseArea, is most suited to give behaviours to object in the scene, should as changing the scale to make them bigger when the mouse hover over them, or changing the colour when the object user "selects" the object by clicking on it.
While is very useful in its own right, it has several limitations:
Picking as described above is implemented by casting a ray through the scene and looking for objects which intersect the ray. This is roughly how the process works:
As you can imagine, on complex scene graphs with lots of primitives, this can be expensive. Qt 3D attempts to optimise things by using a bounding volume hierarchy to quickly eliminate entire portions of the scene graph which don't intersect with the ray. It also uses a thread pool to test multiple objects in parallel. Best results can be achieved by properly constructing scene graphs and avoiding single objects with large numbers of triangles.
The algorithm can be controlled globally by using the pickingSettings property of the QRenderSettings class, which is an instance of the QPickingSettings class.
The most important property is the pickMethod:
BoundingVolumePicking which is the fastest method: it will only test for bounding volume intersection. While this is very fast, it is also very imprecise. Internally, Qt 3D uses bounding spheres which can grossly exaggerate the actual volume of the object. Also, a small object positioned in front of a large one may be entirely encompassed by the large bounding sphere and thus be unpickable.TrianglePicking (and since 5.10 to LinePicking and PointPicking) to ask for hit tests to be made with individual primitives. This is of course far more time consuming.Another important setting is the pickResultMode property:
NearestPick (the default), only the closest hit will trigger an eventAllPicks, an event will be generated for every intersection. If you imagine picking a sphere, in most cases this will generate two events. Events will be generated from front to back but they will be delivered separately and there is no way of knowing when the last event is received.There are also settings to control how the face orientation affects picking and a world space tolerance factor to control line and point picking.
The ray casting operations described above are those used by the QObjectPicker backend. As explained before, they rely on mouse events in order to compute the ray that will be used to do the hit tests.
But there are many uses cases for more generalized ray casting. Still in screen space, an application may use a specific cursor that can be controlled independently from the mouse cursor (like a gun sight). Or in an VR environment, you may want to find the object at the centre of the screen to implement gaze selection.
Also in a VR setting, you may want to implement a 3d "wand" which is manipulated using the VR controls. A 3d ray with a position, direction and length matching the virtual wand could be used to find intersecting objects. Using action buttons, you could then implement grabbing and dragging the intersecting objects.
This is an abstract class that cannot be instantiated directly. If defines common properties for all sub-classes. Note that ray caster components cannot be shared between multiple entities.
The runMode property controls how often ray casting tests are performed. All ray caster instances are disabled by default. When enabled is set to true, a ray casting test will be done at the next frame. If runMode is set to SingleShot (the default), the component will disable itself and no further tests will be performed until it is enabled again. If runMode is set to Continuous, tests will be performed at every frame until the node is disabled.
The filterMode property and the list of QLayer instances can be used to control which portions of the scene graph are considered when casting rays. You may want to disable picking on certain portions of the scene graph, or use different rays in different sub-graphs.
This can be achieved by using QLayer components and then adding them to the relevant ray caster node. The filterMode will control if the layers enable or disable ray casting and how to evaluate multiple layers.
Remember the QLayer has a recursive property to control if the layer applies to the specific entity only or if it should affect all of its children.
When a ray caster test completes, the list of results will be assigned to the hits property of the component. This list will be empty if nothing was found. A change notification will always be emitted even if two successive tests produce the same result.
Each entry in the list contains the details of a hit, sorted front to back. It contains:
QPickingSettingsWhile the C++ implementation used a properly typed class, the QML api uses a QJSValue containing an array of javascript objects with the relevant properties included.
QScreenRayCaster is used to do ray casting queries based on window coordinates without relying on the user using the mouse pointer. A test will actually be performed for every active render target.
The 3d ray is computed by constructing a ray at the specified coordinates between the front and back clipping planes and transforming that ray back in to model coordinates using the relevant camera projection and transformation.
The component has helper methods to trigger a test. All that basically does is enable the component so that a test happens at the next frame.
QRayCaster defines a ray in the local coordinate system of the entity the component belongs to. This means the actual ray is affected by the world transformation applied to the entity.
The ray has an origin and a direction.
The ray also has a length, any hit further away from the origin will be ignored. If that length is less or equal to zero, the ray is considered to be infinite. Bear in mind that it has an origin and a direction, so it's only half infinite :)
The component also has helper methods to trigger a test, optionally setting the properties of the ray.
Qt 3d source code contains a basic QML example for using both ray caster classes.
ScreenRayCaster {
id: screenRayCaster
onHitsChanged: printHits("Screen hits", hits)
}
MouseHandler {
id: mouseHandler
sourceDevice: MouseDevice {}
onReleased: { screenRayCaster.trigger(Qt.point(mouse.x, mouse.y)) }
}
function printHits(desc, hits) {
console.log(desc, hits.length)
for (var i=0; i<hits.length; i++) {
console.log(" " + hits[i].entity.objectName, hits[i].distance,
hits[i].worldIntersection.x, hits[i].worldIntersection.y, hits[i].worldIntersection.z)
}
}
In the case of the ScreenRayCaster, a test is triggered whenever the mouse button is released.
RayCaster {
id: raycaster
origin: Qt.vector3d(0, 0, 4)
direction: Qt.vector3d(0., 0., -1.)
length: 5
onHitsChanged: printHits("Model hits", hits)
}
KeyboardDevice { id: kbDevice }
KeyboardHandler {
id: kbHandler
focus: true
sourceDevice: kbDevice
onPressed: {
if (event.text.toLowerCase() == "a") { raycaster.origin.x -= .1; raycaster.trigger() }
if (event.text.toLowerCase() == "f") { raycaster.origin.x += .1; raycaster.trigger() }
if (event.text.toLowerCase() == "s") { raycaster.origin.y += .1; raycaster.trigger() }
if (event.text.toLowerCase() == "d") { raycaster.origin.y -= .1; raycaster.trigger() }
}
}
For the RayCaster, it defines a ray can be moved around the scene using key combinations.
Not the most earth shattering demo :). But it does exercise all the features of the newly introduced nodes and how to do picking without using QObjectPicker.
About KDAB
The KDAB Group is a globally recognized provider for software consulting, development and training, specializing in embedded devices and complex cross-platform desktop applications. In addition to being leading experts in Qt, C++ and 3D technologies for over two decades, KDAB provides deep expertise across the stack, including Linux, Rust and modern UI frameworks. With 100+ employees from 20 countries and offices in Sweden, Germany, USA, France and UK, we serve clients around the world.
Choose between learning advanced QML programming, Modern C++, integrating Rust and Qt or 3d rendering with Vulkan. The KDAB Training Day 2025 will take place in Munich on the 8th of May, right after the Qt World Summit on 6th to 7th of May.
Learn more
Stay on top of the latest news, publications, events and more.
Go to Sign-up
3 Comments
19 - Apr - 2018
lauegi
Mike Krus, thank you ever so for you post.Much thanks again.
23 - Apr - 2018
Yunlong
Very nice, thank you.
27 - Jun - 2019
Alexandre GRANVAUD
Great one The Continuous mode doesn't seem to work though, with a ScreenRayCaster (Qt 5.13)