QML Engine Internals, Part 4: Custom Parsers

This blog post is part of an ongoing series about the internals of the QML engine.
In today's post, we'll examine the concept of custom parsers in QML.

Recap

In the first blog post of the series, we covered how the QML engine loads QML files.
One important concept was that every element in the QML file is backed by a C++ class, and that each property assignment corresponds to an assignment to the Q_PROPERTY of that C++ class. Let's quickly look at an example again:

Here, the Rectangle element corresponds to the C++ QQuickRectangle class, in which radius and color are Q_PROPERTYs. width and height are Q_PROPERTY of the base class, QQuickItem. Same applies for MouseArea, which is represented by the C++ class QQuickMouseArea.

Q_PROPERTYs everywhere?

Let's expand the example a bit and color the rectangle red when the mouse button is pressed:

Same story again, right? PropertyChanges is represented by the C++ class QQuickPropertyChanges. Looking at the header file, the target property indeed has a corresponding Q_PROPERTY. But wait, what about the color property? There doesn't seem to be a Q_PROPERTY for that...

Custom Parsers

Looking closer at the header file, we can see a hint for the mystery of the color property: At the end, there is another class, QQuickPropertyChangesParser, inheriting from QQmlCustomParser.

QML classes can have an associated custom parser, which handles all unknown properties. So in our case, the target property is a Q_PROPERTY in QQuickPropertyChanges and is therefore handled normally. The color property is not a Q_PROPERTY, and is therefore passed to the custom parser.

To see how custom parsers work, let's quickly recap QML file loading first, which was explained in the first blog post. QML file loading consists of two phases:

1. The compilation phase
   A QML file is parsed and compiled once, which creates a QQmlCompiledData object for that QML file, which is basically contains a list of bytecode instructions that are steps what to do when instantiating the QML file. In addition to that, it contains some binary data that supplement some of the instructions. Have a look at the earlier blog post to see how these bytecode instructions look like.
2. The generating phase
   Each time a QML file is instantiated, the QML engine looks at the bytecode instructions stored in the QQmlCompiledData object for that file, and executes the instructions in a virtual machine.

Custom parsers are invoked in both the compiling and in the generating phase:

1. In the compiling phase, QQmlCustomParser::compile() is called.
   As an argument it gets a list of all properties that were not known to the QML engine, together with the right-hand assignment of the property, which could be a simple value, an object or a binding script function.
   A little bit of debug code confirms that the color property is passed to our custom parser: for (const auto ∝ : props) { qDebug() << prop.name(); for (const QVariant &var : prop.assignedValues()) { const QQmlScript::Variant qmlVar = var.value(); qDebug() << " " << qmlVar.asString(); } }

The output is:

"color" "red"

The custom parser needs to return a binary blob in the form of a QByteArray that stores all the information that it later needs in the generating phase. This bytearray is stored in the QQmlCompiledData object for the QML file.
In the case of the PropertyChanges custom parser, it simply serializes the parameters as-is into the byte array.

1. In the generating phase, QQuickPropertyChangesParser::setCustomData() is called.
   When the VME creates an object that has an associated custom parser, it retrieves the QByteArray that was was generated earlier with QQmlCustomParser::compile() from the QQmlCompiledData, and passes that to the custom parser's setCustomData() method. Since the custom parser created the QByteArray in the first place, it is able to interpret the binary blob.
   In the case of the PropertyChanges custom parser, it simply passes on the data to the QQuickPropertyChanges object. Later, QQuickPropertyChangesPrivate::decode() actually does something with the data: It deserializes it. Basically it creates a list of ExpressionChange objects. These expression changes are processed when the property change becomes active. The most common operation is that it temporarily creates a new binding as long as the property change is active.

How does the QML compiler and the VME know that a class has an associated custom parser? These elements are registered with a special overload of qmlRegisterCustomType() that takes a custom parser as an argument. The property change element is registered in QQuickUtilModule::defineModule() if you want to have a look.

QQmlCustomParser is private API, so it is not easily possible to write your own custom parsers.

Summary

To support arbitrary properties in QML elements, in contrast to using the normal static Q_PROPERTYs, custom parsers are used. These custom parsers get a list of all unknown properties and can do whatever they want with them. As with QML file loading, there are two phases, compiling and generating. In the compiling phase, custom parsers create a QByteArray binary blob, which is stored in the binary data for the QML file. That QByteArray is then again passed to the custom parser in the generating phase, where the custom parser actually acts on the data.

Discussion

Having the custom parser for the PropertyChanges element is quite convenient, simply being able to say color: "red" is quite nice. Even better, you can easily list multiple properties:

Contrast that to the much uglier syntax one has to use when using a PropertyAction:

PropertyAction does not have a custom parser, and therefore the property needs to be specified as a string in the Q_PROPERTY named property. When you want to change two properties, like color and radius, you need two PropertyAction elements.

This is a inconsistency in the API: In one case, one is able to use the nice syntax, in the other case, that is not possible. This is in my opinion quite confusing, especially if one doesn't know about custom parsers.

Another example is the ListElement element, which also has a custom parser. Because its properties are parsed by the custom parser, they don't quite behave like normal properties. Personally, I stumbled upon strange behaviour with them, such as QTBUG-16289.

The last example of a custom parser is the Connections element, which is quite useful when connecting to signals from an object exported from C++:

On the one hand, custom parsers make some of the QML syntax much nicer and easier to use. On the other hand, there are inconsistencies, bugs and lots of code in the implementation.
What do you think, are custom parsers a worthwhile concept?