The future of Qt on Android looks bright

Urbi et Orbi

A new year just started a few weeks ago, but Qt on Android doesn’t have time to rest :).

The secret meeting

At the beginning of the year I attended a very important workshop in Oslo. For two full days KDAB (me) and Digia (Oslo office folks) secretly discussed the future of Qt on Android. We talked about lots of features that we are planning to implement in Qt 5.3, 5.4 and even in 5.5 and beyond. Well, today I want to share with you some of the secret :)!

In this article I’m going to highlight only one of them.

The prom queen

“Support for multiple surfaces and mixing raster/GL content” was by far the hottest topic, because lots of other features depend on it. It was also my top priority feature on my TODO list for Android.

What is wrong with current implementation?
Well current implementation supports only a single android native surface, which can be used either by raster only apps with multiple top level widgets (e.g. the drop down list of a combobox is a top level widget) or by OpenGL apps, but is limited to a single top level widget (the OpenGL one). This limitation is very annoying because it makes QGLWidgets apps unusable on Android. It also makes it impossible for Qt to use the android native media video player, which is (seems to be?) required to play DRM videos.

A small step for Qt a giant leap for Qt on Android

Folks, I have good news on this matter! Even though the holiday period is over, KDAB started this year with a great gift for all Qt on Android devs out there! A few days ago I managed to have a single Android platform plugin which can be used by apps that mix QGLWidgets with QWidgets.
The mandatory screenshot:

To accomplish this task I had to find a way to create multiple native android surfaces. On most (all?) the platforms supported by Qt it is a pretty easy job to have multiple native windows, but on Android it is a real pain in the ... you know where :). More about this topic on the “behind the scenes” part.

Why didn’t I do it until now? Why did it take me so long to do it? Well, I had this feature in my mind for a very, very long time, but sadly I didn’t have enough time to check if it really works. KDAB gave me the needed time to do it :) !

Behind the scenes

This is the boring technical part, if you really don’t care how it works, you can jump to the limitations chapter, but for a better understanding of the limitations I do recommend you spend a few more moments and read this part.

Why do we need multiple native android surfaces?

For a better understanding let’s take a simple example: an application that needs a GL surface and also has some (old fashioned) QWidget-based controls. For this application Qt needs two surfaces, one for the GLWidget and one from the QWidget.

Why is it so hard to create another native android surface?

Well, there is no problem to create one more native android surface, the problem comes when you need to control its Z order. The Z order is very important. Let’s assume that you need to show a dialog on top of a GLWigdet, but you can’t guarantee that the new surface will be created on top of the GLWigdet surface. It is possible it will be created behind the GLWigdet surface so the user will not be able to see the new dialog.

setZOrderMediaOverlay saved the day! This method does all the magic, it allows us to control whether the new surface is placed on top of another regular surface. We still can’t control the Z order of two surfaces that have the same ZOrderMediaOverlay flag! It just helps us to create one more drawing layer.

How does it work for Qt?

Now Qt uses three different layers:

• the topmost layer contains a custom AbsoluteLayout view which is used by Qt to create a dummy (fully transparent) control every time you want to edit something. The dummy control is needed by Android window manager to pan/resize your application window when the software keyboard appears. The custom AbsoluteLayout view will be used in the future to add native controls (e.g. WebView).
• the middle layer, is used by Qt to create a single SurfaceView with ZOrderMediaOverlay flag set to true! This surface is used by Qt to draw all raster windows. Yes, one surface is enough for all QWidget based top level widgets. Maybe in the future we can use it to push a single Qt GL Window that will be on top of another GL Windows.
• the bottommost layer is used by Qt to create SurfaceView(s) which are used for GL windows and by native android media video player.

Limitations

As I said previously, ZOrderMediaOverlay only helped us to create a new layer (the middle one) but we still have the Z order problem for the surfaces created on the same layer.

• the last layer surfaces can’t overlap (e.g. you can’t overlap correctly multiple GL Windows).
• because the native controls will be placed on the topmost layer, Qt can’t put anything over them.
• only basic transformations can be applied to native controls (position and size).

Other changes done to android platform plugin

Besides the work on creating multiple native surfaces, I did the necessary changes to have a single android platform plugin and reworked the raster and opengl code (got rid of eglfs and fbconvenience dependencies :)).