Qt Multimedia

This article may require cleanup to meet the Qt Wiki's quality standards. Reason: Auto-imported from ExpressionEngine. Please improve this article if you can. Remove the {{cleanup}} tag and add this page to Updated pages list after it's clean.

Qt Multimedia (Qt 5)

The purpose of this page is to give an overview of the current state of Qt Multimedia in Qt 5 and propose a roadmap for future releases.

For an API overview, please see "Qt Multimedia Overview":http://doc.qt.io/qt-5.0/qtmultimedia/multimediaoverview.html

For an overview of what is currently supported by each multimedia backend, see Qt Multimedia Backends

Planning

High priority

• Improve documentation.

Some classes have incomplete documentation and out of date code snippets. It would also be nice to add QML demos showing MediaPlayer and Camera features.

• Improve auto-tests.

Most backend auto-tests are marked as insignificant and are not run on the CI system. These tests should make sure Qt Multimedia behaves consistently across all backends.

• Port the GStreamer backend to GStreamer 1.0 (currently 0.10).

This would allow to get out-of-the-box support for HW-accelerated video decoding with VA-API. Some work has to be done to benefit from this in QtQuick.

• Add camera support in the WMF plugin

Qt Multimedia currently falls back to the DirectShow plugin for camera support on Windows, but that plugin has an incomplete implementation and DirectShow is being gradually deprecated by Microsoft anyway.

• Make the WMF plugin compile with MinGW.

The Media Foundation headers are incomplete in MinGW, making it impossible to build the plugin. The DirectShow plugin is built instead when using MinGW. We should probably generate these headers ourselves and include them in Qt.

• Deprecate QuickTime 7 plugin

AVFoundation is available since MacOS 10.7. Deprecate the QuickTime 7 plugin for 5.2.

Normal priority

• Add missing APIs to QML Multimedia Items
  • Playlist support
  • QAudioRecorder has no QML equivalent
  • There is no way to select which physical camera to use with the Camera item
  • Add a way to query supported values for camera parameters such as capture resolution, ISO, flash modes and so on…

• Fullscreen video window support in QML

Multimedia backends often have optimizations for rendering video inside a window or a fullscreen window (rather than in a texture or in main memory; especially true on mobile platforms). There could be an API in the VideoOutput item to enable fullscreen but also to choose the kind of surface the video should be rendered into. The implementation is already partly done as a fallback for backends that don't support rendering to a texture or to main memory. Currently this fallback always puts the video window on top of the QML scene, there should be a way to have the window behind the QML scene as well.

• Backend API cleanup

Several control classes, such as QCameraFeedbackControl or QMediaGaplessPlaybackControl, exist but are not used anywhere. Should we remove these or should we use them where needed. Additionally, some controls have missing signals and missing virtual destructors…

• What to do with the DirectShow plugin?

It is the only multimedia backend available on Windows XP. Additionally DirectShow has virtually support for any existing codec (through third-party codecs). Media Foundation lacks this extensive codec support. It probably can't go away for now, should it be improved or left as is? It could be shipped together with the WMF plugin and the right plugin would be selected at run-time depending on the needed codec.

Unprioritized

• Single cross-platform backend

Instead of having a different backend for each platform, we could use a single cross-platform backend. It would eliminate inconsistencies across platforms and reduce code maintenance. The single backend could be GStreamer (might not perform well everywhere) or libVLC (legal issues with codecs?).

• New features
  • Subtitle support
  • Multiple tracks (or streams) support in QMediaPlayer. Some media containers can have multiple video, audio and subtitle tracks. Add a way to list them and select the active one.
  • QMediaPlayer playback loops
  • We have QAudioDecoder, in a similar fashion we could have QVideoDecoder
  • New QAudioEncoder and QVideoEncoder classes
  • Video thumbnail API
  • Audio effects API. e.g. equalizer, bass boost, reverb, etc…