PySide2

From Qt Wiki
Revision as of 12:21, 12 May 2017 by AlexBlasche (talk | contribs)
Jump to navigation Jump to search

Pyside for Qt 5.x

The Pyside 2 project aims to provide a complete port of PySide to Qt 5.x. The development started on GitHub in May 2015. The project managed to port Pyside to Qt 5.3, 5.4 & 5.5. During April 2016 The Qt Company decided to properly support the port (see details ).

This wiki page tracks the progress of PySide for Qt 5.x development and provides further information concerning the effort.

Tools

The following tools are used to develop PySide:

Getting started

Getting started with PySide2

Missing Qt bindings

Missing Qt bindings

Backlog

The list below represents some short term targets and long term ideas for Pyside2 development. There is no guarantee that this will happen as written down. It is meant as a short reminder and idea collection for future development. In the long term this list is likely to move to https://bugreports.qt.io/browse/PYSIDE.

  1. First time with Pyside (Document how to get started on this wiki)
  2. Testing Infrastructure
    1. What has been done so far?
    2. Qt CI support (PYSIDE-431)
      1. make test
      2. cmake based build system (all repos)
      3. pip installable Pyside
        1. CI has to setup virtual Python environment
        2. Use pip to install/build Pyside (release bundling)
  1. Future changes to PySide
    1. Gaps (modules & API)
      1. Identify the extent of the gaps by porting existing examples from Qt4 to Qt5 (done)
      2. Plugin system (in what form or shape do we need it is uncertain - some failing tests exist already)
      3. Prepare a summary of gaps for other modules
    2. Fundamental Architecture changes
      1. Shiboken
        1. Clang support (PYSIDE-323)
      2. QtQML support (PYSIDE-355)
    3. MyPy (type annotation - fully typed Python API)
    4. General Python features and their adoption in PySide (e.g. Async I/O)
    5. Creator and tooling
      1. Fix QtC debugger to understand Python/C++ mixed internals
    6. Documentation generation
    7. Deployment of apps
    8. Debugging support


Pyside Development Progress Notes

The most current view of the progress can be found in Jira:

  1. Unresolved issues
  2. All issues (including resolved)

The second link is useful to monitor the progress of the backlog. The best way to achieve this is to sort the list by the "Updated" column. Larger backlog/feature items are filed as "User Stories" in Jira.

12. May 2017

- looked into the Windows tests failures, and the reason might be related to how we copy around build artifacts, leading to an incorrect folder structure. Coin team to look at it.
- Fixed a small regression in 5.9 with building on macOS after some cmake changes that happened in 5.6
- Implemented, tested, reviewed and committed the multimedia widgets examples
- documentation from Pyside1 days can not be generated anymore
  -> new approach required using exiting Qt5 doc tools
- QGL* port to Pyside has started -> missing support for arrays in shiboken
 - more thorough design discussion needed
- Coin
 - provisioning for clang in Qt CI ongoing: https://codereview.qt-project.org/#/c/192517/10

04. May 2017

- Coin
 - libclang 4.0 provisioning available (code reviews ongoing -> enabler for Pyside and qdoc testing)
 - 5.6 coin issues resolved
 - macOS related CI/cmake failures were resolved
 - some CI sync issues among pyside submodules (always submodules HEAD is tested)
 => may have to change once we come closer to the final release ->
 => fine as long as we are in heavy development mode
 - discussion around Ubuntu and OpenSuse configurations
 - minimal cmake requirements not met on all distros
   - 5.6 only be build using RedHat 6.6
   - 5.9 builds have updated Linux distros -> may be able to obtain a new cmake version without hacking the standard toolchain
- QtMultimedia and QRaster* API bindings further completed
- added tests and examples for the above bindings/APIs
- PYSIDE-510 (required for simplified and more generic testing of bindings)
- PYSIDE-504 work is ongoing
- PYSIDE-507 -> avoid hardcoded includes when certain Qt modules are not available
- brief discussion whether to merge the shiboken and pyside repo
 -> advantages from a CI perspective and reduces pyside setup complexity

26. April 2017

- Coin 
  - provision for 5.9+ branches -> we would like to use libclang 4.0
  - test still not yet executed due to branch mismatch which pulls in dev branches
- missing bindings in QtCore/QtGui/QtMultimedia - work in progress
- clang support done (except for a Mac issue)
  - Pyside clang version required Qt 5.7 or later
- PYSIDE-500 done
- work on PYSIDE-500, 502, 504, 497 will continue
- qdoc work started trying to recover whatever documentation is left from Pyside 1.x

20. April 2017

- Fix for PYSIDE-331 broke the Windows build; under investigation
- Refactoring, fixed build warnings
- Looked at debug builds on Windows

13. April 2017

- discussion around merge policies between 5.9 and 5.6 branch
- Linux debug library issues fixed
- PYSIDE-488 fixed via workaround
- fixed a unit tests
- worked on qdoc/html generation in Pyside (failed so far - waiting for qdoc maintainer feedback)
- PYSIDE-331 patches merged -> more bugs found in the meantime

6. April 2017

- 5.6 branch created
- 5.9 branch to be created as well (easier setup for Qt 5.9 testing in Coin)
- Coin
   - not passing build platforms have been disabled
   - otherwise COIN passing on 5.6 (except for 2 failing test) -> soon to be enforced
   - license checker passing in dev
- clang changes merged to dev branch (after reviews)
  - clang provisioning on Coin still missing
- for now shiboken is not a generic C++ bindings generator (targeting Qt use cases only)
- TODO: enabling the generation of documentation from repos
- fixed mixed usage of debug and release build (no debug builds for windows)
- [PYSIDE-331] work continuing - fixed, tests are missing

30. March 2017

- Coin
   - some platforms passing and will be enabled by default
   - MinGW, some OSX and cross compile targets remain out
- will branch pyside dev branches to 5.6, new dev becomes clang branch
   - regular merges from 5.6 -> dev will start happening
   - Pyside 5.6 branch will continue to work against Qt 5.6
- need reviews for PYSIDE-323 to merge clang changes
   - Qt 5.10, 5.9 (on main desktop plawtforms working)
   - for now the Pyside dev development will be based on Qt 5.9 (to be bumped up later on)
- [PYSIDE-331] - in progress
- [PYSIDE-156] - 

23. March 2017

- Coin
  - progressing
- More tests fixed
- wip/clang branch created, will receive Clang parser port with instructions
- Refactoring of shiboken, replaced QtXml classes by QXmlStreamReader allowing for stricter error checking

16. March 2017

- Coin
  - the fixes are still integrating or are under review (no further progress until this is done)
- clang parser replacement PYSIDE-323
  - most test failures related to clang are fixed
  - tests ran pretty much through on Qt 5.9 (Windows and Linux - Mac not yet verified)
  - code cleanups in shiboken
  - will create feature branch on all pyside repos to get the clang patch series under CI control
  - merges from regular pyside branches into clang branch will commence
  - readme required that explains how clang is to be built (if not provided by platform)
- [PYSIDE-331] - in progress

09. March 2017

- Coin
  - lots of changes in Gerrit for Pyside and coin
  - the outcome has to be check once everything merged
  - upcoming clang dependencies in shiboken introduces new requirements for Coin
  - clang changes to be dealt with after pyside branching
- clang
  - first complete bindings generation with Qt 5.6 and Qt 5.9
  - some failing unit tests which have to be looked at individually
  - actual merge depends on pyside branching which in turn depends on recent Coin changes
- smart pointer support
  - patches generally done, gerrit review ongoing
  - rather large patch
- created list of missing bindings in Pyside (Missing Bindings)
  - mostly class level view, global functions at al not covered
- Jira cleanup
  - PYSIDE-464, PYSIDE-217, Pyside-224 fixed
  - most bugs reviewed now and valid ~100 bugs were identified

02. March 2017

- Coin
  - Coin runs tests on Windows and OSX (some failures in the test - to be investigated)
  - Windows provisioning somewhat blocked due to CI issues
  - no progress on Linux
  - OSX 10.8 ran tests with expected results
  - OSX 10.9 has issues due to Pyside not supporting namespace builds of Qt
- clang
  - preprocessor has remaining issue
      -> with clang compiler but gcc seems OK
      -> Qt Core wrapper close to compiling (QHash issues remaining)
- Smart pointer work continues
  - some uses cases work, other use cases might remain
- PYSIDE-364 was fixed (patch pending)
- General error review on bugreports.qt.io
  - 50% done (prioritized bugs are handled/checked bugs - "done" bugs)
- ran address and leak sanitizer over Pyside which resulted in some worrying cases (need to be addressed going forward)

22. February 2017

- PYSIDE-462 Improved solution approved
- PYSIDE-205 Hard-to-detect memory leak plugged
  - slow memory leak found in shiboken with thousands of false positives
  - used differential analysis of valgrind output
- Clang: Fixed Preprocessor handling (set defines, include paths), got minimal binding tests to pass, now adapting MetaBuilder and PySide to what Clang finds when parsing Qt. Good news: Very little need to set Q_DOC or other magic defines
- Coin: Slow progress, currently struggling with provisioning

- extensive bug triaging ongoing in Pyside Jira project

16. February 2017

- Coin (no update - other priorities in release/CI team)
- Clang (progressing on pre-processor work)
- PYSIDE-315 closed
- PYSIDE-462 more review needed
- Shared pointer API very slowly progressing (lots of different fixes needed)
- QMimeDataBase support added
- QUrlQuery support blocked on QDOC defines in Qt sources (shiboken sets the wrong define)
- chasing memory leaks (PSYIDE-205)
- extensive bug triaging ongoing in Pyside Jira project
  - confirming and testing the reported bugs (70 of 200 processed)
  - some smaller bugs were fixed in the process
- quite a few bugs were discovered this week while working on the above items, e.g.:
  - deployment of pyside applications partly broken due to hard-coded install paths in Qt libs
  - Qt events are swallowed when Pyside error/exception occurs
- 

9. February 2017

- Clang progressed
  - pure C++ tests for API extractions are passing (C++ parsing based on clang)
  - even works with cmake
  - old C++ preprocessor still to be replaced
  - working through old preprocessor and checking what magic it did and how clang might be able to replace it
- PYSIDE-315 & PYSIDE-462 fixes pending for review
- COIN
  - continue progress an getting more platforms to pass (unblocking one step at a time)
  - Windows (32bit) and mac are building, Win 64bit still not building
  - OSX and Windows are stopping in the test runner phase
  - missing cmake update still on Linux (provisioning update required)

26 January 2017

- PYSIDE-315
  - principle fix merged but some minor improvements still pending
- Pyside-462
  - essentially a feature request as C++ default parameters are not yet supported by Pyside
  - discussion ongoing how to address the problem
- COIN 
  - 10.11, 10.9 & 10.8 build passes
  - 10.10 still failing during build
  - Progress on Windows and running into new problems further down the path
  - Redhat failed to general brokenness of the platform in CI
  - Ubuntu no further progress
- clang parser work progressing
  - completeness of work is currently measured by passing unit tests
  - there is a long way to go 
  - see PYSIDE-323 and associated patches for progress monitoring
- Fixed regression which prevented Pyside to compile with Qt 5.5.1
- shared pointer support
- no update on 2nd February 2017 (next week)

19 January 2017

- PYSIDE-315
  - a possible fix is pending, some minor performance improvements are still possible
  - caused by different signal/slot ordering in Qt4 and Qt 5
- COIN 
  - issues on 10.8 and 10.9 platforms should pass now
  - 10.11 & 10.10 still have issues
  - Windows timeout problem fix in COIN (but no COIN update until 5.8.0 released)
  - Linux builds are failing
    - Ubuntu 14.04 fails due to cmake issue
    - Redhat 6.6 & OpenSuse 13.01 deferred
  - current COIN freeze for 5.8.0 release affecting patching of COIN for Pyside
  - next steps in priority -> run tests on 10.8/10.9 and get Linux running
- PYSIDE-79 regression fixed as well (some interaction between PYSIDE-315 & PYSIDE-79)
- PYSIDE-462 to be looked at next
- clang parser work progressing

12 January 2017

- PYSIDE-315
  - sorting of slots/signal connection changed on Qt side and Pyside side has not caught up
  - had similar connection issues on the QML side, need to investigate whether there is a connection
- PYSIDE-79
  - caused a regressions (not yet investigated)
- fixed OpenGL types not being recognized on MacOS (partly fixed)
- API's with shared/smart pointers in Qt don't work
  - has potential long term effects and investigation has started into the reasons
- Refactoring shiboken in preparation for clang
- COIN currently runs tests with namespaced Qt
  - short term fix to exclude namespaced Qt builds with Pyside (broken on MacOS 10.9)
  - other failures: missing libraries on MacOS (10.11), missing provisioning on Windows, cmake issues on 10.10
  - unknown state on Linux (current Redhat too old)
- Prioritization after status round:
  - PYSIDE-315 to be investigated based on recent signal/slot patches for QML
  - Smart pointer issues reduced in priority to provide space for PYSIDE-315 & PYSIDE-79 regression
- verify that COIN runs testrunner (not just building Pyside)
  -> not yet verified since we are still failing builds in COIN

5 January 2017

- COIN update
  - COIN changes merged (no further patches pending)
  - need to run an integration test
  - issues related to different build platforms still to be expected
- Continue with clang
  - backtracking a bit (reusing some node API's during parsing but otherwise use clang to populate the tree)
- test blacklisting reviewed (some removed - mostly signal related ones, one new regression)
- regressions in QtQuick were worked around (caused by recent Qt Quick changes)
- issues with macos framework style includes in Qt
- PYSIDE-315 debugging ongoing, very hard to track down
   - was it a regression from a previous Pyside release (e.g. Qt 5.4?!?
- hard-copied Qt 5.4 based headers still in existence
  - updated needed but would shut Qt 5.4 users out

22 December 2016

- lots of discussions around the COIN patches
  - source archive setup being under review 
  - Qt 5.6 provisioning patches merged
  - CI uses Python 2.7 at this stage
- clang C++ parsing continues
- fix of some Pyside unit tests (now have a clean slate again)
- finished QtQuick port
- some overflow problems have been fixed in shiboken

15 December 2016

- COIN changes pending for testing infrastructure
    -> https://codereview.qt-project.org/177136 (Pyside change)
    -> https://codereview.qt-project.org/176968 (Coin change)
    -> https://codereview.qt-project.org/179662 (Provisioning changes)
    -> https://codereview.qt-project.org/179663 (Provisioning changes)
- waiting for COIN development team to review/accept the pending changes
- no update in PYSIDE-315 (under investigation)
- Flushed out a couple of shiboken, QML, qml example bugs
- Some bugs related to parser delayed until clang parser task done (https://bugreports.qt.io/browse/PYSIDE-322)
- most basic shiboken API extractor test to pass (global enumeration test)
- some trouble with int size data types
    -> Python 2 & 3 are different and the relevant C++ data types are yet again different from platform to platform

8 December 2016

- COIN patches pending approval
  - Windows provisioning reviewed
  - Linux, Pyside, COIN itself
- Clang changes progressing, comparing AST tree from old parser and clang
- PYSIDE-79 done
- PYSIDE-315 under investigation
- QtQuick patches taking shape (some template magic and function pointer 
  features in Qt cannot be parsed by shiboken)

1 December 2016

- COIN 
- PYSIDE-79 there seems to be a final work around (https://codereview.qt-project.org/#/c/176374/)
   - fixing a few tests in the process
- clang update
  - dumping AST, identifying the required info
  - work in progress...
- problems with global static QObjects on the Qt side calling back into Python during app exit
  - may require some changes on the Qt side

24 November 2016

- automatic COIN triggering for submodules work in progress
  - several discussions on this topic this week
- Clang investigation (what library to use)
- fixing bugs on Qt side for Pyside
- Qt QML support almost done, work will continue with Qt Quick
- PYSIDE-79 work story continues
  - reference counting not quite accurate but point of deallocation of ref count not identifiable

17 November 2016

- Pyside team suffering from sickness -> not much progress this week
- Pyside-79 fix had a lot of negative side effects (breaking existing tests)
  - internal object reference counting is the predominant issue at hand
  - continue to work on the bug (no resolution yet as delayed due to sickness)
- COIN some progress but still open discussions on branching policy required

11 November 2016

- PYSIDE-79 being fixed
- PYSIDE-315 to be addressed
- Shiboken and Clang
  - familiarizing with clang and its parser's inner working
  - added experimental qmake project definition for shiboken (makes work in Qt Creator easier)
- Qt QML on Pyside work progressing
- work on COIN did not progress due to conflicting priorities inside COIN development

3 November 2016

- working through the QML stack esnuring all required API's are exported (https://codereview.qt-project.org/#/c/175682/)
- small build system patches
- PYSIDE-79 work progressing
- OpenGL support fixed (PYSIDE-314)
- COIN issues
   - repo interdepencies not working yet, suggestion under discussion and to be implemented
   - most license checks have been fixed
   - Qt 5.6 based Pyside to skip check
   - after branching for C++11 work the dev branch should work from license check perspective
   - eventually the entire CI needs to run through (more hidden problems could be hidden)
- Started working on C++11 compliant parser for shiboken
   - libclang will be used
   - first target is to replace the AST tree implementation
   - requires clang setup in the CI

27 October 2016

- Work on WebKit/WebEngine support
- Further work on COIN support
- Planning meeting
- Workshop for C++11 support in Pyside (PYSIDE-323)

20 October 2016

- Qt Quick support submitted
- QtOpenGL support submitted
- Further work on COIN support, license headers

13 October 2016

- Qt CI update
  - Coin changes have merged but integration not yet working
  - Qt CI enables on Pyside side merged 
  => COIN integration fails with license issues
  => requires review of license conditions for all files, some files are not even relicensable
  => skipping license check for now, most likely to be done later again but requires changes to license check script
  - more issues of interworking between QtCI and Pyside expected (won't be visible until license problems resolved)
  Open issue: does a change in Pyside repo trigger a rebuild of everything?
- Qt QML support progressing and first patches merged
  - serious bugs have been fixed, more complex QML examples are now working

29 September 2016

- Coin integration close but not merged
- Bugfixing in particular on the shiboken parser side
- QML/Python binding and tests fixed -> general check all day-to-day aspects of QML are working
- QML examples porting 
   -> still some failing tests
   -> Check all QML/Quick class are exported

22 September 2016

- Refactored Shiboken, udnerstanding build sequences
- black list for unit tests defined and tested
- fixing of tests
- QML example fixing continued (QML bindings not working)
- Pyside side for COIN done (pending integration checks)
- COIN integration still wip due to long test and retest cycles
- pyside and shiboken repo to be relicensed similar to other Qt products
=> this should address any issues regarding the status of generated code too

16 September 2016

- Pyside side for CI testing ready for testing
- status of COIN side to be determined. Code exists need help from CI team to confirm status.
- build system infrastructure improvements in pyside setup
- Qt logging now working
- and more
- Update on bug handling
  - PYSIDE-88 continuing
  - PYSIDE-349 (Multimedia ported)
  - PYSIDE-344 fix pending on codereview
  - make debug builds of Pyside work (OSX works, Linux has work in progress patch, Windows side awaiting contribution)
- QML support work in progress
  - examples are slowly ported with aim to identify bugs

25 August 2016

- Properly implementing QML experience in Pyside
- Unit test fixing
- automated CI testing for Pyside
  - working locally but still failures occurring on various other test machines

4 August 2016

- Additional Pyside examples under review
- Fixed warnings coming from Shiboken

21 July 2016

- main Pyside 2 example port done
- additional examples to fill gaps are being ported (as per prio list)
- CI patches running, but still gaps (should be done by next week)
- about 86% of auto test working (failing count of test 80+ auto test)
- Project Test Status

14 July 2016

- Pyside 2 examples ported
- OpenGL & SVG not working
- QtQuick 2 is in strange situation (QtQml depends on QtQuick)
- Python 3 realed Unicode handling not working with Qt
- QMessageBox hangs
- no documentation for any example
- lots of warnings when building wrapper
- results: https://paste.kde.org/pgje2toyh/mlbqic
- COIN setup for pyside https://codereview.qt-project.org/#/c/158336/
- Qt 5.7 still blocked due to missing C++11 support in shiboken
   - food for ideas: https://steveire.wordpress.com/


7 July 2016

- automated CI testing
- patches for Pyside and Qt CI side required
- Testing somewhat more complicated due to closed nature of Qt CI 
- script to port examples to Qt 5 -> https://bugreports.qt.io/issues/?jql=labels%20%3D%20exampleport5
- all examples to remain BSD licensed