Qt for Python Development Notes: Difference between revisions

From Qt Wiki
Jump to navigation Jump to search
No edit summary
mNo edit summary
Line 3: Line 3:


==== 21. June 2018 ====
==== 21. June 2018 ====
- Publishing snapshot wheels was disabled for some time, but is now back online.
  - The team has been mainly solving issues related to the first set of wheels from the Technical Preview.
  - The team has been mainly solving issues related to the first set of wheels from the Technical Preview.
   - Missing DLLs for the Windows wheels
   - Missing DLLs for the Windows wheels

Revision as of 05:55, 25 June 2018

2018

21. June 2018

- Publishing snapshot wheels was disabled for some time, but is now back online.
- The team has been mainly solving issues related to the first set of wheels from the Technical Preview.
 - Missing DLLs for the Windows wheels
 - Examples incompatibility due to previous changes, missing resources, etc.
 - Wrong paths associated to the QLibraryInfo class
- Documentation:
 - There are many snippets that contains the C++ counterpart, so there is a lot of effort to translate them to Python

14. June 2018

- Official Tech Preview released yesterday: http://blog.qt.io/blog/2018/06/13/qt-python-5-11-released/
- focus has been on release details and following up on comments and feedback after the release
- The team is grateful for all the suggestions and constructive feedback.

07. June 2018

- Last steps for TP
 - Latest changes merged (PEP related plus other features)
  - ActivePython 3.6.0 broken
 - PyPi wheel size issue: Still no answer from PyPi :(
 - Documentation check (code examples need to work)
 - Blog posts
 - wheels upload? documentation url change? etc
- Things to do
 - pyside2-tools update sha
 - merge the last commits from 5.9 to 5.11 to dev
 - create release branch
 - update version number 
 - create a tag once we know the final version.

31. May 2018

- ready for TP?
 - 32bit win wheels not available yet
 - request to upload larger than usual pypi wheels filed with pypi/warehouse project
 - some wheel renaming related patches still pending to comply with pypi requirements
 - PEP changes have merged earlier this week
 - example patches still under review
 - documentation not packaged as part of release (published on doc.qt.io on day of release)
 - ByteArray changes re-based on top of PEP patches -> to be merged until end of week
 - have release candidate wheels early next week for testing
 - aim to release it on 13th June 2018 (might shift till Monday (18th June) afterwards)
- assessed missing bindings situation

24. May 2018

- discuss details of pypi uploads (accounts, processes and limits)
  - need package size extension from pypi
  - naming issues for packages
  - download.qt.io will be backup distribution source
  - need to verify pypi working 
- package finalization and testing (limits on changes?)
  - entering code freeze mode very soon (limited commitment for time being) 
  - pep changes 
     -> 1 test failure, 1 build failure
     -> attribution and doc improvements still to be done
  - ByteArray changes (interdependence with PEP)
  - potential doc changes
   -> tab browser example to be finished
   -> latest logo files need to be pushed 
  - final release testing
  - TP blog post

17. May 2018

- updated icons/images for Qt for Python project
- PEP updates
 - completed the feature implementation but some bug fixing still open
 - discussed how heap type change and PEP changes could be easily reviewed/merged (huge patch set series)
- coin
 - 5.11 & 5.6 broken, 5.9 & dev working
 - waiting for larger coin update which will enable QtForPython to be more independent from Qt CI configurations
 - Win32 (VS2015 only) pip received some changes but not quite available yet (subject to coin update)
 - Temporary Win32 wheels tests looked promising
- missing bindings for QtCore/Network/Multimedia/GUI/widgets added (https://wiki.qt.io/Qt_for_Python_Missing_Bindings needs updating)
- bindings for QtScxml in the works
- known issue when dealing with unicode data/strings (seg faults)
- various documentation fixes
- dynamic System clang builds with Linux supported
- small build fixes
- Decreased wheel sizes
- more work on blog posts and examples.

10. May 2018

- skipped due to public holidays

03. May 2018

- coin
 - 5.6 still blocked, 5.9/5.11/dev working
 - Win32 pip still on backlog
- documentation
 - references to Shiboken being worked on
 - dot images finally working on https://doc-snapshots.qt.io/qtforpython/index.html
 - various fixes to non-class related docs
- PEP updates
 - integration attempts have revealed some issues with shiboken, private ctors/dtors
 - one patchset series for PEP support and Heap Type changes pending
- license updates 
- working on Python 2 & 3 type differences (avoiding developers having to deal with these differences)
- Tab Browser example merged

26. April 2018

- New icons for project in work
- 5.9 branch working  (5.11 branch is broken due to issues on the Qt side)
- snapshot tool for Windows broken (crashing) -> fix pending but blocked by CI
- in the mean time only mac and Linux snapshots working at the moment
- 5.6 branch broken as well as 5.9 changes for qt5.git need to be merged to 5.6 too
- delivery of 32bit Windows binaries not started
- tiny bug fixes and other types of cleanups (e.g. build scripts) left and right
- General replacement of Pyside name against Qt for Python (module name remains as Pyside)
- APIExtractor documentation still being merged to git
- style sheet issues being addressed
- QtWebEngineCore bindings finally merged
- PEP work ongoing (and coin to work)

19. April 2018

- COIN finally back in working shape
- Further fixes for documentation, module pages, experimentally tried to show in Qt Assistant
- Tab Browser example further improved
- Qt for Python blog posts being worked on
- Continued working on WebEngine issues
- Some investigation and preparations to make WebEngine apps work with PyInstaller (very early phase, still doesn't work)
- Pending patch for QtWebEngineCore bindings
- Installation adjustments for standalone builds on Windows
- Heap types are complete. Starting integration with PEP 384.
- Finished the integration between ApiExtractor and Shiboken documentation.
- Worked on a couple of blog posts.
- Wiki structure and started to write the Tutorials

12. April 2018

- Getting Started guide under works by doc engineer
- focus on fixing coin on Mac & RedHat 7.4 (looks promising so far)
- shiboken documentation and diagrams being updated
  - old and unused docs dir shall be cleaned out
- Tab Browser example improved
- License page generated based on attribution files
- lots of discussion when to release Qt for Python TP
  - after Qt 5.11 release (scheduled for 31.5.) 
  - likely to be mid June (stay tuned)
- WebEngine issues
  - needs updated qt.conf files to be deployed for apps
- Heap types (work in progress)
- still more work in pipeline to get code PEP 384 ready (Heap types are prerequisite)
- Qt for Python blog posts being worked on

5. April 2018

- first doc snapshot is up and running: https://doc-snapshots.qt.io/qtforpython/
- more work to be done to automate generation of snapshots
- briefly talked about technical blog post opportunities
- pdb files from Windows wheels removed (reducing the wheel size)
- improving the tab browser example
- coin currently not accepting changes
  - currently lots of pending patches start to create backlog
- Heap type taking longer than expected
- qApp macro reference leaks finally resolved
- quite a few formatting changes to enforce a common coding style
- lots of smaller fixes

29. March 2018

- skipped due to Easter holidays

22. March 2018

- Release process (do we follow Qt?) 
  - Release manager provides packages
  - RTA testing to be setup -> for each package we run RTA
  - Request to be put on mailing list
  - Current Test setup proposal
    - Win10 (Python 3.6)
    - Linux (Python 3.6/2.7)
    - Mac (Python 3.6/2.7)
  - To be defined a test set (//TODO)
  - Testing information to be collected in wiki
- Documentation
  - QML advanced tutorial documentation updated
  - Landing page in the works
  - snapshot still delayed by other qdoc bugs
  - style sheet update considered
  - so far only dealt with class docs, started to include unrelated doc pages
- snapshot testing (angle & webengine not properly packaged on Windows)
- enum support improved (dealing with scoped enums and enum values)
- general bugfixing to increase stability and address mem leaks
- Talks to PyData submitted
- Qt Charts examples ported
- Proper support for QFlags being worked on
- PYSIDE-595 fixed
- PYSIDE-595 first heap type protoype under review proving that concept works
- PYSIDE-560 depends on PYSIDE-595
- CI side
  - coin did Python updates which temporarily broke Linux wheels
  - briefly discussed manylinux request on mailing list (Qt forms bottom line, no point to extend beyond scope of Qt)
  - final list of wheels for release (win, linux, mac) x 2 (2.7 & 3.x Python)

15. March 2018

- Fix to create 3.6 Linux wheels under review.
- Fix scriptableapplication build on Windows, and other small bugs.
- Two QtCharts examples were ported and abstracts were written to submit PyData talks
- Updating the QML advanced tutorial and adding index.rst and gettingstarted.rst
- Tried the wheel pkgs for python 27 on Linux. Works OK except the webengine example, which gives random crashes.
- Docs: PYSIDE-620 Shadow build issue fixed, fixed usage of deprecated Sphinx API, 
- Continued on PYSIDE-560, currently  Windows problem with Linkage
- Debugging on PYSIDE-595, final stages
- Fixed PYSIDE-258

08. March 2018

- msvcrt.dll to be shipped on Windows
- wheel snapshots are automatically generated
  - start with public availability under http://download.qt.io/snapshots/ci/
  - snapshots to public pypi server under discussions
- discussion on how to increase value of official pyside mailing list http://lists.qt-project.org/mailman/listinfo/pyside
- looking at build issues on Windows (improving error handling)
- Participation in Python User Group meeting in Berlin (https://www.meetup.com/Python-Users-Berlin-PUB/events/xhqwhpyxfblb/)
- PYSIDE-595 - Heap Type conversions ongoing
  - test bed created for easier development
- Documentation
  -> Managed to make libxml, libxsl an optional dependency falling back to QtXmlPatterns (despite QTBUG-66925)
  -> Small fixes to doc modifications
  -> Fix class inheritance diagrams for nested classes

01. March 2018

- automatic snapshot process almost in place (pending reviews) (RTA to follow)
- general bug fixing (including build and version problems)
- most community Pyside1 pages point to official Pyside 2 (Qt for Python) wiki pages
- PYSIDE-595 - Heap Type conversions ongoing
- documentation improving
  -> class diagrams being generated
  -> missing links and references
  -> dependencies to qdoc bugs
  -> snapshot process in works
- added support for scoped enums in pyside
- Important things discussed:
  - PySide2 versioning (current proposal WIP patch at https://codereview.qt-project.org/#/c/221767/)
    - Do official releases follow Qt branch names? - 
      -> yes (minor release of Pyside is bound to same minor release of Qt
      -> patch level version may differ
    - Will we have snapshots, and if so, will they be uploaded to pip? How will we handle snapshot versions?
      -> tentative yes, if public pip server supports snapshots Pyside should be good citizen
    - alpha / beta / RC versioning?
      -> Pyside should follow the Qt schema
    - Probably need to take into account PEPs for versions pushed to Pypi.
    - Shiboken / PySide2 versions in sync?
      -> yes, they should be in sync
    - Will we specify a version for 5.6 branch?
      -> yes, but no packages being made
    - Are we sticking to the package name PySide2, but follow Qt 5.x versions as PySide2 versions?
      -> Pyside2_5.11.0 as in https://codereview.qt-project.org/#/c/221767/
  - Do we want to deploy msvcrt dlls into Windows packages?
    -> needs more research - is it really needed ?
  - Do we want to ship debug symbols for Qt inside the PySide2 packages? 
    -> not for now
  - Do we want to ship debug Qt shared libraries for macOS packages? (not possible on Windows, not applicable on Linux afaik).
    -> not for now
  - How will the pip upload process look like? Manual uploads? Who takes care of it?
    -> Qt release team will take care of it
  - Current branching policies:
    -> 5.6 branch stops being merged into 5.9 (after pending patches have merged) -> cherry-pick mode
    -> 5.9 is base line for patches and being forward merged to 5.11
    -> no packages from 5.9 & 5.6 (interested parties need to get it from git)
    -> 5.11 becomes branch for first TP
    -> dev is feature development
    -> all first TP release related patches go to 5.9 (and 5.11 subsequently)

22. February 2018

- dev branch is still/again broken in CI, seems like pyside issues
- changes to create automatic snapshots out of CI is still under review
- Numerous small fixes to the doc generation, significantly bringing down the number of documentation warnings
- Massaged build rules to improve built packages
  - Removed unnecessary _utils.py file PYSIDE-600
  - Changed wheel package names to contain both PySide2 version and Qt version against which it was built PYSIDE-613
  - Finally got libclang to be deployed into packages
  - Improved build process not to copy unnecessary symlinks and duplicating libraries when creating wheel (so that you don't get 3x size increase because of 3x number of WebEngine libraries) PYSIDE-495
  - Fixed wheel package names to contain correct minimum deployment target with which macOS was built with (on further analysis, this will need further improvement) PYSIDE-612
  - Fixed rpath handling for QtWebEngineProcess when creating standalone packages on macOS PYSIDE-605
- Added CMake build rules for scriptable application example PYSIDE-597
- Investigated and fixed a heap corruption bug in a test that caused constant crashes on macOS CI runs
- Reduced number of warnings when running shiboken to create user bindings (like scriptable application example) PYSIDE-587
- PYSIDE-595: Making slow progress, still debugging. It's a difficult process, but assumption is that it will work.
- Havent's merge this new patch because we should be 200% sure it's properly working.
- Working on a few bugs PYSIDE-106 PYSIDE-570
- One of the bugs PYSIDE-611 was valid only for 5.6, but I noticed a strange behaviour on 5.9, so I will keep working on it, even after we move 5.6 to cherry pick mode.

15. February 2018

- CI broken, 5.6 & dev branch broken, 5.9 just got working again
- first TP will be 5.11 based (needs 5.11 git branches, 5.6 branch goes into cherry-pick mode, 5.9 branch remains main target)
- more modules added to doc, qt3d & qtquick has some special handling
- snippet work ongoing
- snapshot generation on the agenda
- lots of bug fixing e.g.: PYSIDE-104
- PYSIDE-560

8. February 2018

- Looking through previous EuroPython talks to write a proposal for this year.          
- Working on a few bugs (fixed a couple related to objects references),                 
  - that sadly will open new issues with memory management. A further analysis on this matter could improve the current memory leaks on PySide.                           
- Trying to "organise" a little the typesystem XML files.                               
- Working on PYSIDE-104 that ended up being a general issue on PySide about a proper verification between slots and signals arguments, to find the best match.
- Merged a few build related changes in 5.6
  - Pending change for PYSIDE-593
  - Pending change for PYSIDE-605
  - Merged changes for PYSIDE-603 and PYSIDE-604
- Investigated and attached minimal reproducible example for PYSIDE-585
- Investigated some macOS build related aspects for package release
- 5.6 CI is unblocked, but 5.9 CI is blocked due to qtbase issues (version bumping)
- PYSIDE-560 waiting for view
- PYSIDE-595 work ongoing, not ready yet
- PYSIDE-363  Documentation: Fixed images, fix handling of qdocconf files  that contain several modules. Looking at snippet resolution.
- Problems with Qt3D due to refactoring of Qt3Dextras in 5.10. Pushed  patch suggestion.

1. February 2018

- Continued work on documentation PYSIDE-363, fixed static pages, add more modules, fixed parameters to qdoc, images
- The GettingStarted wiki is getting better, besides a cleaning it a bit
  - moved the topics around to have guidelines by supported platform (a la Qt)
  - have received feedback and trying to add more information
- Addressed bugs:
  - PYSIDE-34
  - PYSIDE-264
  - cleaning a bit Shiboken's XML files.
  - PYSIDE-560
    - ported to 5.9, extended to 18 header files
    - solved unicode problem
    - reduced the patches to be minimal (avoid many #ifdef's, use a verbose macro name if possible)
    - made everything compile but one announced file: "object.h" depends on the heaptypes solution (PYSIDE-595). Removed WIP status,review pending.
  - PYSIDE-595
    - Started experimenting
    - turned out that the signature extension barfed because it could no longer find the type "__name__". The function PyType_Type.tp_new needs to be called to provide the "ht_name" attribute that is used in heaptypes.
- solving the attribution file issues.
- Pending patches to fix libICU issues
- Pending patches to fix OpenSSL issues PYSIDE-599
- Pending patches to deploy libclang library for shiboken
- Improved documentation about OpenSSL dependency
- Pending patches for cleaning up build scripts a bit (logging and refactoring)
- Pending patches to reduce barrage of warnings PYSIDE-587
- Pending patch for adding CMake build rules for the scriptable_application example (useful as a reference on how to use shiboken for custom bindings) PYSIDE-597
- Old 5.6 CI integration issue fix, but a new issue appeared that was fixed for 5.9 and dev, but still blocks 5.6
- Updated the qml basics tutorial

25. January 2018

- Pyside not renamed
- Releasing preparation
  - pip packages created by coin
  - Automated release testing (RTA) (installing pip, launching an example)
  - final selection of release package targets to be define closer to release (depending on current stable ABI work)
- Investigation doc issues when using prefix & shadow builds
- Need to identify prime demos/examples
- Started to rework Getting Started Guide wiki 
- Addressed cmake build issues
- Handling libICU issues (causing linker issues when importing QtCore)
- Continued work on PYSIDE-560

18. January 2018

 - macOS framework fix (should ignore qt4 system headers)
 - another patch on top to fix OpenGL integer types on mac / Linux (GLuint64, GLint64)
 - Fixed so that PySide 5.6/5.9 is not linked against the Python shared / static library (this was wrong because the symbols need to be picked up by the dynamic loader from the python interpreter itself). http://bugreports.qt.io/browse/PYSIDE-522 
 - Completed move of examples from pyside-examples.git to pyside-setup
 - added additional OpenGL2 core example
 - fixed testrunner (these silently failed since forever) https://codereview.qt-project.org/#/c/216732/
 - fixed issues whereby wrong QObject ctors were selected
 - removed webkit from Pyside2 build  
 - talk about Windows specific instructions for Getting Started Guide 
 - Qt3D issues in Qt 5.10 fixes
 - clazy warning fixes
 - Stable ABI work (started to convert implementations) (see PYSIDE-560)
   - requires adoption of HeapTypes
 - QML tutorial updates
 - Doc snapshots being prepared

11. January 2018

- multi pyside packages (not easy to do)
  - unlikely to achieve until TP
  - a package per python release (3.4, 3.5, 3.6, 2.7) 
  - package per platform (win 32bit|64bit, mac, linux)
  - long term task to convert to reduced Python API (to support multi 3x packages)
  => have to cut some targets: one 3x and 2.7 release
  -> create doc howto to encourage custom builds for not covered platform combo 
  -> do we need commercial for release
- pysinstaller is an installer 
  -> deployqt tool not necessary
  -> documentation required stating the option
- rename import options
  - rename via alias possible 
  -> still an issue when looking up particular types but probably manageable   
  - renaming option: Qt4Pyton, QtPy, QtPython, Python4Qt, Qt
- example shift
  - https://codereview.qt-project.org/#/c/215900/
- Basics for doc generation toolchain
- Issues where cmake picks up platform Qt references where it should use a Qt custom install
- Qt3D bindings still under progress
- PYSIDE-34 & PYSIDE-560 
- Documentation
  - minor doc fixes
  - documenting examples

4. January 2018

- multi pyside packages (not easy to do)
- renaming Pyside to "Qt for Python"
  - quite challenging due to implied import and lib changes
  - alias for existing imports would be beneficial
- discussion around existing function test
  -> working assumption is that we ignore added function and check for existing base line only
  -> effort too large to maintain every single new addition
- discussed issues around concurrent existing Qt builds while building pyside (potential cross leakage)
- various fixes to doc generation steps (see PYSIDE-363)
- discussed need for good examples as part of documentation
  -> considered options to cleanup and remove need for submodule

2017

2016

Backlog

The list below represents some short term targets and long term ideas for Pyside2 development. There is no guarantee that this will happen as written down. It is meant as a short reminder and idea collection for future development. In the long term this list is likely to move to https://bugreports.qt.io/browse/PYSIDE.

  1. First time with Pyside (Document how to get started on this wiki)
  2. Testing Infrastructure
    1. What has been done so far?
    2. Qt CI support (PYSIDE-431)
      1. make test
      2. cmake based build system (all repos)
      3. pip installable Pyside
        1. CI has to setup virtual Python environment
        2. Use pip to install/build Pyside (release bundling)
  1. Future changes to PySide
    1. Gaps (modules & API)
      1. Identify the extent of the gaps by porting existing examples from Qt4 to Qt5 (done)
      2. Plugin system (in what form or shape do we need it is uncertain - some failing tests exist already)
      3. Prepare a summary of gaps for other modules
    2. Fundamental Architecture changes
      1. Shiboken
        1. Clang support (PYSIDE-323)
      2. QtQML support (PYSIDE-355)
    3. MyPy (type annotation - fully typed Python API)
    4. General Python features and their adoption in PySide (e.g. Async I/O)
    5. Creator and tooling
      1. Fix QtC debugger to understand Python/C++ mixed internals
    6. Documentation generation
    7. Deployment of apps
    8. Debugging support
  1. Must have tasks for first release
    1. Packaging (e.g. pip deployment? )
    2. Clarify Installer story
    3. Documentation
    4. Product placement decisions
      1. Licensing
      2. Naming
      3. etc
    5. Complete CI coverage