Updating the repository

This document is intended for contributors who wishes to update or modify this repository.

Setup

Before we get started, let's make sure we're on the same page.

• You are running a 64-bit version of Windows 7 or above
• You have downloaded and installed Visual Studio 2013
• You have downloaded and installed the MSVC 2013 version of Qt
• You have downloaded and installed Python 2.7.9 x86-64
• You have downloaded and extracted sip and PyQt5
• You can run Python by typing python from your command prompt

Downloading developer resources

As we bundle, we'll be using a few snippets of code to test our work and make sure we're on the right track. If you would like to follow along, this is how what you do.

$ git clone https://gist.github.com/042043b9680ca88c225e.git resources

Compilation

Head on over to your terminal and let's get compiling.

Please note that the compilation can take over an hour to finish.

Compiling Sip

# Setup Visual C++ x64
$ "c:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\amd64\vcvars64.bat"
$ cd sip-4.16.5
$ python configure.py
$ nmake
$ nmake install

Compiling PyQt5

$ cd ..\PyQt-gpl-5.4
# This part requires that the Qt binaries are on your PATH
$ set PATH=%PATH%;C:\Qt\5.4\msvc2013_64\bin
$ python configure.py
$ nmake
$ nmake install

When trying to build PyQt5.5.1 if you run into a problem with missing qgeolocation.h the solution according to the mailing list is to use: python configure.py --disable QtPositioning. If you require the functionality other solutions are provided in that thread.

Several minutes later, you should have a working copy of PyQt5 and sip in your Python's site-packages directory.

c:\Python27\Lib\site-packages\PyQt5

Testing to be sure

$ cd ..\resources
$ test1.py

You should be seeing a window with a single clickable button in it.

Bundling

Now for the tricky part.

                             \ 
   ____________               \                         ____________
  |            |               \                       |            |
  |            |                \                      |            |
  | c:\Qt      |                 \                     |            |
  |            |----------------- \ ------------------>|            |
  |____________|                   \                   |            |
                                    \                  |  PyQt5     |
   ____________                      \                 |            |
  |            |                      \                |            |
  |            |                       \               |            |
  | System     |----------------------- \ ------------>|            |
  |            |                         \             |            |
  |____________|                          \            |____________|
                                           \
                                            \

Qt, and therefore PyQt, have a number of requirements on the environment in which it is to be run. Right off the bat, you'll notice that if you run run the same test from above in a new terminal, it will fail.

$ test1.py
ImportError: DLL load failed: The specified module could not be found.

The reason it worked before was because of the PATH environment variable we set up for compilation, remember? The one pointing to the Qt \bin directory? But we can't ask users to first install PyQt5, and then install Qt. We'd like them to install just PyQt5 and be done with it.

And that's exactly what we'll be doing in this part.

Table of contents

1. Bundling binaries
2. Bundling plugins
3. Bundling QML libraries
4. Bundling sip
5. Bundling extras
6. Bundling Visual C++ redistributable

1/6 - Bundling binaries

The first and most obvious fact is that we need to include the Qt binaries into the distribution. PyQt is merely a C-to-Python translator for these binaries and without them it won't do much.

We can prove this by including the binaries to the PATH and running the test again.

$ set PATH=%PATH%;C:\Qt\5.4\msvc2013_64\bin
$ test1.py

The \bin directory is however very large, how do we know which ones to include? I'm glad you asked. In a nutshell, without rigorous testing and assumption making, the safest thing we can do is to include all of them, except for..

• Files ending with "d.dll"
• Files ending with ".pdb"
• Full exclusion list here

Which is only used for development.

Copy binaries

1. Copy everything under C:\Qt\5.4\msvc2013_64\bin
2. To C:\Python27\Lib\site-packages\PyQt5
3. Run clean_binaries.py

$ cd c:\Python27\Lib\site-packages\PyQt5
$ clean_binaries.py
...
Removing .\Qt5Widgetsd.dll
Removing .\Qt5Widgetsd.pdb
Removing .\Qt5WinExtrasd.dll
Removing .\Qt5WinExtrasd.pdb
Removing .\Qt5Xmld.dll
Removing .\Qt5Xmld.pdb
Removing .\Qt5XmlPatternsd.dll
Removing .\Qt5XmlPatternsd.pdb
Finished

Testing

With that out of the way, let's test it. In a new terminal.

$ test1.py

2/6 - Bundling Plugins

Qt ships with a number of plug-ins; one of these plug-ins is the QPA, for Qt Platform Abstraction. Without this, we will not be able to make much use of PyQt5.

We can prove this by temporarily renaming the plugins directory of the original Qt distribution.

Proving the dependency of plugins

1. Rename plugins in C:\Qt\5.4\msvc2013_64
2. To _plugins
3. Run test1.py

$ test1.py
This application failed to start because it could not find or load the Qt platform plugin "windows".

To bundle the plugins

1. Copy plugins from c:\Qt\5.4\msvc2013_64
2. To C:\Python27\Lib\site-packages\PyQt5
3. Run clean_binaries.py to clean up unwanted files.
4. You should now have a directory C:\Python27\Lib\site-packages\PyQt5\plugins
5. Run test1.py

$ test1.py

It's still not working? That's because although we have copied the plugins, Qt doesn't know where to look for them. We can tell it by including a special file called qt.conf, which looks like this.

qt.conf

[Paths]
Prefix = Lib/site-packages/PyQt5
Binaries = Lib/site-packages/PyQt5

This file must be located next to the executable of our program, which in our case is the Python 2.7 interpreter; C:\Python27\python.exe.

1. Save this file
2. To C:\Python27\qt.conf
3. Run tests to confirm that it is now working.

python.exe however is not the only executable we might need. For example, Qt ships with qmlscene.exe which can be used as a standalone executable for testing QML programs. If we run this file, we will get the same error as before.

$ C:\Python27\Lib\site-packages\PyQt5\qmlscene.exe test2.qml
This application failed to start because it could not find or load the Qt platform plugin "windows".

As just mentioned, Qt will look to qt.conf to tell it where to find it's dependencies and qt.conf must be located next to the executable, which in this case is qmlscene.exe.

To fix qmlscene.exe and other executables in this directory

1. Save this file.

[Paths]
Prefix = .
Binaries = .

2. To C:\Python27\Lib\site-packages\PyQt5\qt.conf
3. Re-run test2 to confirm that it is working.

Read more about QPA and Qt.conf here

• http://doc.qt.io/qt-5/qt-conf.html
• http://doc.qt.io/qt-5/qpa.html

3/6 - Bundling QML Libraries

As mentioned above, if we want to be able to use QML, we must include it's libraries. Let's start by confirming that without the original c:\Qt directory, running a QML application currently does not work.

1. Rename c:\Qt
2. To c:\_Qt
3. Run test2

$ C:\Python27\Lib\site-packages\PyQt5\qmlscene.exe test.qml
module "QtQuick" is not installed

To bundle the QML libraries

1. Copy qml from C:\Qt\5.4\msvc2013_64
2. Run clean_binaries.py to clean up development files.

$ cd C:\Python27\Lib\site-packages\PyQt5
$ clean_binaries.py

3. To C:\Python27\Lib\site-packages\PyQt5
4. You should now have a directory called C:\Python27\Lib\site-packages\PyQt5\qml
5. Run test2

$ C:\Python27\Lib\site-packages\PyQt5\qmlscene.exe test2.qml

We can also confirm that the integration of QML and Python works by running test3.py.

4/6 - Bundling sip

Sip is a dependency of PyQt5, we can prove this by temporarily renaming it from the Python site-packages directory.

Prove that sip is a dependency

1. Rename sip.pyd in C:\Python27\Lib\site-packages
2. To _sip.pyd
3. Run test1

$ test1.py
ImportError: No module named sip

Bundling sip

Because we want to be able to have both PyQt4 and PyQt5 on our PYTHONPATH, we must bundle sip within PyQt5.

1. Copy sip.pyd from C:\Python27\Lib\site-packages
2. To C:\Python27\Lib\site-packages\PyQt5
3. In C:\Python27\Lib\site-packages\PyQt5\__init__.py
4. Add this.

import os
import sys
sys.path.insert(0, os.path.dirname(__file__))

4. Run test1 to confirm

$ test1.py

5/6 - Bundling Extras

Apart from including everything Qt, we are also going to want to include the work done by Riverbank Software.

To bundle all extras

1. Download and install the [PyQt5 for Python 3][py3qt5] distribution (you won't need Python 3 installed).
2. Copy the following directories from C:\Python34\Lib\site-packages\PyQt5

examples
include
mkspecs
qsci
sip
translations
uic

3. Into C:\Python27\Lib\site-packages\PyQt5
4. Remove C:\Python27\Lib\site-packages\PyQt5\uic\port_v3 (as it only applies to Python 3)

6/6 - Bundling the Visual C++ Redistributable

The next and final step is coming to us from Visual Studio-land. And that is the Visual C++ re-distributable.

msvcr120.dll
msvcp120.dll

# Located in
C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\redist\x64\Microsoft.VC120.CRT

To prove this, let us temporarily rename this file and run our test.

To prove the dependency on Visual C++ redistributable

1. Rename either msvcr120.dll or msvcp120.dll in C:\Windows\System32
2. To _msvcr120.dll or _msvcp120.dll
3. Run test1

$ test1.py
ImportError: DLL load failed: The specified module could not be found.

4. Restore the names.

To bundle the Visual C++ redistributable

1. Copy msvcr120.dll and msvcp120.dll from C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\redist\x64\Microsoft.VC120.CRT
2. To C:\Python27\Lib\site-packages\PyQt5
3. Run test1

$ test1.py

You should now be able to run the test successfully regardless of the files in your System32 directory.

Testing

Run all tests, make sure they work.

$ test1.py
$ test2.py
$ test3.py

Copy the PyQt5 directory to a new computer, and re-run the tests.

Versioning

In addition to bundling, we'll need to append versioning information to the package so as to keep track of which python-qt5 release is the latest one.

1. Start by copying the original __init__.py
2. To C:\Python27\Lib\site-packages\PyQt5
3. Overwrite existing

Next we'll increment the version number according to our toolchain.

To append version numbers

1. Open C:\Python27\Lib\site-packages\PyQt5\__init__.py in your favorite text editor.
2. Increment the following.

version_info = (0, 2, 0)

The scheme is:

• (0, 2, 0) means fixes
• (0, 2, 0) means updates to the Qt library (such as this).
• (0, 2, 0) means backwards-incompatible changes.

See Versioning for details

Versioning Extras

In addition, we'll also increment the PyQt and Qt versions that were used in the making of this distribution.

pyqt_version_info = (5, 4, 0)
qt_version_info = (5, 4, 0)

The reason we need both is because it's possible that these two differ.

Pushing

Now that we've got a working copy of PyQt5, it's time to update the repository. The first thing we'll need to do is clone the exiting state of the repository.

$ cd c:\
$ mkdir github
$ cd github
$ git clone https://github.com/pyqt/python-qt5.git
$ cd python-qt5

Get rid of the old**

Now we will rip it apart, leaving only the what is needed for GitHub and Travis to function.

1. Remove the c:\github\python-qt5\PyQt5 directory.
2. Remove everything in c:\github\python-qt5\src

In with the new

Now we fill it back up with our newly compiled distribution.

1. Copy PyQt5 from C:\Python27\Lib\site-packages
2. To c:\github\python-qt5
3. Copy the source distributions PyQt-gpl-5.4 and sip-4.16.5
4. Into c:\github\python-qt5\src
5. The GPL requires that we include these.

Test

Before we push it out into the wild, let's make sure it works.

To make sure it works

1. On another machine/environment
2. Install with pip.

$ pip install virtualenv
$ virtualenv test-qt5 --no-site-packages
$ test-qt5\Scripts\activate
(test-qt5) $ pip install c:\github\python-qt5
(test-qt5) $ python c:\github\test1.py

Give it a big push

Assuming everything worked out, we can now push.

$ git add .
$ git commit -m "Updating Qt to 5.4"
$ git push

Upload to PyPI

This potion requires that you have write access to the repository. If you don't, please feel free to push to your own fork and submit a pull-request and I'll ensure it gets uploaded to PyPI as soon as possible!

1. Go to the Releases page on GitHub.
2. Tag the release with it's version name; e.g. 0.2.0
3. From here, Travis will compile and upload the release to PyPI.
4. Monitor here
5. Confirm here

See Releases for examples of previous releases.

And you're done. Thanks for taking the time to update the PyQt5 distribution and happy programming!

Todo

Help wanted.

• Script this guide
• More testing
  • Number of files installed via pip vs. number of original files (they should match)
• Simplify setup.py:get_package_data()
• Update exclusion_list.yaml