Image taken from Sqreen.io
It’s been almost an year since I have been a maintainer for MusicBrainz Picard, a cross-platform multi-lingual desktop app, that allows you to tag your music files via this very cool service called MusicBrainz.
Picard, I’d say is a fairly large python app with about ~35k SLoC. With a python app of such size, come challenges. One of the toughest challenges I faced this last year has been packaging Picard for all the three platforms that it supports, Linux, macOS and Windows after I ported it to Python 3/PyQt5 for my GSoC project. You can read more about that here.
“Freezing” your code is creating an executable file to distribute to end-users, that contains all of your application code as well as the Python interpreter.
The advantage of distributing this way is that your application will “just work”, even if the user doesn’t already have the required version of Python (or any) installed. On Windows, and even on many Linux distributions and OS X, the right version of Python will not already be installed.
Testing my options
Our existing setup used py2exe and py2app to freeze Picard for Windows and macOS respectively. Since they don’t entirely support Python 3 and PyQt5, I was on the lookout for a new freezing tool. I finally settled on PyInstaller after testing waters with cx_Freeze.
PyInstaller — A happy surprise
One thing I’d like to say about PyInstaller — I was absolutely surprised how easy it was to freeze my python application, while putting minimal efforts from my side, chasing mythical dependencies. It supports Python 2 and 3 and all 3 desktop OSes and even allows you to create portable all-in-one binaries for each. How cool is that!
I plan on giving you a small glimpse of how powerful and simple PyInstaller is, and how, coupled with AppVeyor and TravisCI, you can package your python apps for Windows and macOS without even having access to either of them.
Getting started
In this part of the blog, we will be making a PyInstaller spec file and freezing our package. In the next part, we will be looking into TravisCI and AppVeyor for continuous delivery.
Installing PyInstaller
All you need to do is pip install pyinstaller . It is as simple as that. You can either install it globally or in a virtual environment housing your project. The latter is obviously preferable. This will give you access to mainly 2 scripts that we will be using in the rest of this tutorial — pyinstaller and pyi-makespec .
Project info and structure
Let’s start with a very basic structure to introduce PyInstaller and make adjustments from there on, as per our needs.
package_dir├── package│ ├── submodule_bar│ ├── submodule_foo│ ├── __init__.py│ └── main.py├── entry_point.py└── setup.py
The above assume that you have an entry point script called entry_point.py which launches your application. See python-packaging for help on how to package your app.
Now comes the magical part. All you need to do to freeze your app is
pyinstaller entry_point.py -n foobar
It is as simple as that! PyInstaller will automagically figure out all the dependencies, include all the dynamic libraries that need to be loaded and create adistdirectory with the frozen app named **foobar**.
The output should as follows
package_dir├── dist│ └── foobar│ ├── ...│ ├── ...│ ├── ...│ └── foobar├── package│ ├── submodule_bar│ ├── submodule_foo│ ├── __init__.py│ └── main.py├── entry_point.py├── foobar.spec└── setup.py
You can execute your app by launching dist/foobar/foobar (Of course it will be foobar.exe or foobar.app on Windows and macOS respectively.
Portable apps, what is this magic?
Now let’s take things a bit further. What if you want you entire app bundled with all its dependencies as a single portable executable? Simple, just pass the --onefile flag to PyInstaller.
pyinstaller entry_point.py -n foobar --onefile
PyInstaller will output a single portable executable in the dist folder named foobar which can easily be launched. Again, PyInstaller will automagically find and bundle all the dependencies inside that one file!
It can’t surely be that simple? Can it? What if I need to…
Bundle Libraries
PyInstaller supports a lot of major frameworks and libraries out of the box. This includes —
Babel, Django, IPython, matplotlib, numpy, pillow, PyGTK, PyQt4, PyQt5, scipy, sphinx, SQLAlchemy, wxPython and many more.
If your app depends on any of the above libraries, you don’t need to worry about the hassles of including dependent libraries, dlls, hidden imports, packages or anything else for that matter. PyInstaller takes care of everything for you. It inspects your code recursively and figures out all the dependencies.
Add and bundle resources
Resources can be anything, images, icons, textual data, translation strings. There is a very simple recipe to bundle and access your resources. For simplicity, let’s assume all your resources are available inside a directory called resources as below —
package_dir├── package│ ├── submodule_bar│ ├── submodule_foo│ ├── __init__.py│ └── main.py├── resources│ ├── bar.dat│ └── foo.png├── entry_point.py└── setup.py
Bundling the resources
Run pyi-makespec entry_point.py -n foobar --onefile. The pyi-makespec script accepts the same arguments as pyinstaller but instead of actually running PyInstaller, it creates a foobar.spec spec file for you to customise, which can then be called with pyinstaller foobar.spec.
Your foobar.spec file should look something like this —
The spec file is simply a python script albeit with some special callables as shown above. To add resources, you simply need to create an array with a list of tuples —
- The first string specifies the file or files as they are in this system now.
- The second specifies the name of the folder to contain the files at run-time.
A simple script to do the same would be —
You will be adding the above code to the spec file, which should now look like this —
Notice the call to _get_resources()_ in _a.datas_ .
Accessing bundled resources
Quoting from the PyInstaller wiki —
You may need to learn at run-time whether the app is running from source, or is “frozen” (bundled). For example, you might have data files that are normally found based on a module’s
__file__attribute. That will not work when the code is bundled.
The PyInstaller bootloader adds the name
frozento thesysmodule. So the test for “are we bundled?”
To summarise, this is what you need to do to access any resources you have bundled —
Add the following two variables to your utility section —
You can then use this in your entry_point.py as follows —
You can now load your resources in main.py as follows —
Bundle binaries
PyInstaller should automagically bundle any .so or .dll files by inspecting your python module. But in case it fails to do so, it is easy to add them.
Bundling binaries or libraries that your app depends on is pretty much similar to how you would bundle data files.
Assuming the following directory structure —
package_dir├── bin│ ├── bar.so│ ├── bar.dll│ └── bar.dylib├── package│ ├── submodule_bar│ ├── submodule_foo│ ├── __init__.py│ └── main.py├── resources│ ├── bar.dat│ └── foo.png├── entry_point.py└── setup.py
Let’s say your app depends on a shared library bar, and you have binaries available for it for all 3 operating systems.
You might go around including them as follows —
You might ask, what’s the difference between adding a file as a data file or a binary file, well quoting from the PyInstaller Wiki —
Binary files refers to DLLs, dynamic libraries, shared object-files, and such, which PyInstaller is going to search for further binary dependencies. Files like images and PDFs should go into the
datas
So make sure you are adding any dlls or so files as binaries instead of data files.
Freeze a GUI app
You will probably want to pass the --windowed flag to pyinstaller in order to make sure there is no console while opening the App.
Freeze a macOS app
If you are freezing a one-file windowed macOS app you will want to add an additional callable to your spec file like so —
See the PyInstaller-Wiki for more information about these options.
Note: For simple cases, you can also accomplish all the of the above through flags passed to the
pyisntallerorpyi-makespecscripts. See Using Spec Files for more information.
What’s next?
The above recipes should be more than enough for all general use cases. I hope the above provides a basic guide on how to use PyInstaller. For more advanced use cases, you can sift through the PyInstaller Wiki.
If you want to see the above guide in action, you can have a look at the Picard github repo.
In the next part of this blog, we will learn how to make use of AppVeyor and TravisCI along with PyInstaller to bundle our applications.
HALLLLP, I am stuck!
If you find yourself unable to comprehend any part of the guide or have a very particular use-case, leave a comment below, I will be happy to help if I can :)