I had the chance to read an early version of “Hypermodern Python Tooling” by Claudio Jolowicz, which is set to be released in April 2024. I read the June 2023 version and was motivated to write a review since I haven't found a book about this subject previously.
The topics covered in the book were previously only available in conference videos, technical articles, and tutorials, depending on who is more comfortable with what type of content. In my opinion, other sources, if there is any attention paid to these issues, it is very superficial. It usually ends up being various variations of installing the Python interpreter itself.
As of June 2023, the early access book has 7 sections:
- Installing Python;
- Environments in Python;
- Packages in Python;
- Dependency Management;
- Tracking with Poetry;
- Testing with Pytest;
- Measuring coverage with Coverage.py
Overview of the book sections
The first section describes the release cycle of Python versioning. It describes different installation methods for different operating systems: Windows, Linux, macOS, the use of pyenv to install many versions of Python interpreters, not only CPython-based, but also PyPy, IronPython, Jython, Pyston, and others. An example of using pylauncher, also for Unix systems, is given. Alternatively, an example of installation using Anaconda is given.
The second section discusses how the Python environment is organized: interpreter, entry point scripts, modules (including both the standard library and third-party modules), and shared libraries (containing custom code compiled from low-level languages such as C). The use of virtual environments, which can be created using the built-in venv module introduced in Python3, is mentioned, as well as the differences in enabling them for different operating systems. The package manager is introduced, both the built-in pip and the third-party pipx. General information on how to search for modules is given.
The third section introduces the Python package index. We give an example of a simple script and show the whole workflow from creating a project to loading it into both the test and real package indexes. The pyproject.toml file is used to specify meta information when building an application package. A modern and standards-compliant project manager hatch (https://hatch.pypa.io/latest/) is used as the build backend. The official PyPI upload tool: Twine is used as the package upload tool. Package options are listed: sdists and wheels. A list of the most essential fields with meta information in the pyproject.toml file for a project is given.
The fourth sectionraises probably the most ambiguous problem: the problem of dependency management in a project. There are two options: requirements.txt with possible variations and combinations, and pyproject.toml. Surprisingly, there won't be a story about poetry here, but its description won't take long in the fifth section. It was quite interesting to learn about the existence of a tool for generating the so-called dependency locus using the pip-tools tool.
The fifth sectiondeals with the well-known poetry, which provides an automated tool for routine tasks: dependency management, creating a package and publishing it in the package index, etc. The differences to pyproject are described. We describe the differences in pyproject.toml from what was described in the previous chapters when we familiarized ourselves with its syntax. There are alternatives to poetry in the Python ecosystem, such as hatch and maturin.
Section six describes a short kmb (young fighter course) on testing with refactoring elements for Python developers, mentioning two testing frameworks: unittest and pytest. These are the basics, but other sources with more detailed descriptions of the topic are also provided for diving in. The only thing not mentioned is nose, but in practice pytest has been the de facto leader in recent years. In the case of dependency constraints, the choice is obvious: unittest. An example of mini-refactoring code to create a more flexible basis for future code support and simplify testing is given.
In the seventh section, we discuss the issue of test coverage using the standard trace module as well as the third-party coverage module. But it is still recommended to use the third-party module, which can be run on different versions of the Python interpreter, and then get a combined report on code coverage by following the rule coverage run → coverage combine → coverage report. Eventually, everyone faces a dilemma at some point on a project: “To test, you need to refactor the code; but refactoring without tests is too risky”.
Here, most likely, everything will depend on the project itself, its scope, the quality of the code itself and the availability of tests as such. It seems to me that you should take a pragmatic way, and you should not always achieve a reference 100%, it is often not required. Most likely, at the time of reading this, the chapter with automating the whole story using the Nox tool was not ready yet.
Before summarizing, let's install pipx and try to use it.
PIPX in action
On the official website, the developers of the tool position it like this:
“Overview: What is pipx?
pipx is a tool to help you install and run end-user applications written in Python. It's roughly similar to macOS's brew, JavaScript's npx, and Linux's apt.
It's closely related to pip. In fact, it uses pip, but is focused on installing and managing Python packages that can be run from the command line directly as applications.”
It's great news if the package also comes with a CLIinterface, as it will be possible to run the desired package directly, with a single command, without pre-installation. Hello to the npx tool from the NodeJS world ;)
After installing pipx locally, let's run 5 small projects to see how fast and easy it is.
- Utility for generating audio files from google-speech text
In the console, run the following command: $ pipx run google-speech -l en -o hello_world.mp3 "Hello, world!"
In this case, the file hello_world.mp3 is created in the directory where the command was executed, and can be listened to with a media player.
- Utility for calculating checksums of files and directories chksum-cli
In the console, run the following command: $ pipx run chksum-cli ./hello_world.mp3 b754e299a96327caaddd44b453f66ed9439c5d64d0d6e55c7e32471115757e5e sha256
The result will be displayed whether the checksums of the files match or not.
- Utility for sending requests via HTTP protocol httpy-cli
In the console, run the following command: $ pipx run httpy-cli mail.ru
An HTTP response with headers and request body will be output.
- A utility to display a pseudo-graphic clock in the console with the ability to set a timy-cli timer
In the console, run the following command: $ pipx run timy-cli -с
An analog clock will be displayed, with the ability to stop it using the CTRL + C key combination.
- Console chess game with the possibility of network play cli-chess
In the console, run the following command: $ pipx run cli-chess
A menu for selecting the game type will start:
To view additional options, you can add the --help key to the commands above.
Impressions of the book
Overall, the book is easy to read, but a bit confusing in places. Perhaps it lacks some non-trivial examples. The book describes a sample script to retrieve a random article from Wikipedia.
What I liked:
Personally, I discovered the pipx tool (analogous to npx in the Node.js world), which is very handy when you need to quickly familiarize yourself with something and take a look. Especially good for projects where there is a cli interface to run;
It was interesting to learn that there are alternatives to poetry (https://python-poetry.org/), such as hatch (https://hatch.pypa.io/latest/) and maturin (https://www.maturin.rs/);
Example of using backports for different versions of the Python interpreter; for example, importlib_metadata is used as a backport for Python3.7, as opposed to importlib.metadata for Python3.8. Sample code from the book:
try:
from importlib.metadata import metadata
except ImportError:
from importlib_metadata import metadata
What I think is missing from the book:
Not covered is the option of using a package manager to install Python on the Windows family of operating systems, such as chocolatey (https://chocolatey.org/);
The way to build Python from source for Unix distributions is not covered;
No way to run Python in a Docker container. There are many builds now, starting from the most minimalist one based on alpine;
Not considered the way to bring the project to run the executable .exe file in Windows (in my practice there was such a case. Yes, I agree that it is a very niche task, but still);
No example of alternative project distribution, e.g., using the standard package manager apt for Debian-like distributions with the help of setuptools and dh-virtualenv with *.deb package formation);
No chapter describing linters like pylint, flake8, isort, black, mypy, pyright, etc.
But this is my subjective opinion, you may disagree.
Despite the fact that the book is not yet in its final version (scheduled for release in April 2024), I can recommend it to anyone interested in Python.
Author: Daniil Nikitin, Senior Software Developer at Usetech