Analyzing Python Compilers: CPython Vs. Cython Vs. PyPy

Python is among the most used and most loved programming languages of recent times. It is used across many technology areas, including web development, GUI development, data science, AI, ML, and scientific computing. Python is also used widely in universities and colleges to teach programming to students.

The simplicity of python has made even young kids fall in love with
programming. Python is simple, expressive, concise, and offers English-like
syntax, but it is interpreted language and hence a bit slow.

To overcome the slowness of python, developers have created multiple
different compilers to compile python either to other programming languages, to machine code, or to do Just in Time compilation. Some of the popular python compilers include Cython, Nuitka, Brython, PyPy, and Iron Python.

Alright, so where are we headed towards with so many python compilers out there in the wild? Let us dive deep and try to understand what these compilers are trying to achieve.

Python is a programming language. It is not a compiler or interpreter in itself. Any improvements in Python language specifications or new features would result in nothing till the time those are incorporated in the interpreters or compilers.

Talking about CPython, it is the default implementation of Python specifications. It comes with the python distribution itself and interprets the python programs for the machine.

Given that it is the default implementation of Python, whenever Python is
upgraded with new features, language developers upgrade the CPython version as well and include it in the distribution that you download to set up Python on your machine. So, Python and CPython are different yet can be used together. They can work hand in hand and are upgraded together.

Cython is a separate compiler that can understand both Python and C
specifications. It is more like a superset of Python compiler and allows you to work with Python and C together in a single project.

How does this help? Alright, so, Cython compiles the hybrid python and C
code into a very efficient C program which in turn can be compiled to the
machine code using C compilers.

Given the speed gains we get using Cython it is used heavily to develop
python extension modules. Many popular python modules like SciPy are written in Cython itself.

Just to clarify a bit more, the mixed programs that you write in C and
Python can be termed as a new programming language itself. That language is built using very popular Pyrex specifications.

So, while Cython is a compiler in these terms, it has its own language
specifications as well that, of course, comply with the Python specifications.

Cython vs. Python debate gets to the next level because of speed gains too. It is also worth noting that the speed gains from Cython programs are up to 15x compared to the raw python code interpreted using CPython (the default interpreter).

PyPy works on the Just in Time compilation principle. Like interpreters, JIT compilers also pick up the raw code but turns the code into efficient machine code just before the execution.

There are a lot of optimization techniques involved in JIT compilers and
that makes JIT compilers work much faster than the raw interpreters. PyPy also consumes lesser memory at run time and hence has an advantage over raw Python interpreters.

You might want to understand the fact that while PyPy gives performance
gains and memory advantages, it is not always true. JIT compilers have
overheads as the compilers need to load at the run time, and for that reason, if your python execution processes are very small as compared to the JIT compiler overheads, the gains might not be evident.

However, for long-running processes, PyPy speed gains generally overweigh the initial overheads. Developers claim to get speed gains of around 5x any many benchmarking experiments.

While there are multiple compilers, they all try to solve different problems and work differently. Looking at the statistics and usage, Cython seems to be the leader in the game and provides the fastest execution speeds, has a great community, good documentation, and is a potential candidate to get included in the default python distribution itself.

PyPy is a JIT compiler. It will have its own space, improving over the
period of time, and for sure here to stay. You might want to do some
experiments with PyPy and see if it fits your use case.

I hope you liked the article, do let me know your experience working with various Python compilers. Cheers!!