The Problem
With the ubiquitous use of microservices, docker, k8s, and other containerization technologies, I noticed an interesting change. Microservices - although they can be written in different programming languages, use different databases, cache systems, and other backend technologies, they tend to be uniform. This is a very important and correct step since such services are easy to manage, deploy and maintain.
Take this stack, for example: Python, FastApi, Sqlalchemy, Postgres, Docker, Github Actions. As you can see, we have not the smallest list of technologies used, and we will just remember that for production we need more linters, security checkers, tests. If all this is set separately, then it can take quite a long amount of time, plus do not forget that when repeating routine actions, errors and misconfigurations occur. But there is a great way to avoid routine configuration: standardization and templating of repositories.
The first solution I came across was the Github Template.
Github Template is a built-in feature in Github that allows you to make any available repository a template and then use the template as an init commit when creating a new repository.
The main advantage is that we got rid of the complete configuration of a new empty repository, plus if we write a simple application in a separate branch, we can also check the full performance and configure CI \ CD using Github Actions. Which already saves us from almost all unnecessary actions.
But here it is a corner case - what if I want to deploy to Kubernetes to new namespaces, as well as different branches in different namespaces with different parameters, and I can do the second one using CI \ CD, but I will need to explicitly specify the namespaces.
This implies one drawback: all these parameters will have to be prescribed by hand.
Since we are automating the process, we must automate it completely.
I remember the excellent template engine Jinja2 - there was a great idea to replace them with placeholders with the name of the variables and replace them with the desired values. But after a bit of searching, I found a utility like Cookiecutter.
Cookiecutter
A command-line utility that creates projects from cookiecutters (project templates), e.g. creating a Python package project from a Python package project template.
It is based on Jinja2, which allows you to make modifications to paths, file names and file contents by replacing placeholders with values.
Features:
- Cross-platform: Windows, Mac, and Linux are officially supported.
- You don’t have to know/write Python code to use Cookiecutter
- Works with Python 2.7, 3.5, 3.6, 3.7, 3.8 ,PyPy and PyPy3.
- Project templates can be in any programming language or markup format:
- Python, JavaScript, Ruby, CoffeeScript, RST, Markdown, CSS, HTML, you name it.
- You can use multiple languages in the same project template.
Installation:
pip3 install cookiecutterUseful Links:
List of ready-made cookiecutters:
http://cookiecutter-templates.sebastianruml.name/
Cookiecutter Demo
Simplest usage:
python3 -m cookiecutter "git@github.com: audreyr / cookiecutter-pypackage.git"Allows you to create a project using a template from the repository. Archives are also supported.
A notable feature was the use of the interactive mode. very conveniently allows you to get acquainted with the new template through the initial configuration. There is also a configuration file mode, which will speed up the cookiecutter itself, useful for quick deployments and when you are familiar with the whole project.
The main magic lies in the cookiecutter.json file. It stores various variables that are needed to generate a new project, and also contains default values.
In addition to project templates, you can also template individual files, for example, a good approach for writing a CI \ CD template where you can select the necessary predefined stages.
In order to generate a new resource using a template, you need to:
- Install Cookiecutter in a convenient way, for example using pip: pip install cookiecutter;
- In the directory where you want to create a new project, execute cookiecutter <path to template>. The path to the template can be either a path to the file system or a link to a repository or zip archive, for more details see the documentation;
- Respond to requests for variable values from cookiecutter.json and, as a result, get the generated project in the current directory;
Combined option.
It's a great option to use both approaches at the same time. The main benefit will be branching available with git, and having a template will allow you to create a single repository for a specific language. The branches will become a short designation of the template, for example, python3.7-fastapi-sqlalchemy-postgres, and the contents of the branches will become cookiecutter templates.
This approach will allow you to store all templates in one place and allows you to customize each in more detail, plus we can attach the security check as a weekly cron job and learn about outdated or unsafe dependencies before the next project is created. and will also serve as an excellent signal to update existing projects.