A little background: over the last few months, I have been contributing in open source organization FOSSASIA, where I’m working on a project called BadgeYaY. It is a badge generator with a simple web UI to add data and generate printable badges in PDF. BadgeYay uses Flask which is a Python based micro framework.
I wanted to include a test coverage reporter in my continuous integration flow , which would ideally deploy to Heroku whenever my TravisCI build passes. For that, I decided to use codecov.io for integrating code coverage.
First , let’s have a look what is this “code coverage” thing; then we shall move on how to integrate Codecov with help of Travis CI.
What is Code Coverage ?
Simply put, code coverage is a measurement used to express the degree to which the source code is executed when a test suite runs. A program with higher code coverage means that the test suite has had more of the source code executed when it runs. Thus, it implies that the source code has lesser chances of containing undetected bugs. We use three primary terms to describe source code lines executed.
- hit indicates that the given source code was executed by the test suite.
- partial indicates that the source code was not fully executed by the test suite; there are remaining branches that were not executed.
- miss indicates that the source code was not executed by the test suite.
Coverage is the ratio of hits / (hit + partial + miss). A code base that has 5 lines executed by tests out of 12 total lines will receive a coverage ratio of 41% . I’m not boasting, but as of this writing, BadgeYaY has 100% code coverage! 😎
Code coverage is a like tool for building tools, and is integrated to other such tools like Git and Travis CI. Source: XKCD
How does CodeCov help in Code Coverage ?
Codecov focuses on integration and promoting healthy pull requests. It delivers, or “injects” coverage metrics directly into the modern source code management workflow to promote more code coverage. This especially adds convenience of checking coverage in pull requests where new features and bug fixes commonly occur.
We can change the configuration of how Codecov processes reports and expresses coverage information. Let us see how I configured it to suit BadgeYaY — by integrating it with Travis CI.
Now, Codecov works great with Travis CI by using just one line :
bash < (curl -s https://codecov.io/bash)
BadgeYaY uses Python unit tests for assertion, with the help of Selenium for web browser automation. Basically, that’s used for testing the code. I configured CodeCov in such a way that it produces coverage report based on results of this testing, so I added the following to scripts in travis.yml
“scripts”: {— nosetests app/tests/test.py -v — with-coverage}
Now I created a codecov.yml file which tells the configuration of generated report after code coverage. Here’s the code :
codecov:notify:require_ci_to_pass: yes
coverage:precision: 2round: downrange: “70…100”
status:project: yespatch: yeschanges: no
comment:layout: “reach, diff, flags, files, footer”behavior: defaultrequire_changes: no
Here is some of the code of travis.ymlfrom the project repository of BadgeYaY which integrates codecov after successful build.
Script:- python app/main.py >> log.txt 2>&1 &- nosetts app/tests/test.py -v — with-coverage- python3 -m pyflakesafter_success:- bash < (curl -s https://codecov.io/bash)
The other two scripts are not related to codecov integration, but feel free to look them up if you’re interested.
Once all of this is set up, Codecov’s ready for action. Now, when anyone makes a pull request to BadgeYaY, Codecov will analyze it according to the above configuration and generate a report showing its code coverage.
With that, we have reached the end of our discussion on integrating Travis CI and Codecov into a Python-based project. I wrote this post as a solution to this issue in BadgeYaY project. If you liked this post, consider having a look at my other work on GitHub 🙂.
Sources : codecov.io, Wikipedia
Did you like the read? Medium doesn’t offer partner program in my country―so I ask people to buy me coffee instead.