Why not using only Pytest and GitHub Actions for testing and CI ?

[sapetnioc]

I recently had problems with AppVeyor on Capsul 3.0. We also had problems with Travis CI. So I started to look at GitHub actions and, one thing leading to another, I ended up asking myself about testing in general. This questioning leaded to two main questions:

• What testing framework should we use ?
• What CI infrastructure should we use ?

The goal is not to change all the working testing infrastructures of our projects but if we have to work on tests setup for an existing project (such as Capsul) or a new project, what should be the right direction to go.

After a short time of testing and reading, I already think that Pytest and GitHub Actions are a good solution. Since I do not like the idea of spending a lot of time in investigation on this topic, I deliberately chose to ask the question in relation to these technologies: why not using only Pytest and GitHub Actions for our tests and CI ?

It would be too long to detail here all the pros ans cons of various technologies. I am working on an example with Pytest and GitHub Actions on a small demo project. I will present this work on a BrainVISA dev thursday meeting but in the meantime, those interested can start discussing the topic.

Here are the main features I would like to have from a testing and CI infrastructure (and we can have them with Pytest/Github Actions):

• Possibility to launch tests, coverage, flake8, etc. automatically on some GitHub events (e.g. push, pull request, etc.)
• Having badges (for testing, coverage, etc.) included in GitHub README page and corresponding report accessible when clicking on the badge.
• Branches fully taken into account. Possibility to have separate events, tests, badges, reports, etc. for each branch.
• Ability to build and access the reports locally (on my computer without the requirement to commit or push)

[DimitriPapadopoulos]

• Github Actions is certainly simpler, and more robust and reliable than Travis CI - for now. Don't have experience with AppVeyor.
• In addition to pytest, lots of Python projects use tox to set up proper Python dependencies, or even poetry.