Tox Vs Nox

Deciding between them is not choosing between good or bad or even good and better, but between how they work and what fits your project better.

if you want to run tests in Python, you can just run PyTest

Now you may want to somehow encode how to run your test runner to make sure it’s run always the same way.

What tox and Nox sets apart is that they allow you to run that PyTest command line in different kinds of Python environments.

For example, different Python versions with and without optional dependencies and with various versions of dependencies optional or not.

If you want to run your tests on various Python versions, each with the oldest and newest supported versions of your dependencies, that’s two dimensions right there, the jobs steps are run a total of 15 times, five Python versions times three operating systems with tox.

tox and Nox offer you something similar but focused on Python packaging related features.

They can run any command you want, Nut that’s not where they shine.

History

Tox is really old, its first public release hit PyPI in 2010, and this is also the reason why tox is used by many more projects and has more mindshare. It’s been around forever, which makes it the obvious default choice.

Its original author is Holger Krekel, who also started the PyTest project, and a huge rewrite happens in 2021, led by Bernát Gábor, who currently maintains the virtualenv package.

Nox is a lot younger than tox, since in 2018. It was started by Thea Flowers: she needed something more flexible than tox.

Tox - How it works

Tox uses an ini-based declarative domain specific language or DSL, which is the simplest possible configuration format, essentially sections in square brackets and key value pairs separated by equal signs, and on the other hand, Nox uses Python.

Let’s say we want to install the current project into a test virtualenv together with an optional requirement group called tests that contains all dependencies required to run your tests like pytest, and we want to run pytest on Python 3.8 to Python 3.12, which as of recording this video, are the supported Python versions.

It’s only defined by the five Python versions specified in tox’s own version format consisting of the word “py” and the major and minor versions without any dots or any other separation, tox calls this a factor.

Next, we define the base environment that’s always called testenv and that other environments will inherit from.

And the posargs thingy in the commands field is a nice way to allow passing arguments to PyTest while invoking tox simply add them behind a double dash.

You save this file as tox.ini, run tox, and wait.

And a quick side note, do not try to type check your entire code base on all possible Python versions. It’s not useful and often not even possible, but do make sure that you don’t type check your API surface only against the latest version.

This gives us five times two equals 10 environments, and we can list them using tox -l, which is kind of useful to check what’s happening if you get entangled in all those patterns.

And now you can run any of them by passing their full name to tox run -e, or as of tox version 4, by specifying one of the factors with -f.

So tox run -f tests will run tests on all Python versions, but no type checks, while tox run -f py312 will run all environments that use Python 3.12.

The Python version has no special meaning.

Tox commands

tox parses those commands and then runs them using Python’s subprocess module.

Otherwise, tox wouldn’t be cross-platform, and tox works wonderfully on any Python-supported platform, from Windows to FreeBSD.

But it means that shell things like pubs and redirects of output in general, or even conditionals, don’t work in tox.

Nox

What tox calls environments, Nox calls sessions: you write functions and decorate them as sessions with parameters like what Python version you want to run it with. And the function is then run once for each Python version it’s declared for.

When invoked, it gets the session object passed as an argument that you can use to interact with the virtual environment that Nox has created for you, like installing dependencies or running commands.

Python versions are a first class concept in Nox. So when you run Nox —python 3.12, it runs all sessions defined to run under 3.12.

Unlike with tox and its factors, the version does not have to be part of the name of the session, so configuring a session to use Python 3.12 is enough. This can be useful in CI if you want to have one container per Python version.

Secondly, choosing a session by its name without a Python version like tests no-cov here will run it against all versions that the session is configured for, so the session name and its parameterization are two distinct concepts. Speaking of parameterization, Nox can also parameterize session like you might know from pytest, which is really nice for optional dependencies.

Nox has the concept of tags.

Tags allow you to group sessions together, like in this example, to run both the sessions with and without coverage by passing Nox the argument --tags tests.

Now since Nox files are just Python, you can implement your own features because turns out Python is Turing-complete. So we can implement anything in a Nox file.

Now add my build-and-inspect-python-package GitHub action that does exactly what the name says to drive your CI.

Nox does not offer an official plugin interface, and consequently, there isn’t a lot of Nox-specific helpers on PyPI.

Since a tox.ini is a domain-specific language based on the simplest possible file format, and every feature in tox has to be implemented outside of your tox.ini and inside of tox. Therefore, tox has a very powerful plugin interface, and PyPI is full of plugins.

tox is by most metrics the more active project both by the number of PyPI releases and by git commit activity.

However, as far as maintenance goes, Nox is just fine. It has multiple active maintainers.

How do I choose?

With only like two lines of any code, your tox runs can get radically faster by building the package only once into a wheel and then reusing that wheel in all test environments instead of building a source distribution and installing it into each environment.

One wheel is much faster than many sdists and in Nox, you can/must implement this yourself.

So I’m going to say there are two main factors that for me make me use Nox.

The biggest one being, do I need any logic in the tox files that would inevitably lead to running shell scripts or Python scripts from it? Python is a perfectly adequate scripting language.

And before I write an ini file to call a bash script, I just go with Python in the one place.

The second reason is the handling of minimum dependencies: when you have a package that promises that it works with a certain version of a certain package, you have to test it to be sure it’s true.

This only reliably works with tox by adding constraint files to the repo that duplicate information from package metadata and pass them as a dependency into your tox.ini.

And that’s kind of annoying and error prone and currently doesn’t work with tox-uv

Now, with Nox, I can pass the constraint along with the install call, and it works always because it’s just one call to pip or uv pip.