How to split a Python requirements file?

More often than not you’ll only need a single requirements.txt file. But in certain cases where you have continuous integration or unit tests, you may not need to install every package in the remote test runtime. The common example are plotting and debugging packages.

You can easily merge and split requirements using the following setup:

# ci-requirements.txt
numpy

# ---

# dev-requirements.txt
matplotlib
jupyterlab

# ---
# requirements.txt
-r ci-requirements.txt
-r dev-requirements.txt

Using -r filename, we can incorporate other requirements file into the main one and use each one separately. So to develop you would install all the requirements but for continuous integration you would do pip install -r ci-requirements.txt installing only what is necessary.