Requirements¶
TLDR
When you
import
a package for the first time in your application code, add its name torequirements.in
in alphabetical order.When you
import
a package for the first time in your test code, that is never imported in your application code, add its name torequirements_dev.in
in alphabetical order.After updating a
.in
file, update the.txt
files withpip-compile; pip-compile requirements_dev.in
. Never edit the.txt
files directly.To update your local environment, run
pip-sync requirements_dev.txt
.
Now that you have a directory layout, you can declare the project’s requirements.
The requirements of applications (not packages) are managed by four files:
requirements.in
names all direct requirements needed in the production environment, i.e. all packagesimport
’ed by the application.If the application is incompatible with older or newer versions of a requirement, use the least specific version specifier possible, for example:
Newer versions:
foo>=1.2
, notfoo>=1.2.3
Older versions:
foo<2
Versions range:
foo>=1.2,<2
requirements_dev.in
names all direct requirements needed exclusively in the development environment, and not in the production environment, e.g.pytest
orpip-tools
itself.This file should include the direct requirements needed in the production environment, by having a first line of
-r requirements.txt
.
requirements.txt
names all direct and indirect requirements needed in the production environment, all locked to specific versions by pip-tools.requirements_dev.txt
names all direct and indirect requirements needed in the development environment, all locked to specific versions bypip-tools
.
This ensures that:
All environments use the same versions of production requirements, to ensure consistent and replicable deployments and to avoid errors or surprises during or after deployment due to differences between versions (e.g. a new version of Django requires upgrading application code).
Different developers and continuous integration use the same versions of development requirements, to avoid test failures due to differences between versions (e.g. a new version of pytest requires upgrading test code, or a new version of flake8 has stricter linting rules).
Get started¶
pip install pip-tools
A common starter requirements.in
for Django is:
dj-database-url
django<5
psycopg2
sentry-sdk
A common starter requirements_dev.in
for linting in Django is:
-r requirements.txt
black
coverage
coveralls
flake8
isort
pip-tools
pre-commit
If not using Django, remove coverage
and add:
pytest
pytest-cov
Add a requirement¶
Add the requirement in alphabetical order to the appropriate .in
file. Then, run:
pip-compile
pip-compile requirements_dev.in
If running pip-compile
introduces unexpected differences, upgrade pip-tools
to the latest version, and check that you are using the same version of Python as for other runs.
See also
psycopg2¶
psycopg2
is recommended for production. However, installing psycopg2
for development can be difficult on operating systems like macOS. In that case, you can:
Put
psycopg2
inrequirements.in
Put
psycopg2-binary
inrequirements_dev.in
Run:
pip install psycopg2-binary
Note
You must keep the locked versions of psycopg2 and psycopg2-binary in sync.
Install requirements¶
In development:
pip-sync requirements_dev.txt
In production:
pip-sync -q --pip-args "--exists-action w"
Upgrade requirements¶
Requirements should be periodically updated, both for security updates and to better distribute the maintenance burden of upgrading versions over time.
Upgrade one requirement, for example:
pip-compile -P requests
pip-compile -P requests requirements_dev.in
Upgrade all requirements:
pip-compile --upgrade
pip-compile --upgrade requirements_dev.in
Linting¶
Continuous integration runs test_requirements.py, which checks whether any requirements are missing or unused.
If a requirement is reported as unused but is required:
Make sure that a related package sets extras correctly. For example, moto has optional dependencies for each AWS service. You must do, for example:
moto[s3]
If the package is optional – for example, it is imported in a
try
andexcept ImportError
block – use theSTANDARD_MAINTENANCE_SCRIPTS_IGNORE
environment variable.If the package is used exclusively outside of application code – for example, as a command in the production environment or in a GitHub workflow, or in an entry point or as a Sphinx extension – use the
STANDARD_MAINTENANCE_SCRIPTS_IGNORE
environment variable.
If the repository uses Pre-commit hooks, and you see errors originating from pip
, it is likely that you need to upgrade pip-tools
in both .pre-commit-config.yaml
and requirements_dev.txt
.