Linting

Before writing any code, set up formatters and linters.

Configuration

New projects should use Black. All projects must use flake8 and isort with line lengths of 119 (the Django standard). If using Black, configure it as follows:

pyproject.toml
[tool.black]
line-length = 119

[tool.isort]
profile = 'black'
line_length = 119
setup.cfg
[flake8]
max-line-length = 119
extend-ignore = E203
pyproject.toml
[build-system]
requires = ["setuptools>=61.2"]
build-backend = "setuptools.build_meta"

[tool.black]
line-length = 119

[tool.isort]
profile = 'black'
line_length = 119

[tool.pydocstyle]
match_dir = '(?!tests).*'
ignore = 'D100,D104,D200,D203,D205,D212,D400,D415'
setup.cfg
[flake8]
max-line-length = 119
extend-ignore = E203

[metadata]
name = {{ cookiecutter.package_name }}
version = 0.0.1
author = Open Contracting Partnership
author_email = data@open-contracting.org
license = BSD
description = {{ cookiecutter.short_description }}
url = https://github.com/open-contracting/{{ cookiecutter.repository_name }}
long_description = file: README.rst
long_description_content_type = text/x-rst
classifiers =
    License :: OSI Approved :: BSD License
{%- if cookiecutter.os_independent == "y" %}
    Operating System :: OS Independent
{%- else %}
    Operating System :: POSIX :: Linux
{%- endif %}
    Programming Language :: Python :: 3.8
    Programming Language :: Python :: 3.9
    Programming Language :: Python :: 3.10
    Programming Language :: Python :: 3.11
    Programming Language :: Python :: 3.12
    Programming Language :: Python :: Implementation :: CPython
{%- if cookiecutter.pypy == "y" %}
    Programming Language :: Python :: Implementation :: PyPy
{%- endif %}

[options]
packages = find:
install_requires =

[options.packages.find]
exclude =
    tests
    tests.*

[options.extras_require]
test =
    coveralls
    pytest
    pytest-cov
docs =
    furo
    sphinx
    sphinx-autobuild
pyproject.toml
[tool.black]
line-length = 119
exclude = '/(migrations|node_modules)/'

[tool.isort]
profile = 'black'
line_length = 119
skip = ['migrations']

[tool.coverage.run]
omit = ['*/migrations/*']
setup.cfg
[flake8]
max-line-length = 119
extend-ignore = E203
exclude = migrations

Repositories should not modify or otherwise use pyproject.toml, setup.cfg, .editorconfig or tool-specific files, except to ignore generated files like database migrations. pyproject.toml is preferred to tool-specific files; if it’s not supported, setup.cfg is preferred – like for Flake8 and Babel.

Maintainers can find and compare configuration files with:

find . \( -name pyproject.toml -or -name setup.cfg -or -name .editorconfig -or -name .coveragerc -or -name .flake8 -or -name .isort.cfg -or -name .pylintrc -or -name pylintrc -or -name pytest.ini \) -not -path '*/node_modules/*' -exec bash -c 'sha=$(shasum {} | cut -d" " -f1); if [[ ! "45342d1e1c767ae5900edbcbde5c030adb30a753 ed723d5329bb74ab24e978c6b0ba6d2095e8fa1e 29418dd6acf27bb182036cf072790cb640f34c9c" =~ $sha ]]; then echo -e "\n\033[0;32m{}\033[0m"; echo $sha; cat {}; fi' \;

Pre-commit hooks

To avoid pushing commits that fail formatting/linting checks, new projects should use pre-commit (add pre-commit to the requirements_dev.in file). For example, if Black is configured as above, create a .pre-commit-config.yaml file:

ci:
  autoupdate_schedule: quarterly
  skip: [pip-compile]
repos:
  - repo: https://github.com/psf/black
    rev: 23.9.1
    hooks:
      - id: black
  - repo: https://github.com/pycqa/flake8
    rev: 6.1.0
    hooks:
      - id: flake8
        additional_dependencies: [flake8-comprehensions]
  - repo: https://github.com/pycqa/isort
    rev: 5.12.0
    hooks:
      - id: isort
  - repo: https://github.com/jazzband/pip-tools
    rev: 7.3.0
    hooks:
      - id: pip-compile
        name: pip-compile requirements.in
        files: ^requirements\.(in|txt)$
      - id: pip-compile
        name: pip-compile requirements_dev.in
        files: ^requirements(_dev)?\.(in|txt)$
        args: [requirements_dev.in]

ci:
  autoupdate_schedule: quarterly
repos:
  - repo: https://github.com/psf/black
    rev: 23.9.1
    hooks:
      - id: black
  - repo: https://github.com/pycqa/flake8
    rev: 6.1.0
    hooks:
      - id: flake8
        additional_dependencies: [flake8-comprehensions]
  - repo: https://github.com/pycqa/isort
    rev: 5.12.0
    hooks:
      - id: isort
  - repo: https://github.com/pycqa/pydocstyle
    rev: 6.3.0
    hooks:
      - id: pydocstyle
        additional_dependencies: [toml]
        files: ^(?!tests)

To ignore generated files, you can add, for example, exclude: /migrations/ to the end of the file.

Note

pre-commit/pre-commit-hooks is not used in the templates, as the errors it covers are rarely encountered.

Skipping linting

isort:skip and noqa comments should be kept to a minimum, and should reference the specific error, to avoid shadowing another error: for example, # noqa: E501.

The errors that are allowed to be ignored are:

  • E501 line too long for long strings, especially URLs

  • F401 module imported but unused in a library’s top-level __init__.py file

  • E402 module level import not at top of file in a Django project’s asgi.py file

  • W291 Trailing whitespace in tests relating to trailing whitespace

  • isort:skip if sys.path needs to be changed before an import

Maintainers can find unwanted comments with this regular expression:

# noqa(?!(: (E501|F401|E402|W291)| isort:skip)\n)

Continuous integration

Create a .github/workflows/lint.yml file. The Django and Pypackage Cookiecutter templates contain default workflows.

See also

Optional linting

Note

This section is provided for reference.

codespell finds typographical errors. It is especially useful in repositories with lengthy documentation. Otherwise, all repositories can be periodically checked with:

codespell -S '.git,.pytest_cache,cassettes,fixtures,_build,build,dist,target,locale,locales,vendor,node_modules,docson,htmlcov,redmine,schemaspy,*.csv,*.json,*.jsonl,*.map,*.po,european-union-support'

flake8’s --max-complexity option (provided by mccabe) is deactivated by default. A threshold of 10 or 15 is recommended:

flake8 . --max-line-length 119 --max-complexity 10

Note

Complexity is best measured by the effort required to read and modify code. This cannot be measured using techniques like cyclomatic complexity. Reducing cyclomatic complexity typically means extracting single-caller methods and/or using object-oriented programming, which frequently increase cognitive complexity. See the note under Create products sustainably.

pylint and pylint-django provides useful, but noisy, feedback:

pip install pylint
pylint --max-line-length 119 directory

The Python Code Quality Authority maintains flake8 (which includes mccabe, pycodestyle and pyflakes), isort and pylint.