Django#

Use the Django Cookiecutter template, and add the repository to pre-commit.ci:

pip install cookiecutter
cookiecutter gh:open-contracting/software-development-handbook --directory cookiecutter-django

For the stefanzweifel/git-auto-commit-action action, add the repository to the Robots team, and set the Permission level to “Admin”.

Directory layout#

  • Maintain the distinction between app directories and the project directory.

  • Organize apps into logical units. Don’t mix everything into one app.

  • Do not nest app directories within the project directory. (While this avoids errors due to app names colliding with package names, it tends to produce a worse separation of concerns.)

  • Delete empty files auto-generated by python manage.py startapp.

Filename conventions#

  • Use core as the project name.

  • Use either nouns (like exporter) or verbs (like export) for apps. Don’t use both.

Model Template View#

The view should interact with the models and return a context for the template, based on the request.

  • A view is concerned with fulfilling the request. Add new methods to models for complex or repeated processing, instead of putting that logic in the view.

  • A template is concerned with formatting the context provided by the view. Use custom template tags and filters for complex or repeated formatting, instead of putting that logic in the view.

  • A template should not perform user-based logic, like filtering which model instances to display. Instead, use a custom manager (or custom queryset).

  • A model should not concern itself with other models’ objects or with the filesystem.

URLs#

Sitemap#

Public sites should serve a sitemap.xml file.

Models#

  • Use from django.db import models, as convention.

  • Use help_text and verbose_name to describe fields.

  • Use TextField, not CharField. There is no performance difference in PostgreSQL.

  • Do not use null=True with TextField or CharField, as recommended.

  • Do not use null=True with JSONField, if possible. Instead, use default=dict, default=list or default="".

  • Use the pk property and the pk lookup shortcut instead of id.

  • The table related to a ManyToManyField field is not visible to the Django ORM. If you need to operate on it, create an explicit model with foreign keys to the other models, instead of operating on it via the other models.

Forms#

  • Use help_text and label to describe fields.

Views#

Templates#

See also

HTML and CSS

  • If an inclusion tag contains no logic other than returning a context, use an include tag instead.

  • In many cases, you can achieve the same outcome using either context processors or inclusion tags. If the variables that the template uses are constant (e.g. from a Django settings file), use a context processor. Otherwise, use an inclusion tag.

Admin#

  • Configure list views for easy of use using list_display, list_editable, list_filter

  • Configure fieldsets (or fields if there are only a few) to group and order fields logically

  • Configure readonly_fields, so that the administrator knows whether to edit a field

Settings#

To simplify the configuration of Django projects, use the template below for the settings file.

In other modules, import settings from django.conf, as recommended:

from django.conf import settings

See also

Settings guide, for the general approach to configuration

Environment variables#

DJANGO_ENV=production

Sets DEBUG = False. Sets HTTPS-related settings, if LOCAL_ACCESS is not set and ALLOWED_HOSTS is set.

LOCAL_ACCESS

If set, HTTPS-related settings are not set.

DJANGO_PROXY

If set, proxy-related settings are set. This requires the web server to be properly configured (see the warning about SECURE_PROXY_SSL_HEADER). For example:

RequestHeader unset X-Forwarded-Proto
RequestHeader set X-Forwarded-Proto https env=HTTPS

ProxyPass / http://127.0.0.1:8000/
ProxyPassReverse / http://127.0.0.1:8000/
ALLOWED_HOSTS

Set to a comma-separated list of host names. Localhost connections are always allowed.

SECRET_KEY

Set to:

python manage.py shell -c 'from django.core.management import utils; print(utils.get_random_secret_key())'
DATABASE_URL

Set according to dj-database-url’s documentation.

SECURE_HSTS_SECONDS

Set according to Django’s documentation.

SENTRY_DSN

Set to the project’s client key (DSN) from Sentry.

FATHOM_ANALYTICS_ID (and FATHOM_ANALYTICS_DOMAIN)

Set to the site’s ID (and custom domain) from Fathom Analytics. Remember to configure your project to render the embed code (example).

Using the template#

SECRET_KEY

Replace !!!SECRET_KEY!!! with:

python manage.py shell -c 'from django.core.management import utils; print(utils.get_random_secret_key())'
INSTALLED_APPS

Do not enable more applications than necessary. Among the default applications:

django.contrib.admin (tutorial)

Remove, unless using the Django admin (check for occurrences of admin).

django.contrib.auth (topic)

Remove, unless using django.contrib.admin or authenticated users (check for occurrences of auth or user).

django.contrib.messages

Remove, unless using django.contrib.admin or one-time messages (check for occurrences of messages).

django.contrib.contenttypes

Remove, unless using django.contrib.admin, django.contrib.auth or otherwise dependent.

django.contrib.sessions (topic)

Remove, unless using django.contrib.admin, django.contrib.auth, django.contrib.messages or anonymous sessions (check for occurrences of session).

django.contrib.staticfiles (how-to)

Remove, unless the project contains static files.

django.contrib.sitemaps

Remove, if the application is private. (Added by the Cookiecutter template.)

Then, make any corresponding changes to urls.py, and MIDDLEWARE, TEMPLATES, STATIC_URL and AUTH_PASSWORD_VALIDATORS in settings.py.

DATABASES
LOGGING

See Logging.

Add additional settings for:

  • Django under # Project-specific Django configuration

  • Dependencies under # Dependency configuration

  • Your project under # Project configuration

Template#

This template is based on the default settings.py file. You can also refer to the default Django settings. Replace core with the project’s module name and remove the Jinja syntax if not using the Cookiecutter template:

"""
Django settings for the project.

Generated by 'django-admin startproject' using Django 4.2.2.

For more information on this file, see
https://docs.djangoproject.com/en/4.2/topics/settings/

For the full list of settings and their values, see
https://docs.djangoproject.com/en/4.2/ref/settings/
"""

import os
from glob import glob
from pathlib import Path

import dj_database_url
import sentry_sdk
from sentry_sdk.integrations.django import DjangoIntegration
from sentry_sdk.integrations.logging import ignore_logger

production = os.getenv("DJANGO_ENV") == "production"
local_access = "LOCAL_ACCESS" in os.environ or "ALLOWED_HOSTS" not in os.environ

# Build paths inside the project like this: BASE_DIR / 'subdir'.
BASE_DIR = Path(__file__).resolve().parent.parent


# Quick-start development settings - unsuitable for production
# See https://docs.djangoproject.com/en/4.2/howto/deployment/checklist/

# SECURITY WARNING: keep the secret key used in production secret!
SECRET_KEY = os.getenv("SECRET_KEY", "!!!SECRET_KEY!!!")

# SECURITY WARNING: don't run with debug turned on in production!
DEBUG = not production

ALLOWED_HOSTS = [".localhost", "127.0.0.1", "[::1]", "0.0.0.0"]
if "ALLOWED_HOSTS" in os.environ:
    ALLOWED_HOSTS.extend(os.getenv("ALLOWED_HOSTS").split(","))


# Application definition

INSTALLED_APPS = [
    "django.contrib.admin",
    "django.contrib.auth",
    "django.contrib.contenttypes",
    "django.contrib.sessions",
    "django.contrib.messages",
    "django.contrib.staticfiles",
    "django.contrib.sitemaps",
]

MIDDLEWARE = [
    "django.middleware.security.SecurityMiddleware",
    "django.contrib.sessions.middleware.SessionMiddleware",
    "django.middleware.locale.LocaleMiddleware",
    "django.middleware.common.CommonMiddleware",
    "django.middleware.csrf.CsrfViewMiddleware",
    "django.contrib.auth.middleware.AuthenticationMiddleware",
    "django.contrib.messages.middleware.MessageMiddleware",
    "django.middleware.clickjacking.XFrameOptionsMiddleware",
]

ROOT_URLCONF = "core.urls"

TEMPLATES = [
    {
        "BACKEND": "django.template.backends.django.DjangoTemplates",
        "DIRS": [],
        "APP_DIRS": True,
        "OPTIONS": {
            "context_processors": [
                "django.template.context_processors.debug",
                "django.template.context_processors.request",
                "django.template.context_processors.i18n",
                "django.contrib.auth.context_processors.auth",
                "django.contrib.messages.context_processors.messages",
{%- if cookiecutter.use_fathom == "y" %}
                "core.context_processors.from_settings",
{%- endif %}
            ],
        },
    },
]

WSGI_APPLICATION = "core.wsgi.application"


# Database
# https://docs.djangoproject.com/en/4.2/ref/settings/#databases

DATABASES = {
    "default": dj_database_url.config(default="postgresql:///{{ cookiecutter.database_name }}?application_name={{ cookiecutter.application_name }}")
}


# Password validation
# https://docs.djangoproject.com/en/4.2/ref/settings/#auth-password-validators

AUTH_PASSWORD_VALIDATORS = [
    {
        "NAME": "django.contrib.auth.password_validation.UserAttributeSimilarityValidator",
    },
    {
        "NAME": "django.contrib.auth.password_validation.MinimumLengthValidator",
    },
    {
        "NAME": "django.contrib.auth.password_validation.CommonPasswordValidator",
    },
    {
        "NAME": "django.contrib.auth.password_validation.NumericPasswordValidator",
    },
]


# Internationalization
# https://docs.djangoproject.com/en/4.2/topics/i18n/

LANGUAGE_CODE = "en-us"

TIME_ZONE = "UTC"

USE_I18N = True

USE_TZ = True


# Static files (CSS, JavaScript, Images)
# https://docs.djangoproject.com/en/4.2/howto/static-files/

STATIC_URL = "static/"

# Default primary key field type
# https://docs.djangoproject.com/en/4.2/ref/settings/#default-auto-field

DEFAULT_AUTO_FIELD = "django.db.models.BigAutoField"


# Project-specific Django configuration

LOCALE_PATHS = glob(str(BASE_DIR / "**" / "locale"))

STATIC_ROOT = BASE_DIR / "static"

# https://docs.djangoproject.com/en/4.2/topics/logging/#django-security
LOGGING = {
    "version": 1,
    "disable_existing_loggers": False,
    "formatters": {
        "console": {
            "format": "%(asctime)s %(levelname)s [%(name)s:%(lineno)s] %(message)s",
        },
    },
    "handlers": {
        "console": {
            "class": "logging.StreamHandler",
            "formatter": "console",
        },
        "null": {
            "class": "logging.NullHandler",
        },
    },
    "loggers": {
        "": {
            "handlers": ["console"],
            "level": "INFO",
        },
        "django.security.DisallowedHost": {
            "handlers": ["null"],
            "propagate": False,
        },
    },
}

# https://docs.djangoproject.com/en/4.2/howto/deployment/checklist/
if production and not local_access:
    # Run: env DJANGO_ENV=production SECURE_HSTS_SECONDS=1 ./manage.py check --deploy
    CSRF_COOKIE_SECURE = True
    SESSION_COOKIE_SECURE = True
    SECURE_SSL_REDIRECT = True
    SECURE_REFERRER_POLICY = "same-origin"  # default in Django >= 3.1

    # https://docs.djangoproject.com/en/4.2/ref/middleware/#http-strict-transport-security
    if "SECURE_HSTS_SECONDS" in os.environ:
        SECURE_HSTS_SECONDS = int(os.getenv("SECURE_HSTS_SECONDS"))
        SECURE_HSTS_INCLUDE_SUBDOMAINS = True
        SECURE_HSTS_PRELOAD = True

# https://docs.djangoproject.com/en/4.2/ref/settings/#secure-proxy-ssl-header
if "DJANGO_PROXY" in os.environ:
    USE_X_FORWARDED_HOST = True
    SECURE_PROXY_SSL_HEADER = ("HTTP_X_FORWARDED_PROTO", "https")


# Dependency configuration

if "SENTRY_DSN" in os.environ:
    # https://docs.sentry.io/platforms/python/logging/#ignoring-a-logger
    ignore_logger("django.security.DisallowedHost")
    sentry_sdk.init(
        dsn=os.getenv("SENTRY_DSN"),
        integrations=[DjangoIntegration()],
        traces_sample_rate=0,  # The Sentry plan does not include Performance.
    )


# Project configuration

FATHOM = {
    "domain": os.getenv("FATHOM_ANALYTICS_DOMAIN") or "cdn.usefathom.com",
    "id": os.getenv("FATHOM_ANALYTICS_ID"),
}

See also

The LANGUAGE_CODE is en-us. See Internationalization (i18n) for details.

Performance#

In order of importance:

  • Reduce the number of SQL queries (avoid N+1 queries):

    • Avoid queries inside a loop. For SELECT, perform a single query before the loop (or do the work in batches). For INSERT, use the bulk_create method after the loop (or do the work in batches).

    • Use select_related to reduce the number of queries on ForeignKey or OneToOneField relations.

    • Use prefetch_related to reduce the number of queries on ManyToManyField and reverse ForeignKey relations.

    • Use count if you only need to count. However, if you also need to use the result of the queryset, use len().

    • Use assertNumQueries in tests.

    • Set the django logger’s level to DEBUG in development to add SQL queries to the runserver output.

  • Cache results:

  • Optimize queries:

    • Add indices to fields that are frequently used for filtering. To find slow queries, you can use the PostgreSQL log in production or the SQL panel of Django Debug Toolbar in development.

    • Use the update_fields argument to the save() method for frequent operations.

  • Minimize memory usage:

    • Use iterator when iterating over a queryset whose result is only accessed this once.

Read:

Deployment#

To perform deployment checks locally, run:

env DJANGO_ENV=production ALLOWED_HOSTS=example.com SECURE_HSTS_SECONDS=1 ./manage.py check --deploy --fail-level WARNING

Static files#

DO NOT commit the files generated by collectstatic. These commands are either run during deployment or when creating Docker images.

Use ManifestStaticFilesStorage for cache-busting.

Generated files#

DO NOT commit the files generated by compilemessages. These commands are either run during deployment or when creating Docker images.