Django¶

Use the Django Cookiecutter template:

uv tool install cookiecutter
cookiecutter gh:open-contracting/software-development-handbook --directory cookiecutter-django

uv venv
uv pip sync requirements_dev.txt

uv tool install pre-commit
pre-commit install

1. Add the repository to pre-commit.ci 1. Add the repository to the Robots team, and set the Permission level to “Admin” (for the stefanzweifel/git-auto-commit-action action in the lint workflow).

Directory layout¶

Maintain the distinction between app directories and the project directory.
Organize apps into logical units. Don’t mix everything into one app.
Do not nest app directories within the project directory. (While this avoids errors due to app names colliding with package names, it tends to produce a worse separation of concerns.)
Delete empty files auto-generated by python manage.py startapp.

See also

Directory layout guide

Filename conventions¶

Use core as the project name.
Use either nouns (like exporter) or verbs (like export) for apps. Don’t use both.

Model Template View¶

The view should interact with the models and return a context for the template, based on the request.

A view is concerned with fulfilling the request. Add new methods to models for complex or repeated processing, instead of putting that logic in the view.
A template is concerned with formatting the context provided by the view. Use custom template tags and filters for complex or repeated formatting, instead of putting that logic in the view.
A template should not perform user-based logic, like filtering which model instances to display. Instead, use a custom manager (or custom queryset).
A model should not concern itself with other models’ objects or with the filesystem.

URLs¶

Use hyphens as separators in paths.
Use hyphens as separators in named URL patterns.
Use Django REST Framework, instead of writing endpoints by hand. (See Preferred packages.)

Sitemap¶

Public sites should serve a sitemap.xml file.

Do not set changefreq or priority. (“Google ignores priority and changefreq values.”)
Do not use django.contrib.sitemaps.ping_google(). (“Sitemaps ping endpoint is going away.”)

Models¶

Use from django.db import models, as convention.
Use help_text and verbose_name to describe fields.
Use TextField, not CharField. There is no performance difference in PostgreSQL.
Do not use null=True with TextField or CharField, as recommended.
Do not use null=True with JSONField, if possible. Instead, use default=dict, default=list or default="".
Use Meta.indexes, not db_index=True, with ForeignKey, as recommended.
Use the pk property and the pk lookup shortcut instead of id.
The table related to a ManyToManyField field is not visible to the Django ORM. If you need to operate on it, create an explicit model with foreign keys to the other models, instead of operating on it via the other models.

See also

Forms¶

Use help_text and label to describe fields.

Views¶

Avoid setting cookies and using sessions, where possible.
- Set the user’s language in the URL path, using the i18n_patterns function.
- Edit the set_language view to not use CSRF protection.

Templates¶

See also

HTML and CSS

If an inclusion tag contains no logic other than returning a context, use an include tag instead.
In many cases, you can achieve the same outcome using either context processors or inclusion tags. If the variables that the template uses are constant (e.g. from a Django settings file), use a context processor. Otherwise, use an inclusion tag.

Admin¶

Configure list views for easy of use using list_display, list_editable, list_filter
Configure fieldsets (or fields if there are only a few) to group and order fields logically
Configure readonly_fields, so that the administrator knows whether to edit a field

Management commands¶

Use self.stdout and self.stderr to write output for the user.

Note

If implementing a worker (i.e. daemon), use Python logging, instead.
Remember to add __init__.py files to the management and management/commands directories within app directories. Otherwise, their coverage won’t be calculated.

Settings¶

To simplify the configuration of Django projects, use the template below for the settings file.

In other modules, import settings from django.conf, as recommended:

from django.conf import settings

See also

Settings guide, for the general approach to configuration

Environment variables¶

DJANGO_ENV=production

Sets DEBUG = False. Sets HTTPS-related settings, if LOCAL_ACCESS is not set and ALLOWED_HOSTS is set. If using the Django Cookiecutter template, the Dockerfile sets it.

LOCAL_ACCESS

If set, HTTPS-related settings are not set.

DJANGO_PROXY

If set, proxy-related settings are set. This requires the web server to be properly configured (see the warning about SECURE_PROXY_SSL_HEADER). For example:

RequestHeader unset X-Forwarded-Proto
RequestHeader set X-Forwarded-Proto https env=HTTPS

ProxyPass / http://127.0.0.1:8000/
ProxyPassReverse / http://127.0.0.1:8000/

ALLOWED_HOSTS

Set to a comma-separated list of host names. Localhost connections are always allowed.

SECRET_KEY

Set to:

python manage.py shell -c 'from django.core.management import utils; print(utils.get_random_secret_key())'

DATABASE_URL

Set according to dj-database-url’s documentation.

SECURE_HSTS_SECONDS

Set according to Django’s documentation.

SENTRY_DSN

Set to the project’s client key (DSN) from Sentry.

FATHOM_ANALYTICS_ID (and FATHOM_ANALYTICS_DOMAIN)

Set to the site’s ID (and custom domain) from Fathom Analytics. Remember to configure your project to render the embed code (example).

Using the settings template¶

SECRET_KEY

Replace !!!SECRET_KEY!!! with:

python manage.py shell -c 'from django.core.management import utils; print(utils.get_random_secret_key())'

INSTALLED_APPS

Do not enable more applications than necessary. Among the default applications:

django.contrib.admin (tutorial): Remove, unless using the Django admin (check for occurrences of admin).
django.contrib.auth (topic): Remove, unless using django.contrib.admin or authenticated users (check for occurrences of auth or user).
django.contrib.messages: Remove, unless using django.contrib.admin or one-time messages (check for occurrences of messages).
django.contrib.contenttypes: Remove, unless using django.contrib.admin, django.contrib.auth or otherwise dependent.
django.contrib.sessions (topic): Remove, unless using django.contrib.admin, django.contrib.auth, django.contrib.messages or anonymous sessions (check for occurrences of session).
django.contrib.staticfiles (how-to): Remove, unless the project contains static files.
django.contrib.sitemaps: Remove, if the application is private. (Added by the Cookiecutter template.)

Then, make any corresponding changes to urls.py, and MIDDLEWARE, TEMPLATES, STATIC_URL and AUTH_PASSWORD_VALIDATORS in settings.py.

DATABASES

Replace {{ cookiecutter.database_name }} and {{ cookiecutter.application_name }}.
Remember to add dj-database-url to your requirements file.

LOGGING

See Logging.

Add additional settings for:

Django under # Project-specific Django configuration
Dependencies under # Dependency configuration
Your project under # Project configuration

Settings template¶

This template is based on the default settings.py file. You can also refer to the default Django settings. Replace core with the project’s module name and remove the Jinja syntax if not using the Cookiecutter template:

"""
Django settings for the project.

Generated by 'django-admin startproject' using Django 4.2.2.

For more information on this file, see
https://docs.djangoproject.com/en/4.2/topics/settings/

For the full list of settings and their values, see
https://docs.djangoproject.com/en/4.2/ref/settings/
"""

import os
from glob import glob
from pathlib import Path

import dj_database_url
import sentry_sdk
from sentry_sdk.integrations.django import DjangoIntegration
from sentry_sdk.integrations.logging import ignore_logger

production = os.getenv("DJANGO_ENV") == "production"
local_access = "LOCAL_ACCESS" in os.environ or "ALLOWED_HOSTS" not in os.environ

# Build paths inside the project like this: BASE_DIR / 'subdir'.
BASE_DIR = Path(__file__).resolve().parent.parent


# Quick-start development settings - unsuitable for production
# See https://docs.djangoproject.com/en/4.2/howto/deployment/checklist/

# SECURITY WARNING: keep the secret key used in production secret!
SECRET_KEY = os.getenv("SECRET_KEY", "!!!SECRET_KEY!!!")

# SECURITY WARNING: don't run with debug turned on in production!
DEBUG = os.getenv("DEBUG", str(not production)) == "True"

ALLOWED_HOSTS = [".localhost", "127.0.0.1", "[::1]", "0.0.0.0"]
if "ALLOWED_HOSTS" in os.environ:
    ALLOWED_HOSTS.extend(os.getenv("ALLOWED_HOSTS").split(","))


# Application definition

INSTALLED_APPS = [
    "django.contrib.admin",
    "django.contrib.auth",
    "django.contrib.contenttypes",
    "django.contrib.sessions",
    "django.contrib.messages",
    "django.contrib.staticfiles",
    "django.contrib.sitemaps",
]

MIDDLEWARE = [
    "django.middleware.security.SecurityMiddleware",
    "django.contrib.sessions.middleware.SessionMiddleware",
    "django.middleware.locale.LocaleMiddleware",
    "django.middleware.common.CommonMiddleware",
    "django.middleware.csrf.CsrfViewMiddleware",
    "django.contrib.auth.middleware.AuthenticationMiddleware",
    "django.contrib.messages.middleware.MessageMiddleware",
    "django.middleware.clickjacking.XFrameOptionsMiddleware",
]

ROOT_URLCONF = "core.urls"

TEMPLATES = [
    {
        "BACKEND": "django.template.backends.django.DjangoTemplates",
        "DIRS": [],
        "APP_DIRS": True,
        "OPTIONS": {
            "context_processors": [
                "django.template.context_processors.debug",
                "django.template.context_processors.request",
                "django.template.context_processors.i18n",
                "django.contrib.auth.context_processors.auth",
                "django.contrib.messages.context_processors.messages",
{%- if cookiecutter.use_fathom == "y" %}
                "core.context_processors.from_settings",
{%- endif %}
            ],
        },
    },
]

WSGI_APPLICATION = "core.wsgi.application"


# Database
# https://docs.djangoproject.com/en/4.2/ref/settings/#databases

DATABASES = {
    "default": dj_database_url.config(default="postgresql:///{{ cookiecutter.database_name }}?application_name={{ cookiecutter.application_name }}")
}


# Password validation
# https://docs.djangoproject.com/en/4.2/ref/settings/#auth-password-validators

AUTH_PASSWORD_VALIDATORS = [
    {
        "NAME": "django.contrib.auth.password_validation.UserAttributeSimilarityValidator",
    },
    {
        "NAME": "django.contrib.auth.password_validation.MinimumLengthValidator",
    },
    {
        "NAME": "django.contrib.auth.password_validation.CommonPasswordValidator",
    },
    {
        "NAME": "django.contrib.auth.password_validation.NumericPasswordValidator",
    },
]


# Internationalization
# https://docs.djangoproject.com/en/4.2/topics/i18n/

LANGUAGE_CODE = "en-us"

TIME_ZONE = "UTC"

USE_I18N = True

USE_TZ = True


# Static files (CSS, JavaScript, Images)
# https://docs.djangoproject.com/en/4.2/howto/static-files/

STATIC_URL = "static/"

# Default primary key field type
# https://docs.djangoproject.com/en/4.2/ref/settings/#default-auto-field

DEFAULT_AUTO_FIELD = "django.db.models.BigAutoField"


# Project-specific Django configuration

LOCALE_PATHS = glob(str(BASE_DIR / "**" / "locale"))

STATIC_ROOT = BASE_DIR / "static"
{%- if cookiecutter.use_fathom == "y" %}

STORAGES = {
    "default": {
        "BACKEND": "django.core.files.storage.FileSystemStorage",
    },
    "staticfiles": {
        "BACKEND": "django.contrib.staticfiles.storage.ManifestStaticFilesStorage",
    },
}
{%- endif %}

# https://docs.djangoproject.com/en/4.2/topics/logging/#django-security
LOGGING = {
    "version": 1,
    "disable_existing_loggers": False,
    "formatters": {
        "console": {
            "format": "%(asctime)s %(levelname)s [%(name)s:%(lineno)s] %(message)s",
        },
    },
    "handlers": {
        "console": {
            "class": "logging.StreamHandler",
            "formatter": "console",
        },
        "null": {
            "class": "logging.NullHandler",
        },
    },
    "loggers": {
        "": {
            "handlers": ["console"],
            "level": "INFO",
        },
        "django.security.DisallowedHost": {
            "handlers": ["null"],
            "propagate": False,
        },
        "django.db.backends": {
            "handlers": ["console"],
            "level": "DEBUG" if production else os.getenv("LOG_LEVEL", "INFO"),
            "propagate": False,
        },
    },
}

# https://docs.djangoproject.com/en/4.2/howto/deployment/checklist/
if production and not local_access:
    # Run: env DJANGO_ENV=production SECURE_HSTS_SECONDS=1 ./manage.py check --deploy
    CSRF_COOKIE_SECURE = True
    SESSION_COOKIE_SECURE = True
    SECURE_SSL_REDIRECT = True
    SECURE_REFERRER_POLICY = "same-origin"  # default in Django >= 3.1

    # https://docs.djangoproject.com/en/4.2/ref/middleware/#http-strict-transport-security
    if "SECURE_HSTS_SECONDS" in os.environ:
        SECURE_HSTS_SECONDS = int(os.getenv("SECURE_HSTS_SECONDS"))
        SECURE_HSTS_INCLUDE_SUBDOMAINS = True
        SECURE_HSTS_PRELOAD = True

# https://docs.djangoproject.com/en/4.2/ref/settings/#secure-proxy-ssl-header
if "DJANGO_PROXY" in os.environ:
    USE_X_FORWARDED_HOST = True
    SECURE_PROXY_SSL_HEADER = ("HTTP_X_FORWARDED_PROTO", "https")


# Dependency configuration

if "SENTRY_DSN" in os.environ:
    # https://docs.sentry.io/platforms/python/logging/#ignoring-a-logger
    ignore_logger("django.security.DisallowedHost")
    sentry_sdk.init(
        dsn=os.getenv("SENTRY_DSN"),
        integrations=[DjangoIntegration()],
        traces_sample_rate=1.0,
    )


# Project configuration

FATHOM = {
    "domain": os.getenv("FATHOM_ANALYTICS_DOMAIN") or "cdn.usefathom.com",
    "id": os.getenv("FATHOM_ANALYTICS_ID"),
}

See also

The LANGUAGE_CODE is en-us. See Internationalization (i18n) for details.

Performance¶

In order of importance:

Reduce the number of SQL queries (avoid N+1 queries):
- Avoid queries inside a loop. For SELECT, perform a single query before the loop (or do the work in batches). For INSERT and UPDATE, use the bulk_create and bulk_update methods after the loop (or do the work in batches).
  
  Warning
  
  Read the caveats for the bulk_* methods in the Django documentation.
- Use select_related to reduce the number of queries on ForeignKey or OneToOneField relations.
- Use prefetch_related to reduce the number of queries on ManyToManyField and reverse ForeignKey relations.
- Use count if you only need to count. However, if you also need to use the result of the queryset, use len().
- Use assertNumQueries in tests.
- Set the django logger’s level to DEBUG in development to add SQL queries to the runserver output.
Cache results:
- Use Django’s cache framework, and be sure to invalidate the cache when appropriate.
Optimize queries:
- Add indexes to fields that are frequently used for filtering. To find slow queries, you can use the PostgreSQL log in production or the SQL panel of Django Debug Toolbar in development.
- Use the update_fields argument to the save() method for frequent operations.
Minimize memory usage:
- Use iterator when iterating over a queryset whose result is only accessed this once.

Read:

Performance and optimization
Database access optimization
Performance optimizations in the Deployment checklist

Deployment¶

To perform deployment checks locally, run:

env DJANGO_ENV=production ALLOWED_HOSTS=example.com SECURE_HSTS_SECONDS=1 ./manage.py check --deploy --fail-level WARNING

Use the Deployment checklist
Read Deploying Django

See also

Dockerfile for Django

Static files¶

DO NOT commit the files generated by collectstatic. These commands are either run during deployment or when creating Docker images.

Use ManifestStaticFilesStorage for cache-busting.

Generated files¶

DO NOT commit the files generated by compilemessages. These commands are either run during deployment or when creating Docker images.

Development¶

Troubleshooting¶

See also

pdb — The Python Debugger

To access a Python shell with Django configured:

./manage.py shell

To access the default database:

./manage.py dbshell

See also

django-admin and manage.py

To log SQL statements, add this under "loggers" in settings.py:

"django.db.backends": {
    "handlers": ["console"],
    "level": "DEBUG",
    "propagate": False,
},

Learning¶

Tutorials
Topics
How-to, for example: Writing custom django-admin commands.