Shell script

Directory layout

For builds that involve independent command-line tools, use Make, and follow DataMade’s Making Data Guidelines and Clark Grubb’s Makefile Style Guide, for example: standard_profile_template

If a repository has scripts to set itself up and/or update itself, follow GitHub’s Scripts to Rule Them All, for example: deploy and standard_profile_template

Filename conventions

sh and bash scripts should use the .sh extension, unless they are in a script/ directory.

Shell options

Start shell scripts with set -euo pipefail. If the script explicitly handles unset variables, omit -u. To see which command failed due to the -e option, add -x.

See also

The Set Builtin

Code style

Check /bin/sh scripts using checkbashisms.

Check shell scripts using shellcheck.

Style shell scripts using shfmt: for example, shfmt -w -i 4 -sr (shfmt -f .).

Use:

• sh instead of bash, where possible. Bash is needed for:

  • Shell Parameter Expansion (//, ##, %%, etc.)

  • Process Substitution (<())

  • The mapfile Builtin

  • The pipefail Option

• [ ] instead of test

• [ ] instead of [[ ]] in Bash, unless required

• $NAME instead of ${NAME}, unless followed by a word character

• Subshells to temporarily change directory, for example:

  (
     cd subdir/
     mv x.txt y.txt
  )
  

  Instead of:

  cd subdir/
  mv x.txt y.txt
  cd ..  # AVOID
  

And:

• Avoid set -x in scripts run by continuous integration, because it will expand any secret variables

Variable names

Use lowercase names for local variables. (shellcheck reports SC2154) only for lowercase names.)

Reserve uppercase names for environment variables (like LOGNAME or MAILTO) and external variables from sourced files (like /etc/os-release).

Avoid shadowing commands (like host, dir, mail):

• Store a command’s path as <tool>_bin, for example: mail_bin=/usr/bin/mail

• Store a command’s output as <tool>_output, for example: hostname_output=$(hostname)

Continuous integration

Create a .github/workflows/shell.yml file. In most cases, you can reuse the shell workflow, like:

jobs:
  lint:
    uses: open-contracting/.github/.github/workflows/shell.yml@main
    permissions:
      contents: read
    with:
      ignore: file.sh

Otherwise, as a base, use:

name: Lint Shell
on: [push, pull_request]
jobs:
  build:
    if: github.event_name == 'push' || github.event.pull_request.head.repo.full_name != github.repository
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v5
      - run: |
          sudo apt update
          sudo apt install devscripts shellcheck shfmt
      - run: checkbashisms $(shfmt -f .)
      - run: shellcheck $(shfmt -f .)
      - run: shfmt -d -i 4 -sr $(shfmt -f .)

Maintenance

Find repositories with shell scripts but without shell.yml files:

find . \( -path '*/script/*' -o -name '*.sh' \) ! -path '*/.mypy_cache/*' ! -path '*/node_modules/*' ! -path '*/vendor/*' -exec bash -c 'if [[ -z $(find $(echo {} | cut -d/ -f2) -name shell.yml) ]]; then echo {}; fi' \;

Reference

• Shell Command Language

• Bash Conditional Constructs

• Wizard Zines Bite Size series