diff --git a/README.md b/README.md index 7946395..23944fe 100644 --- a/README.md +++ b/README.md @@ -20,31 +20,31 @@ It is to be understood as a software development pattern to standardize the inst The specification is a short guide to set up a task file for a Python project. -- Create a file `task` in your project. +Create a file `task` in your project. ```bash touch task ``` -- Ensure it is executable. +Ensure it is executable. ```bash chmod +x task ``` -- First add the bash shebang. +First add the bash shebang. ```bash #!/usr/bin/env bash ``` -- Then append the abort on error setting. +Then append the abort on error setting. ```bash set -e ``` -- Load environment variables from the `.env` file. +Load environment variables from the `.env` file. ```bash if [[ -a ".env" ]]; then @@ -52,7 +52,7 @@ if [[ -a ".env" ]]; then fi ``` -- Add a help function. +Add a help function. ```bash help-table() { @@ -83,7 +83,7 @@ help() { } ``` -- Setup the command functions. +Setup the command functions. ```bash version() { @@ -105,7 +105,7 @@ lint() { } ``` -- Finish the task file with command switch cases. +Finish the task file with command switch cases. ```bash if declare -f "$1" > /dev/null; then @@ -127,17 +127,25 @@ fi These are the main parts of every task file script. -## Examples +Now see [usage](/usage) on how to use the task file. -Checkout these implementations of the task file standard: +## Naming -- [janikvonrotz/dotfiles](https://codeberg.org/janikvonrotz/dotfiles/src/main/task) -- [janikvonrotz/python.casa](https://codeberg.org/janikvonrotz/python.casa/src/main/task) -- [Mint-System/Ansible-Build](https://github.com/Mint-System/Ansible-Build/blob/main/task) -- [Mint-System/Kubernetes-Build](https://github.com/Mint-System/Kubernetes-Build/blob/main/task) -- [Mint-System/Odoo-Build](https://github.com/Mint-System/Odoo-Build/blob/main/task) -- [Mint-System/Odoo-Wiki](https://github.com/Mint-Wiki/Odoo-Build/blob/main/task) -- [Mint-System/Website](https://github.com/Mint-System/Website/blob/main/task) -- [Mint-System/Wiki](https://github.com/Mint-System/Wiki/blob/main/task) +The naming of functions is important. There are basically two styles: -This website is built with a `task` file. Here is the source: [janikvonrotz/taskfile.build](https://codeberg.org/janikvonrotz/taskfile.build/src/branch/main/task) +1. Action + Object +2. Object + Action + +The task file functions use the first style. The name of the function starts with an action followed by an object. The object name can be singular or plural. + +**Examples for actions**: activate, install, dev, develop, init, build, start, update, remove, delete, enable, disable, template, convert, create, edit, change, get, set, patch, fetch, generate, push, pull, import, export, list, publish, release, test, setup, prepare, restart, stop, store, restore, translate, upgrade, zip, visualize, sync, switch, run, reset, load, dump, checkout, commit, drop, deploy, handle, trigger, render, lint, uninstall, split, parse, fix, refactor, transform, cat, ls, rm, serve, help, show, filter, login, logout, encrypt, decrypt, upload, download, analyse, transpile, compile, minify, copy + +**Examples for objects**: env, venv, submodule, container, database, snippet, model, module, repo, mail, doc, dependency, view, user, vault, file, host, node, log, password, hash, script, requirement, part, component, system, workspace, image, process, state, platform, dir, folder, readme, overview, lang, level, request, response, result, worker, server, proxy, workflow, volume, network, package, field, value, secret, chart, node, edge, function, method, firewall, html, css, image, svg, style, query, native, group, notebook + +**Objects can be tools**: odoo, vupress, nodejs, zsh, bash, fish, podman, kind, minikube, helm, nvim, docker, podman, rust, python, tmux, vim, helix, system, git, pass, llm, sql, dotenv, javascript, vue, vite, astro, typescript, turbo, pnpm, eslint, jenkins, k8s, nextcloud, postgres, metabase, ansible, prometheus, grafana, hugo, deno, bun, babel, panda, gulp, grunt, electron, react, express, mongodb, angular, ionic, meteor, webpack, bower, jupyter + + + +## Style + +The bash scripts follow the [Bash Style Guide | ysap.sh](https://style.ysap.sh/). Have a look at the [AGENTS.md](/AGENTS) for details on the applied styling. diff --git a/_includes/layouts/base.njk b/_includes/layouts/base.njk index 46142d0..143e1bc 100644 --- a/_includes/layouts/base.njk +++ b/_includes/layouts/base.njk @@ -28,11 +28,10 @@
  • taskfile.build
    • diff --git a/examples.md b/examples.md new file mode 100644 index 0000000..76f213c --- /dev/null +++ b/examples.md @@ -0,0 +1,18 @@ +--- +layout: layouts/base.njk +--- + +# Examples + +Checkout these implementations of the task file standard: + +- [janikvonrotz/dotfiles](https://codeberg.org/janikvonrotz/dotfiles/src/main/task) +- [janikvonrotz/python.casa](https://codeberg.org/janikvonrotz/python.casa/src/main/task) +- [Mint-System/Ansible-Build](https://github.com/Mint-System/Ansible-Build/blob/main/task) +- [Mint-System/Kubernetes-Build](https://github.com/Mint-System/Kubernetes-Build/blob/main/task) +- [Mint-System/Odoo-Build](https://github.com/Mint-System/Odoo-Build/blob/main/task) +- [Mint-System/Odoo-Wiki](https://github.com/Mint-Wiki/Odoo-Build/blob/main/task) +- [Mint-System/Website](https://github.com/Mint-System/Website/blob/main/task) +- [Mint-System/Wiki](https://github.com/Mint-System/Wiki/blob/main/task) + +This website is built with a `task` file. Here is the source: [janikvonrotz/taskfile.build](https://codeberg.org/janikvonrotz/taskfile.build/src/branch/main/task) diff --git a/patterns.md b/patterns.md new file mode 100644 index 0000000..57b180e --- /dev/null +++ b/patterns.md @@ -0,0 +1,276 @@ +--- +layout: layouts/base.njk +--- + +# Patterns + +The task file from the specification is very basic. Commands can also have parameters and call each other. + +The following is a collection of more complex task file patterns. + +## Set default parameter + +Fallback to a default value for a parameter. + +```bash +build() { + local platform="amd64" + if [[ -n "$1" ]]; then + platform="$1" + fi +``` + +## Ensure parameter is not empty + +Check the first param and exit if it is empty. + +```bash +deploy() { + if [[ -z "$1" ]]; then echo "\$1 is empty."; exit; fi +``` + +## Prompt for input + +Use `read` to ask for inputs. + +```bash +if [[ -z "$2" ]]; then + read -p "Enter the task description: " task_description +else + task_description="$2" +fi +``` + +## Setup local env vars + +Define env vars at the beginning of the task file. + +```bash +CONFIGURATION_FILE="file.conf" +GIT_BRANCH=$(git symbolic-ref --short -q HEAD) +``` + +## Template with env vars + +Create a parameterized file from a template. Requires `envsubst`. + +```bash +template-with-env() { + echo "Template $CONFIGURATION_FILE" + + export CONFIGURATION_1 + export CONFIGURATION_2=${CONFIGURATION_2:="value"} + + envsubst < "file.conf.template" > "$CONFIGURATION_FILE" +} +``` + +## Create Python virtual env + +Initialize Python virtual env with uv. + +```bash +init-venv() { + if [[ ! -d "venv$GIT_BRANCH" ]]; then + echo "Init venv$GIT_BRANCH with $(uv --version)." + uv venv "venv$GIT_BRANCH" + fi +} +``` + +## Activate Python virtual env + +```bash +activate-venv() { + echo "Source virtualenv venv$GIT_BRANCH." + source "venv$GIT_BRANCH/bin/activate" + echo "$(python --version) is active." +} +``` + +## Call a Python script + +Run a Python script. + +```bash +generate-password-hash() { + activate-venv + if [[ -z "$1" ]]; then echo "\$1 is empty."; exit; fi + export PASSWORD_PLAIN="$1" + bin/password-hash +} +``` + +The Python script: `bin/password-hash` + +```python +#!/usr/bin/env python3 + +import os +from passlib.context import CryptContext +crypt_context = CryptContext(schemes=['pbkdf2_sha512', 'plaintext'], deprecated=['plaintext']) + +password = os.environ.get('PASSWORD_PLAIN') +print(crypt_context.hash(password)) +``` + +## Command with named parameters + +Assuming you have a `docker-compose.yml` and would like to start selected or all containers. + +```bash +start() { + if [[ "$1" =~ "db" ]]; then + docker compose up -d db + fi + + if [[ "$1" =~ "admin" ]]; then + docker compose up -d admin + echo "Open http://localhost:8000 url in your browser." + fi + + if [[ "$1" =~ "odoo" ]]; then + docker compose up -d odoo + echo "Open http://localhost:8069 url in your browser." + fi + + if [[ "$1" =~ "mail" ]]; then + docker compose up -d mail + echo "Open http://localhost:8025 url in your browser." + fi +} +``` + +Start all containers with `task start` and selected with `task start db,admin`. + +## Run commands in container + +Use `docker exec -i` to run commands in a container. + +```bash +drop-db() { + local database="$1" + if [[ -z "$database" ]]; then + database="example" + fi + + docker exec -i db psql "postgres://odoo:odoo@localhost:5432/postgres" -c "SELECT pg_terminate_backend(pid) FROM pg_stat_activity WHERE datname = '$database';" + docker exec -i db psql "postgres://odoo:odoo@localhost:5432/postgres" -c "DROP DATABASE \"$database\";" +} +``` + +## Loop over files and folders + +Use `for` to loop over file, folders or arrays. + +```bash +render() { + echo "Update index.html for all folders" + for folder in ./*; do + if [[ -f "$folder/README.md" ]]; then + cd "$folder" || exit + md2html README.md _site/index.html + cd .. || exit + fi + done +} +``` + +## Parse and convert a file + +With this function you can split a file into multiple parts whenever a specific keyword is matching. In this example the keyword is `!vault`. + +```bash +convert-vault-file() { + local file_path="$1" + local temp_file=$(mktemp) + local temp_part_file=$(mktemp) + local write_finished=false + local current_key="" + + while IFS= read -r line; do + # Check for keyword + if [[ "$line" =~ "!vault" ]]; then + # Process previous vault entry if it exists + if [[ "$write_finished" = true ]] && [[ -n "$current_key" ]] && [[ -s "$temp_part_file" ]]; then + # Decrypt part file and write to assemble file + ansible-vault decrypt "$temp_part_file" + local value=$(cat "$temp_part_file") + echo "$current_key: $value" >> "$temp_file" + fi + + # Set up for new vault entry + current_key=$(echo "$line" | cut -d':' -f1) + # Clear the file + : > "$temp_part_file" + # Flag as ready to write + write_finished=true + else + if [[ "$write_finished" = true ]]; then + # Pipe into part file + echo "$line" >> "$temp_part_file" + fi + done + done < "$file_path" + + # Process the final vault entry + if [[ "$write_finished" = true ]] && [[ -n "$current_key" ]] && [[ -s "$temp_part_file" ]]; then + ansible-vault decrypt "$temp_part_file" + local value=$(cat "$temp_part_file") + echo "$current_key: $value" >> "$temp_file" + fi + + # Output assembled file + cat "$temp_file" + # Cleanup temp files + rm -f "$temp_file" + rm -f "$temp_part_file" +} +``` + +## Process data + +This pattern tries to match the worklow of a jupyter notebook. Create a folder with the data processing scripts: + +``` +scripts +├── 01_import-mail-data +├── 02_transform-mail-data +└── 03_export-mail-data +``` + +The following function lists the scripts and asks for the number of scripts to run. Every script until the entered number is concatenated and executed. + +```bash +process-data() { + while true; do + local scripts=($(ls scripts)) + + echo -e "Available scripts:\n" + for index in "${!scripts[@]}"; do + echo "$((index+1)): ${scripts[$index]}" + done + echo "" + read -p "Enter the number of scripts to run (or 'q' to quit): " user_input + if [[ "$user_input" = "q" ]]; then + break + fi + + local temp_file=$(mktemp) + for index in "${!scripts[@]}"; do + if (( $user_input >= $((index+1)) )); then + cat "scripts/${scripts[$index]}" >> "$temp_file" + echo >> "$temp_file" + fi + done + + clear + echo -e "\nPython Output:\n\n---" + python "$temp_file" + local exit_code=$? + echo -e "---\n" + echo -e "Exit code: $exit_code\n" + rm "$temp_file" + done +} +``` diff --git a/usage.md b/usage.md index 67d1ecf..ac89f10 100644 --- a/usage.md +++ b/usage.md @@ -4,312 +4,15 @@ layout: layouts/base.njk # Usage -Running the task file requires a shell alias: `alias task='./task'`\ -Show the available commands with `task help`.\ -From the specification the project can be installed with `task install`.\ +Running the task file requires a shell alias: `alias task='./task'` + +Show the available commands with `task help`. + +From the specification the project can be installed with `task install`. + Execute all commands with `task all`. -## Image - -See [taskfile.build/image](/image) for details. - -## Naming - -The naming of functions is important. There are basically two styles: - -1. Action + Object -2. Object + Action - -The task file functions use the first style. The name of the function starts with an action followed by an object. The object name can be singular or plural. - -**Examples for actions**: activate, install, dev, develop, init, build, start, update, remove, delete, enable, disable, template, convert, create, edit, change, get, set, patch, fetch, generate, push, pull, import, export, list, publish, release, test, setup, prepare, restart, stop, store, restore, translate, upgrade, zip, visualize, sync, switch, run, reset, load, dump, checkout, commit, drop, deploy, handle, trigger, render, lint, uninstall, split, parse, fix, refactor, transform, cat, ls, rm, serve, help, show, filter, login, logout, encrypt, decrypt, upload, download, analyse, transpile, compile, minify, copy - -**Examples for objects**: env, venv, submodule, container, database, snippet, model, module, repo, mail, doc, dependency, view, user, vault, file, host, node, log, password, hash, script, requirement, part, component, system, workspace, image, process, state, platform, dir, folder, readme, overview, lang, level, request, response, result, worker, server, proxy, workflow, volume, network, package, field, value, secret, chart, node, edge, function, method, firewall, html, css, image, svg, style, query, native, group, notebook - -**Objects can be tools**: odoo, vupress, nodejs, zsh, bash, fish, podman, kind, minikube, helm, nvim, docker, podman, rust, python, tmux, vim, helix, system, git, pass, llm, sql, dotenv, javascript, vue, vite, astro, typescript, turbo, pnpm, eslint, jenkins, k8s, nextcloud, postgres, metabase, ansible, prometheus, grafana, hugo, deno, bun, babel, panda, gulp, grunt, electron, react, express, mongodb, angular, ionic, meteor, webpack, bower, jupyter - - - -## Style - -The bash scripts follow the [Bash Style Guide | ysap.sh](https://style.ysap.sh/). Have a look at the [AGENTS.md](/AGENTS) for details on the applied styling. - -## Patterns - -The task file showed above is very basic. Commands can have parameters and call each other. The following is a collection of more complex task file patterns. - -### Set default parameter - -Fallback to a default value for a parameter. - -```bash -build() { - local platform="amd64" - if [[ -n "$1" ]]; then - platform="$1" - fi -``` - -### Ensure parameter is not empty - -Check the first param and exit if it is empty. - -```bash -deploy() { - if [[ -z "$1" ]]; then echo "\$1 is empty."; exit; fi -``` - -### Prompt for input - -Use `read` to ask for inputs. - -```bash -if [[ -z "$2" ]]; then - read -p "Enter the task description: " task_description -else - task_description="$2" -fi -``` - -### Setup local env vars - -Define env vars at the beginning of the task file. - -```bash -CONFIGURATION_FILE="file.conf" -GIT_BRANCH=$(git symbolic-ref --short -q HEAD) -``` - -### Template with env vars - -Create a parameterized file from a template. Requires `envsubst`. - -```bash -template-with-env() { - echo "Template $CONFIGURATION_FILE" - - export CONFIGURATION_1 - export CONFIGURATION_2=${CONFIGURATION_2:="value"} - - envsubst < "file.conf.template" > "$CONFIGURATION_FILE" -} -``` - -### Create Python virtual env - -Initialize Python virtual env with uv. - -```bash -init-venv() { - if [[ ! -d "venv$GIT_BRANCH" ]]; then - echo "Init venv$GIT_BRANCH with $(uv --version)." - uv venv "venv$GIT_BRANCH" - fi -} -``` - -### Activate Python virtual env - -```bash -activate-venv() { - echo "Source virtualenv venv$GIT_BRANCH." - source "venv$GIT_BRANCH/bin/activate" - echo "$(python --version) is active." -} -``` - -### Call a Python script - -Run a Python script. - -```bash -generate-password-hash() { - activate-venv - if [[ -z "$1" ]]; then echo "\$1 is empty."; exit; fi - export PASSWORD_PLAIN="$1" - bin/password-hash -} -``` - -The Python script: `bin/password-hash` - -```python -#!/usr/bin/env python3 - -import os -from passlib.context import CryptContext -crypt_context = CryptContext(schemes=['pbkdf2_sha512', 'plaintext'], deprecated=['plaintext']) - -password = os.environ.get('PASSWORD_PLAIN') -print(crypt_context.hash(password)) -``` - -### Command with named parameters - -Assuming you have a `docker-compose.yml` and would like to start selected or all containers. - -```bash -start() { - if [[ "$1" =~ "db" ]]; then - docker compose up -d db - fi - - if [[ "$1" =~ "admin" ]]; then - docker compose up -d admin - echo "Open http://localhost:8000 url in your browser." - fi - - if [[ "$1" =~ "odoo" ]]; then - docker compose up -d odoo - echo "Open http://localhost:8069 url in your browser." - fi - - if [[ "$1" =~ "mail" ]]; then - docker compose up -d mail - echo "Open http://localhost:8025 url in your browser." - fi -} -``` - -Start all containers with `task start` and selected with `task start db,admin`. - -### Run commands in container - -Use `docker exec -i` to run commands in a container. - -```bash -drop-db() { - local database="$1" - if [[ -z "$database" ]]; then - database="example" - fi - - docker exec -i db psql "postgres://odoo:odoo@localhost:5432/postgres" -c "SELECT pg_terminate_backend(pid) FROM pg_stat_activity WHERE datname = '$database';" - docker exec -i db psql "postgres://odoo:odoo@localhost:5432/postgres" -c "DROP DATABASE \"$database\";" -} -``` - -### Loop over files and folders - -Use `for` to loop over file, folders or arrays. - -```bash -render() { - echo "Update index.html for all folders" - for folder in ./*; do - if [[ -f "$folder/README.md" ]]; then - cd "$folder" || exit - md2html README.md _site/index.html - cd .. || exit - fi - done -} -``` - -### Parse and convert a file - -With this function you can split a file into multiple parts whenever a specific keyword is matching. In this example the keyword is `!vault`. - -```bash -convert-vault-file() { - local file_path="$1" - local temp_file=$(mktemp) - local temp_part_file=$(mktemp) - local write_finished=false - local current_key="" - - while IFS= read -r line; do - # Check for keyword - if [[ "$line" =~ "!vault" ]]; then - # Process previous vault entry if it exists - if [[ "$write_finished" = true ]] && [[ -n "$current_key" ]] && [[ -s "$temp_part_file" ]]; then - # Decrypt part file and write to assemble file - ansible-vault decrypt "$temp_part_file" - local value=$(cat "$temp_part_file") - echo "$current_key: $value" >> "$temp_file" - fi - - # Set up for new vault entry - current_key=$(echo "$line" | cut -d':' -f1) - # Clear the file - : > "$temp_part_file" - # Flag as ready to write - write_finished=true - else - if [[ "$write_finished" = true ]]; then - # Pipe into part file - echo "$line" >> "$temp_part_file" - fi - done - done < "$file_path" - - # Process the final vault entry - if [[ "$write_finished" = true ]] && [[ -n "$current_key" ]] && [[ -s "$temp_part_file" ]]; then - ansible-vault decrypt "$temp_part_file" - local value=$(cat "$temp_part_file") - echo "$current_key: $value" >> "$temp_file" - fi - - # Output assembled file - cat "$temp_file" - # Cleanup temp files - rm -f "$temp_file" - rm -f "$temp_part_file" -} -``` - -### Process data - -This pattern tries to match the worklow of a jupyter notebook. Create a folder with the data processing scripts: - -``` -scripts -├── 01_import-mail-data -├── 02_transform-mail-data -└── 03_export-mail-data -``` - -The following function lists the scripts and asks for the number of scripts to run. Every script until the entered number is concatenated and executed. - -```bash -process-data() { - while true; do - local scripts=($(ls scripts)) - - echo -e "Available scripts:\n" - for index in "${!scripts[@]}"; do - echo "$((index+1)): ${scripts[$index]}" - done - echo "" - read -p "Enter the number of scripts to run (or 'q' to quit): " user_input - if [[ "$user_input" = "q" ]]; then - break - fi - - local temp_file=$(mktemp) - for index in "${!scripts[@]}"; do - if (( $user_input >= $((index+1)) )); then - cat "scripts/${scripts[$index]}" >> "$temp_file" - echo >> "$temp_file" - fi - done - - clear - echo -e "\nPython Output:\n\n---" - python "$temp_file" - local exit_code=$? - echo -e "---\n" - echo -e "Exit code: $exit_code\n" - rm "$temp_file" - done -} -``` - -## Integration - -The task file is intended to be integrated into the shell setup. Where applications provides a domain specific language (DSL) ensure that the DSL only calls the task file. - -### Completion +## Completion With the output of `task help` the task commands can be completed. @@ -356,6 +59,14 @@ esac The completion also works for task alias `alias t='./task'`. +## Image + +See [taskfile.build/image](/image) for details. + +## Integration + +The task file is intended to be integrated into the shell setup. Whenever an applicatio uses a domain specific language (DSL) ensure that the DSL calls the task file. Most you'll see DSLs to describe build pipelines. + ### GitHub Actions Running task file commands in GitHub Actions is highly recommended. This way you can run the same CI/CD procedures in the GitHub runner as you do on your localhost. @@ -417,4 +128,4 @@ pipeline { } } }% -``` \ No newline at end of file +```