CLI reference

cli

Usage:

cli [OPTIONS] COMMAND [ARGS]...

Options:

  --debug / --no-debug
  --help                Show this message and exit.

Subcommands

• info: List benchmarks and/or information about them.
• run: Run benchmarks or benchmark modules.
• software: Manage and install benchmark-specific software.
• storage: Manage remote storage.

info

List benchmarks and/or information about them.

Usage:

cli info [OPTIONS] COMMAND [ARGS]...

Options:

  --help  Show this message and exit.

Subcommands

• diff: Show differences between 2 benchmark versions.
• list-versions: List all available benchmarks versions at a specific endpoint.

diff

Show differences between 2 benchmark versions.

Usage:

cli info diff [OPTIONS]

Options:

  -b, --benchmark PATH  Path to benchmark yaml file or benchmark id.
                        [required]
  -v1, --version1 TEXT  Reference version.  [required]
  -v2, --version2 TEXT  Version to compare with.  [required]
  --help                Show this message and exit.

list-versions

List all available benchmarks versions at a specific endpoint.

Usage:

cli info list-versions [OPTIONS]

Options:

  -b, --benchmark PATH  Path to benchmark yaml file or benchmark id.
                        [required]
  --help                Show this message and exit.

run

Run benchmarks or benchmark modules.

Usage:

cli run [OPTIONS] COMMAND [ARGS]...

Options:

  --help  Show this message and exit.

Subcommands

• benchmark: Run a benchmark as specified in the yaml.
• module:
• validate: Validate a benchmark yaml.

benchmark

Run a benchmark as specified in the yaml.

Usage:

cli run benchmark [OPTIONS]

Options:

  -b, --benchmark PATH   Path to benchmark yaml file or benchmark id.
                         [required]
  -p, --threads INTEGER  The parallelism level for the workflow scheduler.
  -u, --update           Force re-run execution for all modules and stages.
  -d, --dry              Dry run.
  -l, --local            Execute and store results locally. Default False.
  --help                 Show this message and exit.

module

Run a specific module on inputs present at a given folder.

Usage:

cli run module [OPTIONS]

Options:

  -b, --benchmark PATH  Path to benchmark yaml file or benchmark id.
                        [required]
  -m, --module TEXT     Module id to execute  [required]
  -i, --input_dir PATH  Path to the folder with the appropriate input files.
  -d, --dry             Dry run.
  -u, --update          Force re-run execution for all modules and stages.
  --help                Show this message and exit.

validate

Validate a benchmark yaml.

Usage:

cli run validate [OPTIONS]

Options:

  -b, --benchmark PATH  Path to benchmark yaml file or benchmark id.
  --help                Show this message and exit.

software

Manage and install benchmark-specific software.

Usage:

cli software [OPTIONS] COMMAND [ARGS]...

Options:

  --help  Show this message and exit.

Subcommands

• check: Check whether the component {what} is available.
• conda: Manage and install software using conda.
• module: Manage and install software using Easybuild
• singularity: Manage and install software using Singularity.

check

Check whether the component {what} is available.

Usage:

cli software check [OPTIONS]

Options:

  -w, --what TEXT  Binary/functionality to check:

                   --what singularity : singularity

                   --what module      : module tool, typically lmod

                   --what easybuild   : easybuild

                   --what conda       : conda
  --help           Show this message and exit.

conda

Manage and install software using conda.

Usage:

cli software conda [OPTIONS] COMMAND [ARGS]...

Options:

  --help  Show this message and exit.

Subcommands

• pin: Pin all conda env-related dependencies versions using snakedeploy.
• prepare: Pin all conda envs needed for a given benchmark YAML.

pin

Pin all conda env-related dependencies versions using snakedeploy.

Usage:

cli software conda pin [OPTIONS]

Options:

  -e, --env PATH  Conda env YAML.  [required]
  --help          Show this message and exit.

prepare

Pin all conda envs needed for a given benchmark YAML.

Usage:

cli software conda prepare [OPTIONS]

Options:

  -b, --benchmark PATH  Benchmark YAML.  [required]
  --help                Show this message and exit.

module

Manage and install software using Easybuild

Usage:

cli software module [OPTIONS] COMMAND [ARGS]...

Options:

  --help  Show this message and exit.

Subcommands

• build: Build a given easyconfig (and generates the relevant envmodules).
• prepare: Build all envmodules needed for a given benchmark YAML.

build

Build a given easyconfig (and generates the relevant envmodules).

Usage:

cli software module build [OPTIONS]

Options:

  -e, --easyconfig TEXT  Easyconfig.  [required]
  -p, --threads INTEGER  Number of threads.
  --help                 Show this message and exit.

prepare

Build all envmodules needed for a given benchmark YAML.

Usage:

cli software module prepare [OPTIONS]

Options:

  -b, --benchmark PATH   Benchmark YAML.  [required]
  -p, --threads INTEGER  Number of threads.
  --help                 Show this message and exit.

singularity

Manage and install software using Singularity.

Usage:

cli software singularity [OPTIONS] COMMAND [ARGS]...

Options:

  --help  Show this message and exit.

Subcommands

• build: Build a singularity (fakeroot) image for a given easyconfig.
• prepare: Build all singularity (fakeroot) images needed for a benchmark.
• push: Pushes a singularity SIF file to an ORAS-compatible registry.

build

Build a singularity (fakeroot) image for a given easyconfig.

Usage:

cli software singularity build [OPTIONS]

Options:

  -e, --easyconfig TEXT  Easyconfig.  [required]
  --help                 Show this message and exit.

prepare

Build all singularity (fakeroot) images needed for a benchmark.

Usage:

cli software singularity prepare [OPTIONS]

Options:

  -b, --benchmark PATH  Benchmark YAML.  [required]
  --help                Show this message and exit.

push

Pushes a singularity SIF file to an ORAS-compatible registry.

Usage:

cli software singularity push [OPTIONS]

Options:

  -u, --docker_username TEXT  Docker username.  [required]
  -p, --docker_password TEXT  Docker password.  [required]
  -s, --sif PATH              Path to the Singularity SIF file.  [required]
  -o, --oras TEXT             Registry's ORAS static URL, for instance oras://
                              registry.mygitlab.ch/myuser/myproject:mytag.
                              [required]
  --help                      Show this message and exit.

storage

Manage remote storage.

Usage:

cli storage [OPTIONS] COMMAND [ARGS]...

Options:

  --help  Show this message and exit.

Subcommands

• checksum: Generate md5sums of all benchmark outputs
• create-policy: Create a new policy for a benchmark.
• create-version: Create a new benchmark version.
• download: Download all or specific files for a benchmark.
• list: List all or specific files for a benchmark.

checksum

Generate md5sums of all benchmark outputs

Usage:

cli storage checksum [OPTIONS]

Options:

  -b, --benchmark PATH  Path to benchmark yaml file or benchmark id.
                        [required]
  --help                Show this message and exit.

create-policy

Create a new policy for a benchmark.

Usage:

cli storage create-policy [OPTIONS]

Options:

  -b, --benchmark PATH  Path to benchmark yaml file or benchmark id.
                        [required]
  --help                Show this message and exit.

create-version

Create a new benchmark version.

Usage:

cli storage create-version [OPTIONS]

Options:

  -b, --benchmark PATH  Path to benchmark yaml file or benchmark id.
                        [required]
  --help                Show this message and exit.

download

Download all or specific files for a benchmark.

Usage:

cli storage download [OPTIONS]

Options:

  -b, --benchmark PATH  Path to benchmark yaml file or benchmark id.
                        [required]
  -t, --type TEXT       File types. Options: all, code, inputs, outputs, logs,
                        performance.
  -s, --stage TEXT      Stage to download files from.
  -m, --module TEXT     Module to download files from.
  -i, --id TEXT         File id to download.
  --help                Show this message and exit.

list

List all or specific files for a benchmark.

Usage:

cli storage list [OPTIONS]

Options:

  -b, --benchmark PATH  Path to benchmark yaml file or benchmark id.
                        [required]
  -t, --type TEXT       File types. Options: all, code, inputs, outputs, logs,
                        performance.
  -s, --stage TEXT      Stage to list files for.
  -i, --id TEXT         File id/type to list.
  --help                Show this message and exit.