Introduction to High-Performance Computing in Python: Glossary

Errors are there to help us.

To run a Python program, use python3 program_name.py.

Lists store a sequence of elements.

Numpy allows vector maths in Python.

Dicts provide key-value storage of information.

map() applies a function to every object in a data structure.

filter() returns only the data objects for which some condition is true.

Pool.map() will perform an operation in parallel.

Bash scripts are not an efficient way of storing a workflow.

Snakemake is one method of managing a complex computational workflow.

Snakemake follows Python syntax

Rules can have an input and/or outputs, and a command to be run.

Use {output} to refer to the output of the current rule.

Use {input} to refer to the dependencies of the current rule.

You can use Python indexing to retrieve individual outputs and inputs (example: {input[0]})

Wildcards can be named (example: {input.file1}).

Use any named wildcard ({some_name}) as a placeholder in targets and dependencies.

Snakefiles are Python code.

The entire Snakefile is executed whenever you run snakemake.

All actual work should be done by rules.

Use threads to indicate the number of cores used by a rule.

Resources are arbitrary and can be used for anything.

The && operator is a useful tool when chaining bash commands.

Snakemake generates and submits its own batch scripts for your scheduler.

localrules defines rules that are executed on the Snakemake head node.

$PATH must be passed to Snakemake rules.

nohup <command> & prevents <command> from exiting when you log off.

Token files can be used to take the place of output files if none are created.

snakemake --dag | dot -Tsvg > dag.svg creates a graphic of your workflow.

snakemake --gui opens a browser window with your workflow.