High Performance Computing

ncov-recombinant can alternatively be dispatched using the SLURM job submission system.

1. Create an HPC-compatible profile to store your build configuration.

   scripts/create_profile.sh --data data/custom --hpc
   

   2022-06-17 09:16:55     Searching for metadata (data/custom/metadata.tsv)
   2022-06-17 09:16:55     SUCCESS: metadata found
   2022-06-17 09:16:55     Checking for 3 required metadata columns (strain date country)
   2022-06-17 09:16:55     SUCCESS: 3 columns found.
   2022-06-17 09:16:55     Searching for sequences (data/custom/sequences.fasta)
   2022-06-17 09:16:55     SUCCESS: Sequences found
   2022-06-17 09:16:55     Checking that the metadata strains match the sequence names
   2022-06-17 09:16:55     SUCCESS: Strain column matches sequence names
   2022-06-17 09:16:55     Creating new profile directory (my_profiles/custom-hpc)
   2022-06-17 09:16:55     Creating build file (my_profiles/custom-hpc/builds.yaml)
   2022-06-17 09:16:55     Adding default input data (defaults/inputs.yaml)
   2022-06-17 09:16:55     Adding custom input data (data/custom)
   2022-06-17 09:16:55     Adding `custom` as a build
   2022-06-17 09:16:55     Creating system configuration (my_profiles/custom-hpc/config.yaml)
   2022-06-17 09:16:55     Adding default HPC system resources
   2022-06-17 09:16:55     System resources can be further configured in:
   
                           my_profiles/custom-hpc/config.yaml
   
   2022-06-17 09:16:55     Builds can be configured in:
   
                           my_profiles/custom-hpc/builds.yaml
   
   2022-06-17 09:16:55     The custom-hpc profile is ready to be run with:
   
                           scripts/slurm.sh --profile my_profiles/custom-hpc  
   

2. Edit my_profiles/custom-hpc/config.yaml to specify the number of jobs and default-resources to use.

   # Maximum number of jobs to run simultaneously
   jobs : 4
   
   # Default resources for a SINGLE JOB
   default-resources:
   - cpus=64
   - mem_mb=64000
   - time_min=720
   

3. Dispatch the workflow using the slurm wrapper script:

   scripts/slurm.sh --profile my_profiles/custom-hpc
   

   • Tip: Display log of most recent workflow: cat $(ls -t logs/ncov-recombinant/*.log | head -n 1)

4. Use the --help parameter to get additional options for SLURM dispatch.

   scripts/slurm.sh --help
   

   usage: bash slurm.sh [-h] [--profile PROFILE] [--conda-env CONDA_ENV] [--target TARGET] [--cpus CPUS] [--mem MEM]
   
           Dispatch a Snakemake pipeline using SLURM.
   
           Required arguments:
                   --profile PROFILE                Snakemake profile to execute (ex. profiles/tutorial-hpc)
   
           Optional arguments:
                   --conda-env CONDA_ENV            Conda environment to use. (default: ncov-recombinant)
                   --target TARGET                  Snakemake target(s) to execute (default: all)
                   --cpus CPUS                      CPUS to use for the main pipeline. (default: 1)
                   --mem MEM                        Memory to use for the ain pipeline. (default: 4GB)
                   -h, --help                       Show this help message and exit.