Configuring the workflow

Configuring the workflow#

Before running AmpSeeker, we need to select which analyses we want to run (configuration). This is done by editing the file config.yaml in the config directory. The config file contains a number of options and parameters.

If you have any issues configuring the pipeline, please watch the video walkthrough first, and raise an issue on github or email me.

Configuration Parameters#

dataset: lab-strains
panel: ag-vampir
platform: illumina
ploidy: 2

  • dataset: A name for your dataset. This will be used to name output files and directories.

  • panel: The name of the amplicon panel used. Currently, the workflow has special support for the “Ag-vampIR” panel (Anopheles gambiae vector amplicon marker panel for Insecticide Resistance). If not using Ag-vampIR, panel name can be anything.

  • platform: The sequencing platform used. Options are illumina for Illumina sequencing (default) or nanopore for Nanopore sequencing. This determines the alignment and variant calling pipeline used.

  • ploidy: Sample ploidy. Use 1 for haploid organisms (e.g. Plasmodium, bacteria, the haploid stage of fungi), 2 for diploid organisms (default, e.g. most animals including Anopheles), or a higher integer for polyploids (e.g. 4 for tetraploid wheat). Polyploid (>2) is supported for Illumina data only — Nanopore is restricted to ploidy 1 or 2 because clair3 has no polyploid mode.

Cohort Analysis#

cohort-columns:
  - location
  - taxon

  • cohort-columns: List of metadata columns used to define cohorts for downstream analyses. Here, a cohort means a group of samples sharing the same value in a metadata column, for example all samples with location = Obuasi or all samples with taxon = gambiae. These columns are used to colour samples in plots and to calculate cohort-level summaries such as allele frequencies and genetic diversity. Column names must match the metadata header exactly, including capitalization. For reliability, use simple column names beginning with a letter and containing only letters, numbers, and underscores (for example location, taxon, country, collection_year). Avoid spaces and punctuation in cohort column names.

Input Files#

targets: config/ag-vampir.bed
metadata: config/metadata.tsv

  • targets: Path to a BED file containing the genomic coordinates of amplicon targets. This file should have 5 columns: chromosome, start, end, amplicon_id, and target_label with no header. See the input_data.ipynb for more details.

  • metadata: Path to a TSV file containing sample metadata. At minimum, this file should have a “sample_id” column. Any columns listed under cohort-columns must also be present in this file with exactly the same spelling and capitalization as the metadata header. Only required if working directly from fastq files (from-bcl: False).

Reference Genome#

reference-fasta: resources/reference/Anopheles-gambiae-PEST_CHROMOSOMES_AgamP4.fa
reference-gff3: resources/reference/Anopheles-gambiae-PEST_BASEFEATURES_AgamP4.12.gff3
reference-snpeffdb: Anopheles_gambiae
custom-snpeffdb: False

  • reference-fasta: Path to the reference genome FASTA file (can be gzipped with .fa.gz extension).

  • reference-gff3: Path to the genome annotation file in GFF3 format.

  • reference-snpeffdb: Name of the SnpEff database to use for variant annotation. This should match a database name available in SnpEff (e.g., “Anopheles_gambiae”).

  • custom-snpeffdb: Whether to build a custom SnpEff database. Set to True if the reference genome is not available in the standard SnpEff databases.

Illumina Directory#

illumina-dir: resources/250110_M05658_0028_000000000-LTBV4

  • illumina-dir: Path to the Illumina MiSeq run directory containing BCL files. This is only required if converting from BCL to FASTQ (from-bcl: True). If you already have FASTQ files, this can be left empty.

Input File Type#

from-bcl: True
fastq:
  auto: True

  • from-bcl: Whether to convert BCL files to FASTQ files. If True, the pipeline will use the illumina-dir path to find BCL files and convert them to FASTQ format. Only applicable for Illumina data.

  • fastq.auto: If True, the pipeline expects FASTQ files to be in resources/reads/ directory with the naming pattern {sample_id}_1.fastq.gz and {sample_id}_2.fastq.gz for Illumina, or {sample_id}.fastq.gz for Nanopore. If False, the pipeline expects the metadata file to have columns fq1 and fq2 (Illumina) or fq1 (Nanopore) specifying the paths to the FASTQ files.

Quality Control#

quality-control:
  sample-min-mean-target-coverage: 5

  coverage: True
  stats: True
  multiqc: True

  • quality-control.sample-min-mean-target-coverage: Minimum mean coverage across panel targets required for a sample to pass QC. Samples below this threshold are removed at the quality control stage. The default and recommended starting value is 5, corresponding to a minimum mean target coverage of 5x per sample.

  • quality-control.coverage: Whether to calculate and report coverage statistics for each sample.

  • quality-control.stats: Whether to generate alignment and variant calling statistics.

  • quality-control.multiqc: Whether to generate a MultiQC report aggregating various QC metrics.

Analysis#

analysis:  
  sample-map: False # needs lat and longs in metadata/sample_sheet
  population-structure: True
  genetic-diversity: True
  allele-frequencies: True

  • analysis.sample-map: Whether to generate a geographic map of sample collection locations. Requires latitude and longitude columns in the metadata file.

  • analysis.population-structure: Whether to perform population structure analysis using principal component analysis (PCA).

  • analysis.genetic-diversity: Whether to calculate genetic diversity metrics (such as nucleotide diversity).

  • analysis.allele-frequencies: Whether to calculate and visualize allele frequencies across samples and groups.

Jupyter Book#

build-jupyter-book: True

  • build-jupyter-book: Whether to compile all analysis notebooks into a Jupyter Book for convenient browsing of results. The book will be available at results/ampseeker-results/_build/html/index.html.

Additional analyses for Ag-vampIR Panel#

When using the Ag-vampIR panel (panel: ag-vampir), the workflow automatically enables additional analyses:

  1. Species Identification: Identifies Anopheles species using amplicon sequencing data (results in results/notebooks/ag-vampir/species-id.ipynb).

  2. Kdr Analysis: Analyzes knockdown resistance mutations in the voltage-gated sodium channel (results in results/notebooks/ag-vampir/kdr-analysis.ipynb).

Example Configuration#

Here’s a complete example configuration for reference:

Illumina Configuration Example#

# Dataset and panel information
dataset: lab-strains
panel: ag-vampir
platform: illumina
ploidy: 2          # 1 = haploid, 2 = diploid (default), >2 = polyploid (Illumina only)
cohort-columns:
  - location
  - taxon
  - country
targets: config/ag-vampir.bed
metadata: config/metadata.tsv

# Reference genome information
reference-fasta: resources/reference/AgamP4.fa
reference-gff3: resources/reference/AgamP4.gff
reference-snpeffdb: Anopheles_gambiae
custom-snpeffdb: False

# Illumina directory (if using BCL files)
illumina-dir: resources/250110_M05658_0028_000000000-LTBV4
# Input file type options
from-bcl: True
fastq:
  auto: True

# Quality control options
quality-control:
  sample-min-mean-target-coverage: 5
  fastp: True
  coverage: True
  stats: True
  multiqc: True

# Analysis options
analysis:  
  sample-map: False
  population-structure: True
  genetic-diversity: True
  allele-frequencies: True

# Build Jupyter book
build-jupyter-book: True

Nanopore Configuration Example#

# Dataset and panel information
dataset: nanopore-study
panel: custom-panel
platform: nanopore
ploidy: 2          # Nanopore supports 1 (haploid) or 2 (diploid) only
cohort-columns:
  - location
  - taxon
targets: config/custom-panel.bed
metadata: config/metadata.tsv

# Reference genome information
reference-fasta: resources/reference/AgamP4.fa
reference-gff3: resources/reference/AgamP4.gff
reference-snpeffdb: Anopheles_gambiae
custom-snpeffdb: False

# Input file type options (no BCL conversion for Nanopore)
Illumina-dir: ""
from-bcl: False
fastq:
  auto: True

# Quality control options
quality-control:
  sample-min-mean-target-coverage: 5
  fastp: True
  coverage: True
  stats: True
  multiqc: True

# Analysis options
analysis:  
  sample-map: False
  population-structure: True
  genetic-diversity: True
  allele-frequencies: True

# Build Jupyter book
build-jupyter-book: True

Haploid Configuration Example#

For a haploid organism (e.g. Plasmodium falciparum), set ploidy: 1. All downstream analyses (variant calling, PCA, allele frequencies) will treat samples as haploid:

dataset: pf-field-samples
panel: custom-panel
platform: illumina
ploidy: 1
# ... remaining configuration as above

Polyploid Configuration Example#

For a polyploid organism (e.g. tetraploid Triticum), set ploidy to the relevant integer. Polyploid is supported for Illumina data only:

dataset: wheat-survey
panel: custom-panel
platform: illumina
ploidy: 4
# ... remaining configuration as above