New pipeline: nf-core/plasmodiumpopgen

[kathrynmurie]

Pipeline title/name

plasmodiumpopgen

Keywords

genomics, microhaplotype, population, IBD, relatedness, plasmodium

What is it about?

This pipeline will provide a standardized workflow for population-genetic analysis of Plasmodium microhaplotypes generated from targeted sequencing data.

The pipeline will estimate commonly used metrics for Plasmodium, pop gen including:

• Complexity of infection (COI)
• Proportion of polyclonal infections
• Between-host relatedness (IBD)
• Within-host relatedness
• Allele frequencies
• Per-target summary

Per-target summary statistics will include:

• Total and unique allele counts
• Singleton alleles
• Nucleotide diversity
• Number of segregating sites
• Tajima's D

The pipeline will support interoperable tools that users can select through command-line parameters, allowing methods to evolve as the field develops.

Plasmodium targeted sequencing presents a distinct pop gen problem because blood-stage parasites are haploid, while infections are frequently polyclonal and therefore contain unphased mixtures of multiple parasite genomes. Core quantities such as COI, within-host relatedness, allele frequencies, and between-host IBD require methods that explicitly account for this structure.

Although other eukaryotic pathogens can share individual biological features such as recombination or mixed infections, the combination of methods, defaults, benchmarking, and interpretation in this pipeline is designed and validated specifically for Plasmodium. Several core tools, including MOIRE and Dcifer, were developed specifically for inference from polyclonal malaria infections.

The pipeline design and method selection have been guided by the Plasmodium genomics community through the PlasmoGenEpi Network, including extensive benchmarking of methods.

Please provide a schematic diagram of the proposed pipeline

What would a minimal first release of this pipeline include?

The initial release would:

1. Split samples into user-defined populations for parallel analysis.
2. Estimate COI using a naive estimator or MOIRE.
3. Optionally...
   a. Estimate within-host relatedness and allele frequencies using MOIRE.
   b. Estimate between-host relatedness using Dcifer.
4. Generate per-locus statistics using PGEcore.
5. Collate results into standardized outputs.

This would provide an end-to-end workflow from processed targeted-sequencing/microhaplotype data to the principal metrics currently used for population-level analysis of Plasmodium.

I confirm my proposed pipeline will follow nf-core guidelines. Most importantly, my pipeline will:

• be built with Nextflow.
• pass nf-core lint tests and use standardized parameters.
• be community-owned and developed within the nf-core organization.
• open source under the MIT license with proper credits and acknowledgments.
• have a descriptive, all lowercase, and without punctuation name.
• use the nf-core pipeline template and predominantly use official nf-core modules.
• focus on a specific data/analysis type with appropriate scope.
• have properly maintained documentation.
• be bundled using versioned Docker/Singularity containers.

Why do we need a new pipeline?

These analyses are commonly performed across the Plasmodium community, but there is currently no standardized nf-core workflow for running them consistently and at scale. Researchers instead have to connect individual tools using custom scripts, making analyses harder to reproduce and compare across studies.

The proposed pipeline does not introduce new methodology. Its value is in integrating established Plasmodium methods into a reproducible, scalable, community-maintained workflow with standardized inputs and outputs.

Importantly, this is not a generic population-genetics workflow applied to Plasmodium. A sample from a malaria infection can contain multiple genetically distinct clones. The observed alleles therefore represent a mixture of parasites, and important properties of those parasites must be inferred without necessarily being able to reconstruct the individual strains through phasing.

COI, allele-frequency estimation, within-host relatedness, and between-host IBD are consequently interconnected inference problems. The pipeline brings together methods developed specifically to make these population-level inferences from polyclonal Plasmodium infections.

There is currently no nf-core pipeline designed for this type of population-level analysis of Plasmodium targeted sequencing data.

Who would be interested?

Researchers working with Plasmodium targeted sequencing data. We hope this would eventually be useful in public health, although substantial work still needs to be done to translate these metrics to interpretation.

What has been done so far

We already have a prototype implementing some of this workflow in a repo called plasmodiumtransmissionintensity. It currently uses a large shared Docker image, we haven't filled in many parts of the template, and we want to change the name to plasmodiumpopgen. Therefore, we think starting from a fresh nf-core template and migrating the existing functionality will be the simplest path forward.

We actually proposed this pipeline to nf-core a year or so ago, and the proposal was approved as sufficiently distinct to warrant a new pipeline. However, we never settled on a pipeline name, so a dedicated nf-core channel was never created.

In the meantime, we focused on plasmodiumdrugres. We now have a dedicated team with capacity to bring plasmodiumpopgen through to an initial release over the next couple of months. Given the time since the original proposal and a conversation on slack, we're putting in a new proposal.

The PlasmoGenEpi Network has also conducted substantial benchmarking of the methods included in the pipeline and we will be releasing a preprint describing the benchmarking in the next couple of months.

URL to existing work (if applicable)

https://github.com/PlasmoGenEpi/plasmodiumtransmissionintensity/tree/dev

Are there any similar existing nf-core pipelines?

plasmodiumdrugres

[nf-core-bot]

Approval status: ✅ Approved

Required approvals: Either 2 core team members OR 1 core team member + 1 maintainer

Review Status	Team members
✅ Approved (Core)	@ewels, @maxulysse
🕐 Pending (Core)	@mribeirodantas, @mashehu, @JoseEspinosa, @mirpedrol, @matthdsm, @FriederikeHanssen, @kenibrewer, @jfy133, @atrigila, @edmundmiller, @awgymer, @SPPearce, @sateeshperi, @FranBonath, @LouisLeNezet, @christopher-hakkaart, @nvnieuwk
🕐 Pending (Maintainer)	@pontus, @maxulysse, @lpantano, @pinin4fjords, @mashehu, @JoseEspinosa, @mirpedrol, @GallVp, @FloWuenne, @matthdsm, @FriederikeHanssen, @adamrtalbot, @mahesh-panchal, @jfy133, @atrigila, @prototaxites, @ramprasadn, @edmundmiller, @luisas, @SPPearce, @camlloyd, @vagkaratzas, @sateeshperi, @jonasscheid, @famosab, @nictru, @itrujnara, @LouisLeNezet, @Joon-Klaps, @jasmezz, @Felix-Kummer, @sofstam, @nvnieuwk

[kathrynmurie]

@a-hubbard @KirstyMcCann @gtollefson FYI

[kathrynmurie]

Copying my response to bot on slack to here

The core development team will consist of myself, Alfred Hubbard, Kirsty McCann, and George Tollefson. The project will be supported by the wider PlasmoGenEpi community, who will likely contribute in maintenance. We will make modules where needed

[mribeirodantas]

Hi @kathrynmurie

Is it possible that this pipeline could apply to other species eventually, though your initial work will be supporting plasmodium?

[kathrynmurie]

Hi @mribeirodantas

Thanks for taking a look!

I really don't think so. If anything, I think this will become more specific to plasmodium because we would love to go further with some more interpretation

[mribeirodantas]

I see. Do you mind getting into more detail about the reasons why? Could you for example please highlight the tools/approach/steps that are exclusive/specific to Plasmodium?

[jfy133]

@kathrynmurie please could you look through: #134

I know it's a very scary long thread, but it's sort of important: we were getting concerned that the proposal there was actually quite generic based on the tools, and pushed to have it designed as more generic, and set the pathogen specific parameters/steps/databases as options rather than the whole thing.

It would be important to hear:

• How plasmodium specific are the tools in this proposal (duplicating what Marcel says above, but more detail would be nice)
• What do you mean by bespoke code? Things that will be developed and put into a dedicated package (we don't want to be responsible for large number sof custom scripts etc)
• How much overlap is with the proposal for the new pathogenepidemiology pipeline

[a-hubbard]

Hi @jfy133 I've tried to address your questions below. Let me know if this makes sense!

Several of the tools used in this pipeline (MOIRE, Dcifer, SNP-Slice, THEREALMcCOIL) are statistical models for estimating population genetic metrics in eukaryotic pathogens with a high prevalence of polyclonal (i.e., multi-strain) infections. This type of organism requires specific tools because 1) with non-pathogen eukaryotes you never have multiple organisms represented in one sample and 2) sexual recombination precludes the use of methods common in non-eukaryotic pathogens, most notably phylogenetic trees (at the sample level).

Admittedly, there are other eukaryotic pathogens with polyclonal infections aside from plasmodium (trypanosomes, for example), but the vast majority of data and research is on plasmodium. I think keeping the name and focus specific to plasmodium has several benefits:

1. It will be more discoverable for the people likely to use it. Broadening the name to eupathpopgen or something like that will make it harder for malaria researchers to find the pipeline. Conversely, I suspect people doing genomics for, e.g., trypanosomes, know that plasmodium tools might apply and will know to look for that.
2. It correctly implies that the tools were designed and validated for plasmodium. They might work well for other eukaryotic pathogens, but no one has tested that. We don't want to give this hypothetical trypanosome researcher the impression that this pipeline has been robustly tested for their system when in fact it hasn't been.
3. Interpreting the outputs of these tools is quite difficult, and we want to include guidance on doing so. This guidance will be completely plasmodium specific, as there is no validation work for other organisms.

By bespoke code, we mean things that aren't already a published method. These things will all be part of one R package (PGEcore) that will be a dependency of the pipeline.

The goals of this pipeline overlap with that of pathogenepidemiology, but everything else is quite different:

1. Input data: They are using WGS, while we are using targeted deep sequencing data. The latter has a lot of benefits in the case of polyclonal infections and needs custom statistical tools.
2. Scope: pathogenepidemiology includes allele calling. plasmodiumpopgen is purely population genetics, with allele calling already done.
3. Applicability to malaria: pathogenepidemiology does seem to be intended to be more broad-purpose, but possibly at the cost of validity for malaria research. Specifically, I see they incorporate phylogenetic trees. As mentioned above, these do not give meaningful results for malaria at the sample level due to the high rate of recombination.

[ewels]

Thanks for the thoughtful response @kathrynmurie @a-hubbard, much appreciated.

This proposal is very difficult for us because it's effectively going directly against the founding principles of nf-core: making pipelines as general as possible to make them as widely useful as possible, and with the largest pool of maintainers possible. In the past, we have rejected comparable proposals for exactly this reason. That said, your arguments are compelling and the value of the pipeline is clear.

My gut feeling is that we can approve if you are willing to generalise the name a bit. To respond to your points:

1. The pipeline keywords, description, readme and documentation can all mention plasmodium, I'm not convinced that people will be unable to find the pipeline if that specific term is not in the name itself.
2. I don't think that current levels of validation are a reason to gate-keep it as plasmodium only. Others may want to validate for other organisms in the future - you do not need to be responsible for this. I think it would be sufficient to simply say in the docs that it was designed and validated for plasmodium but should be applicable for other organisms (at the user's own risk).
3. I think it's fine to include plasmodium-specific documentation. Again, I don't see a reason for the entire pipeline being scoped to accommodate this. Documentation can have arbitrary numbers of sub-pages (see eg. sarek) and so the top-level docs can be brief and general, with much more detailed usage guides dedicated to plasmodium linked to and available.

What we're trying to avoid is someone else proposing a trypanisomepopgen pipeline in a years' time (or whatever organism). It is fine and quite normal for pipelines to be built with a narrow scope in mind, but if possible it's good to make sure that we don't unnecessarily close doors for future development if we can. As long as it doesn't impose an intolerable cost in terms of complexity or functionality.

Regarding pathogenepidemiology, differing input data type is a good reason for separate pipelines. Excessive outputs is not: that falls under a similar argument as above, skipping non-relevant analysis types is easy and can be documented / pre-configured in different configuration profiles.

To give some examples of comparable historical pipelines:

• nf-core/sarek was written for human tumour vs normal analysis. Once released it has expanded into also being used for genome variant calling, and is used for organisms as wide and varied as yeast to e.coli.
• nf-core/methylseq was build for whole-genome bisulfite sequencing, but now works for multiple different data types and tool choices. Different -profile settings bundle configs for different use cases
• nf-core/viralrecon was build for covid analysis during the pandemic. We kept it generic and it's also been used for monkeypox and a wide variety of other pathogens and data types.

In all these cases, the original authors had a fixed idea of what the pipeline would be used for. But other researchers found that the tools and pipeline were also useful for them. Over time the pipelines ended up serving much wider audiences, without ever losing the utility of the initial target.

Basically, we would prefer this:

nextflow run nf-core/eupopgen -profile plasmodium

to this:

nextflow run nf-core/plasmodiumpopgen

...if it can work. This is exactly the shape of discussion in the other proposal that James linked to.

Pretty much the only reasons I can think of not to do this, to stick to an organism-specific pipeline, is if the analysis tools and steps in the pipeline are primarily organism specific (it simply won't work for any other organism).

Sorry, bit of a monologue here. Does this train of thought make sense? Is there a way forward in any of what I say that would be acceptable to you?

Thanks 😊 🙏🏻

[kathrynmurie]

This all makes sense. How about eupathpopgen?

[ewels]

Works for me! Assuming that it's short for eukaryotic pathogen population genomics? Agree that nf-core/eukaryoticpathogenpopulationgenomics would be a bit of a mouthful 😅