NGS Variant Calling Pipeline — Human (NA12878, chr22 region)

End-to-end, reproducible variant calling pipeline built from raw paired-end Illumina reads through to a benchmarked, filtered VCF — including precision/recall/F1 evaluation against the Genome in a Bottle (GIAB) gold-standard truth set.

Note on the dataset: the goal was human chr20, but this sandboxed environment doesn't have direct network access to NCBI/UCSC/ENA. Instead I used real Illumina HiSeq X sequencing data from NA12878 (the GIAB reference individual) covering a small chr22 region, sourced from the open-source freebayes test suite. This turned out to be a better teaching example: it ships with a matching GIAB truth VCF, so the pipeline can be properly benchmarked rather than just "run and hope." Swapping in a different chromosome/region only requires replacing the files in data/.

Raw FASTQ → QC → Alignment (BWA-MEM) → Sort/Index → Variant Calling
   (bcftools mpileup/call) → Filtering → Benchmarking (vs GIAB) → Visualization (R)

Results at a glance

Metric	Value
Read pairs	1,633
Reads mapped	99.79%
Properly paired	99.57%
Raw variants called	16
Filtered variants (QUAL≥20, DP≥5)	16
SNPs / Indels	14 / 2
Precision vs GIAB	0.875
Recall vs GIAB	1.000
F1 score vs GIAB	0.933

Full walkthrough with commands and output for every stage: docs/

Repository structure

.
├── data/
│   ├── raw/            # Source BAM + extracted paired-end FASTQ reads
│   └── reference/       # Reference FASTA + BWA index
├── scripts/             # Numbered, reproducible pipeline scripts
│   ├── 00_setup_environment.sh
│   ├── 01_get_data.sh
│   ├── 02_quality_control.sh
│   ├── 03_alignment.sh
│   ├── 04_variant_calling.sh
│   ├── 05_benchmark_giab.sh
│   └── 06_visualize_variants.R
├── results/
│   ├── qc/              # FastQC reports
│   ├── alignment/        # Sorted/indexed BAM + flagstat
│   ├── variants/         # Raw + filtered VCFs, bcftools stats
│   ├── benchmarking/      # GIAB truth VCF, isec output, precision/recall/F1
│   └── plots/            # Visualizations (R/ggplot2)
└── docs/                 # Step-by-step write-up with commands and results

Tools used

Tool	Purpose
BWA-MEM	Read alignment to reference
samtools	BAM sorting, indexing, stats, FASTQ conversion
bcftools	Variant calling (mpileup/call), filtering, benchmarking
FastQC	Raw read quality control
GIAB	Gold-standard truth set for benchmarking
R / ggplot2	Result visualization

How to reproduce

bash scripts/00_setup_environment.sh   # install tools
bash scripts/01_get_data.sh            # fetch reads + reference + truth set
bash scripts/02_quality_control.sh     # FastQC
bash scripts/03_alignment.sh           # BWA-MEM alignment + sort/index
bash scripts/04_variant_calling.sh     # bcftools mpileup/call + filter
bash scripts/05_benchmark_giab.sh      # precision/recall/F1 vs GIAB
Rscript scripts/06_visualize_variants.R results/variants/filtered_variants.vcf results/plots

Each script is idempotent and can be re-run independently once its inputs exist.

Author

Harshita