Skip to content

Highlight each locus of a gene name in scatter --gene - #1177

Merged
etal merged 1 commit into
masterfrom
etal/fix/scatter-gene-per-locus
Aug 18, 2026
Merged

etal merged 1 commit into
masterfrom
etal/fix/scatter-gene-per-locus

Conversation

@etal

@etal etal commented Aug 18, 2026

Copy link
Copy Markdown
Owner

CopyNumArray.by_gene groups a gene's bins per locus, but plots.gene_coords_by_name still reported a name's position as the minimum start and maximum end over every bin naming it anywhere in the array. CNVkit therefore held two answers to "where is gene X":

$ cnvkit.py genemetrics test/formats/wgs-chr17.cnr -t 0 | grep -c Y_RNA
26
$ cnvkit.py scatter test/formats/wgs-chr17.cnr -g Y_RNA
Showing 160038 probes and 1 selected genes in region chr17:3046728.0-81407638.0

One gold band 76 Mb wide, asserting that every bin between the first and last Y_RNA occurrence is Y_RNA.

What the fix does

gene_coords_by_name now walks the same runs as by_gene, through the newly public cnvlib.cnary.gene_runs, and returns one region per locus. A locus spans its run's first and last bin, which is precisely the start and end genemetrics publishes for it (reports.group_by_genes emits rows[0] with end = rows.end.iat[-1]), so the two surfaces now agree by construction rather than by coincidence: the 26 loci genemetrics reports for Y_RNA are exactly 26 of the 29 regions scatter highlights, coordinate for coordinate. The three it omits are single-bin loci dropped by its own -t threshold.

The per-locus rule also retires the assumption that a name occupies one chromosome. core.check_unique asserted that, and its AssertionError: Inconsistent DUPGENE keys: chr1 chr2 fired before scatter could report the ambiguity — which is why select_range_genes' own "split across chromosomes" error had been unreachable since 2014. The assertion and its now-unused definition are gone; the ambiguity is resolved by selecting a chromosome with -c, or refused with a message naming both candidates.

Numerical effect on plotted regions

Every gene name in both fixtures, old rule (reimplemented inline) against new:

fixture names span unchanged span changed
test/formats/wgs-chr17.cnr 2276 2270 6
test/formats/amplicon.cnr 79 79 0

All six that move are repeat families, and each shrinks from one span to a set of loci:

name was now
snoU13 1 region, 79,133,335 bp 16 regions, 2,785,752 bp
Y_RNA 1 region, 76,360,910 bp 29 regions, 3,392,141 bp
U3 1 region, 14,391,943 bp 3 regions, 487,087 bp
Vault 1 region, 7,079,244 bp 2 regions, 270,544 bp
SNORA70 1 region, 2,164,137 bp 2 regions, 626,062 bp
SNORA74 1 region, 1,614,279 bp 2 regions, 257,476 bp

cnvkit.py scatter test/formats/amplicon.cnr -g ERBB2 renders a byte-identical PNG before and after (md5 a27dbfd2c98d737c02a9ee4ecd5ac9a5), the control case for the whole class of single-locus genes.

Behaviour changes to be aware of

  • A gene name on several chromosomes now needs one chosen with -c; without it, the refusal names the chromosomes rather than dying in an assert.
  • With an explicit region, only the loci inside it are highlighted, and how many were left out is logged. A named gene with no locus in the region is still refused with the pre-existing message.
  • -g '' alongside -c no longer raises KeyError('popitem(): dictionary is empty'). gene_names was a filter object, always truthy, so the empty-gene-list branch documented in doc/plots.rst was unreachable.
  • A placeholder name such as Antitarget, Background, - or . is no longer findable by -g, since the grouping honours params.IGNORE_GENE_NAMES. It previously plotted a span from the first such bin to the last. genemetrics has always hidden these.
  • A locus straddling the region boundary is dropped rather than aborting the plot, when another locus of the same gene lies inside. The containment test is unchanged; only its consequence for the remaining loci is.

Alternatives measured and rejected

Implementing the lookup on by_gene() itself was the obvious route and costs 1750 ms per call on wgs-chr17.cnr (165,626 bins) against 65 ms for the superseded targeted scan, because it materializes a CopyNumArray for every group on the chromosome merely to read two coordinates. Sharing gene_runs instead costs 147 ms, so scatter --gene keeps its latency and there is still exactly one grouping rule. Moving the lookup into cnary as a CopyNumArray method, as the deleted XXX comment mused, would carry the requested-name filtering and label joining across a layer boundary in the wrong direction; those are presentation concerns and stay in plots.

Left for later

plots.gene_coords_by_range, 25 lines further down the same module, still applies the first-to-last rule within its window: over the whole of chr17 in wgs-chr17.cnr it returns a single Y_RNA region at 7,440,413-46,556,560, a 39 Mb band, so scatter -c chr17 and scatter -c chr17 -g Y_RNA still disagree. That body also carries the comma-joined pseudo-gene defect (ERBB2,MIR4728 returned as one label), which needs a decision about tabular breaks output before either can be fixed; both live in the same twelve lines and one rewrite over gene_runs settles them together.

Tests

test/test_plots.py gains four cases in GeneCoordsTests and a new ScatterSelectionTests class of seven. Run against the previous commit, ten of them fail and four pass, so each guard bites the defect it names. The per-locus case includes a placeholder bin interrupting a run, which is the only cheap way to pin that the ignore set is honoured; one case cross-checks all 29 real Y_RNA loci against by_gene on the fixture. Full suite: 721 passed, 147 subtests.

CopyNumArray.by_gene groups a gene's bins per locus, but plots.gene_coords_by_name
still reported a name's position as the minimum start and maximum end over every
bin naming it anywhere in the array. So CNVkit held two answers to "where is gene
X": genemetrics reported Y_RNA at 26 loci on chr17 of test/formats/wgs-chr17.cnr
while scatter --gene Y_RNA painted one gold band 76 Mb wide across all of them.

gene_coords_by_name now walks the same runs, via the newly public
cnvlib.cnary.gene_runs, and returns one region per locus. Each locus spans its
run's first and last bin, which is the start and end genemetrics publishes for
it, so the two surfaces agree by construction: the 26 loci genemetrics reports
for Y_RNA are exactly 26 of the 29 regions scatter now highlights, coordinate for
coordinate, the other three being single-bin loci its threshold drops. Genes at
one locus are untouched -- 2270 of 2276 names in wgs-chr17.cnr and all 79 in
amplicon.cnr keep their previous span, and the six that move are all repeat
families.

The per-locus rule also removes the assumption that a name occupies one
chromosome, which core.check_unique asserted before scatter could report the
ambiguity, so that assertion and its now-unused definition are gone. A name on
several chromosomes is instead resolved by selecting one with -c, or refused with
a message naming both; a name whose loci lie outside an explicit region is
refused as before, but loci inside it are now highlighted individually. Two
lesser consequences: -g '' alongside -c no longer raises KeyError from popitem,
as doc/plots.rst has always promised, and a placeholder name such as Antitarget
is no longer findable by -g, matching genemetrics.
@codecov

codecov Bot commented Aug 18, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 97.56098% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 76.25%. Comparing base (0431f17) to head (f47647a).
⚠️ Report is 4 commits behind head on master.

Files with missing lines Patch % Lines
cnvlib/scatter.py 96.15% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1177      +/-   ##
==========================================
+ Coverage   76.08%   76.25%   +0.17%     
==========================================
  Files          75       75              
  Lines        8421     8428       +7     
  Branches     1489     1492       +3     
==========================================
+ Hits         6407     6427      +20     
+ Misses       1578     1568      -10     
+ Partials      436      433       -3     
Flag Coverage Δ
unittests 76.25% <97.56%> (+0.17%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@etal
etal merged commit 8e025ea into master Aug 18, 2026
15 of 16 checks passed
@etal
etal deleted the etal/fix/scatter-gene-per-locus branch August 18, 2026 01:04
eskutkaan pushed a commit to eskutkaan/cnvkit that referenced this pull request Sep 15, 2026
get_gene_intervals tallied each bin's whole gene field, so a bin labeled
with two overlapping genes became a third gene that does not exist, and
five such names reached the released breaks table on
test/formats/amplicon.cnr: APC,CTC-554D6.1, ATM,C11orf65, GOPC,ROS1,
CDKN2A,RP11-145E5.5 and KDR,RP11-530I17.1. Anyone filtering that output
for CDKN2A found nothing, though the documented use of the gene column is
exactly that pipeline. It also condensed every occurrence of a name on a
chromosome into one interval, so a segment boundary in the gap between two
loci of a repeat-family name was reported as a breakpoint within the gene,
counting one locus's bins on the left and another's on the right.

Both follow from the same 2014 tally, and both are fixed by grouping bins
into loci with cnvlib.cnary.gene_runs, as by_gene and genemetrics have done
since etal#1173 and scatter --gene since etal#1177. A bin naming several genes now
belongs to each of their loci and is reported once per gene, the rule
adopted for the sibling gainloss report in etal#107.

probes_left and probes_right now count every bin in the locus's run,
including off-target and backbone bins it absorbs: a copy number alteration
affects a genomic region regardless of the gene model, so those bins support
a breakpoint as well as a targeted bin does. This is what genemetrics
already counts in its probes column, and the two sides now sum to it
exactly, where 19 of 58 amplicon rows disagreed with it before.

On amplicon the table goes from 49 rows to 58 and no gene value contains a
comma; real genes recover the bins the invented names held, e.g. APC at
chr5:112090792 from (2, 10) to (2, 39). On a comma-free panel with a SNP
backbone, test/formats/regression/p2-9_2.cnr, the same 42 breakpoints are
reported and two rows' counts change, from absorbing bins labeled CGH.
Against 83 block-mean segments over test/formats/wgs-chr17.cnr, 93 rows
become 218, the 48 comma-joined names become none, and Y_RNA's 38 phantom
cross-locus rows become 6 real ones.
eskutkaan pushed a commit to eskutkaan/cnvkit that referenced this pull request Sep 15, 2026
gene_coords_by_range keyed on a bin's whole gene field, so the label for a
region containing the co-binned pair ERBB2,MIR4728 named a gene that does
not exist, and ERBB2 was highlighted twice: once at its own locus and once
as a nested box under the joined name, with a gap where its co-binned bins
had been excluded. MIR4728 was never labeled at all. The function also
extended a name's region from its first occurrence in the window to its
last, so scatter -c chr17 on test/formats/wgs-chr17.cnr drew Y_RNA as a
single 39.1 Mb stripe while scatter -c chr17 -g Y_RNA, rewritten in etal#1177,
drew its 29 loci separately -- one command, two answers to where a gene is.

Both are fixed by grouping bins with cnvlib.cnary.gene_runs, so the region
form now locates genes as --gene and genemetrics do. Genes whose runs
coincide exactly, as overlapping genes labeled in one bin do, share a
stripe under a joined label rather than stacking identical stripes and
overprinting their labels; that consolidation, and the placeholder-name
guard, are now the one _join_labels helper both entry points call.

On amplicon chr17 the labels go from [ERBB2, 'ERBB2,MIR4728'] to
[ERBB2, MIR4728]; on chr5 APC becomes one contiguous locus,
112,043,331-112,179,950, alongside CTC-554D6.1's own span. Over all of
wgs-chr17 the widest region falls from 39.1 Mb to 1.18 Mb.

The ignore parameter is also no longer widened in place: passing a list
grew the caller's own list by two entries per call.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant