Skip to content

Sparse or zero-valid windows still participate in training #78

Description

@jeffreyHoelzel

The dataset yields all windows, including windows with no valid labeled residues. The trainer handles zero-valid masks by producing zero loss, but the optimizer still performs a step.

Evidence:

  • src/pepseqpred/core/data/proteindataset.py
    • ProteinDataset.__iter__ yields every window from _iter_windows.
    • The final mask may be all zero when a window contains only uncertain or padded residues.
  • src/pepseqpred/core/train/trainer.py
    • _batch_step creates zero loss when mask.sum() == 0.
    • Training still calls zero_grad, loss.backward(), and optimizer.step().

Why this can hurt:

  • Adam step counters still advance on no-information batches.
  • Learning-rate dynamics and optimizer moments can be affected without a meaningful gradient.
  • If some pathogens or proteins have many uncertain/unlabeled regions, they can consume training steps without learning signal.

Planning direction:

  • Add logging for zero-valid windows and zero-valid batches by split, fold, rank, and source.
  • Skip optimizer steps when a batch has zero valid residues.
  • Optionally filter zero-valid windows in ProteinDataset for training.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions