Add seq2seq / text-generation task family #285
Copy link
Copy link
Open
Labels
Milestone
Description
Add a seq2seq / text-generation task family, with BLEU (token-sequence metric) as the named POC consumer that proves the new output structure flows end-to-end.
Context
Task families live in src/raitap/task_families/ (base.py, registry.py); existing: classification, detection. A task family defines how model outputs are interpreted, how predictions/targets are formed, and which metrics/visualisers apply. seq2seq (encoder-decoder / autoregressive generation) is a distinct output structure (token sequences, not class logits).
Scope
- A
seq2seqtask family: output = generated token sequence(s); define prediction extraction, reference/target handling. - Wire it into the task-family registry; capability/compat gating for seq2seq.
- Decide the supported explanation surface (token-level attribution links to Add batch-aware text attribution visualisation #99 text attribution visualisation) and which explainers/assessors are compatible.
POC consumer
- BLEU (or exact-match as the minimal variant) is the concrete POC metric: a small seq2seq model generates sequences and BLEU is computed against references end-to-end. This proves the token-sequence output structure threads through prediction extraction -> metric. Add batch-aware text attribution visualisation #99 token-level attribution is the deeper transparency consumer, follow-on.
Acceptance
- A small seq2seq model runs through the pipeline producing generations + a BLEU score end-to-end.
- Compat gates correctly include/exclude methods for the seq2seq output shape.
- Tests + docs.
Relationships
- Add batch-aware text attribution visualisation #99 (batch-aware text attribution visualisation) is the natural transparency consumer, follow-on.
- Likely needs the text input modality / tokenisation path; coordinate with existing text-modality work.
Reactions are currently unavailable
Activity
stanlrt commentedon Jun 23, 2026
Prerequisite: #340 builds the text input modality (
InputModality.text+ data-layer load branch + tokenisation seam) that this issue assumes. seq2seq output structure + BLEU stay this issue's scope; the input path lands in #340 first.stanlrt commentedon Jul 5, 2026
Prerequisite landed: the text input modality is implemented in #340 (PR #347) —
InputModality.text, thedata/inputsparser registry, HF tokeniser loading viamodel.tokenizer, multi-tensor forward threadingattention_mask, and token attribution viaLayerIntegratedGradients/CaptumTextVisualiser. Text classification now runs end-to-end (contributor-configs/text-classification-sst2/). This issue (seq2seq output structure + its family-ownedload_inputsfor encoder/decoder assembly + BLEU) can now build on that input path. The #340 non-goals held: the litmus was that seq2seq adds a task family +load_inputswithout editing the text loader orInputModality.text.