Skip to content

Add seq2seq / text-generation task family #285

Open

Description

@stanlrt

Add a seq2seq / text-generation task family, with BLEU (token-sequence metric) as the named POC consumer that proves the new output structure flows end-to-end.

Context

Task families live in src/raitap/task_families/ (base.py, registry.py); existing: classification, detection. A task family defines how model outputs are interpreted, how predictions/targets are formed, and which metrics/visualisers apply. seq2seq (encoder-decoder / autoregressive generation) is a distinct output structure (token sequences, not class logits).

Scope

  • A seq2seq task family: output = generated token sequence(s); define prediction extraction, reference/target handling.
  • Wire it into the task-family registry; capability/compat gating for seq2seq.
  • Decide the supported explanation surface (token-level attribution links to Add batch-aware text attribution visualisation #99 text attribution visualisation) and which explainers/assessors are compatible.

POC consumer

  • BLEU (or exact-match as the minimal variant) is the concrete POC metric: a small seq2seq model generates sequences and BLEU is computed against references end-to-end. This proves the token-sequence output structure threads through prediction extraction -> metric. Add batch-aware text attribution visualisation #99 token-level attribution is the deeper transparency consumer, follow-on.

Acceptance

  • A small seq2seq model runs through the pipeline producing generations + a BLEU score end-to-end.
  • Compat gates correctly include/exclude methods for the seq2seq output shape.
  • Tests + docs.

Relationships

Activity

  1. added this to the P2: Medium milestone on Jun 7, 2026
  2. stanlrt commented on Jun 23, 2026

    @stanlrt
    CollaboratorAuthor

    Prerequisite: #340 builds the text input modality (InputModality.text + data-layer load branch + tokenisation seam) that this issue assumes. seq2seq output structure + BLEU stay this issue's scope; the input path lands in #340 first.

  3. stanlrt commented on Jul 5, 2026

    @stanlrt
    CollaboratorAuthor

    Prerequisite landed: the text input modality is implemented in #340 (PR #347) — InputModality.text, the data/inputs parser registry, HF tokeniser loading via model.tokenizer, multi-tensor forward threading attention_mask, and token attribution via LayerIntegratedGradients/CaptumTextVisualiser. Text classification now runs end-to-end (contributor-configs/text-classification-sst2/). This issue (seq2seq output structure + its family-owned load_inputs for encoder/decoder assembly + BLEU) can now build on that input path. The #340 non-goals held: the litmus was that seq2seq adds a task family + load_inputs without editing the text loader or InputModality.text.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions