[stanlrt]

Summary

Adds a first-class text input modality end-to-end and proves it with a runnable text-classification use case. Implements #340.

Data layer

• InputModality.text (.txt / CSV column / JSONL), auto-detected kind="text" / layout="TOKENS" metadata.
• Pluggable input-parser registry (data/inputs hydra group) mirroring the label-parser seam: text_csv, text_jsonl, text_dir variants selected via defaults: [data/inputs: text_csv]. Adding one is a decorator + schema + import. The _target_-resolve-and-instantiate logic is factored into one shared create_parser core that both the input- and label-parser factories delegate to.

Model layer

• ModelConfig.tokenizer (a HuggingFace hub id) loads AutoModelForSequenceClassification + AutoTokenizer via the new text optional-dependencies extra. transformers is lazy-imported, so the core install is unaffected.
• ModelBackend.__call__ widened to accept forward kwargs (attention_mask); every backend override widened to match.

Forward + transparency

• Text loads tokenise to input_ids + attention_mask; the mask threads through both the metrics forward pass (via ForwardContext.extras) and the Captum explainer (via additional_forward_args).
• Token attribution uses LayerIntegratedGradients over the embedding layer, rendered per-token (single and batched) by CaptumTextVisualiser.
• Deps inference now requires the text extra (alongside the torch runtime) for tokenizer-configured models.

Use case + docs

• contributor-configs/text-classification-sst2/ runs a real DistilBERT sentiment model end-to-end (load, predict, token attribution, HTML report) with a passing e2e smoke test.
• New adding-an-input-parser contributor guide; supported-modalities docs updated.

All text behaviour is gated on InputKind.TEXT / a present attention_mask, so image, tabular, and detection paths are byte-identical.

Built with subagent-driven development (11 tasks, per-task spec+quality review, whole-branch review). The batch (B,T) token-rendering slice overlaps follow-on #99 (the minimal necessary part; richer token-label highlighting still open).

Closes #340.

Checklist

• CI — Required checks are green
• Breaking changes — If this PR breaks compatibility (API, configs, file formats, etc.), the title uses ! after the type or scope (e.g. feat!: or feat(transparency)!:). I understand that this change will have a direct, disruptive impact on package users. I ensured that this change is absolutely necessary and cannot be delayed. (This PR is additive: new modality, new default-None config fields, additive **kwargs. Non-breaking.)
• Contributor guide — I've read Pull requests and commit messages before requesting review.

Optional

• Issue (optional) — Linked to Add text input modality + text-classification use case (end-to-end) #340.
• Docs (optional) — User + contributor docs updated (supported modalities, adding-an-input-parser guide, model.tokenizer).
• Tests (optional) — Unit tests per component + an end-to-end text-classification smoke test.

[Copilot]

Pull request overview

This PR adds end-to-end support for a text input modality in RAITAP, including pluggable text input parsers, HuggingFace tokenizer/model loading, attention-mask threading through forward + transparency, and a runnable text-classification (SST-2) example.

Changes:

• Introduces InputModality.text plus a Hydra data/inputs registry (CSV/JSONL/dir-of-txt variants) and text metadata inference (kind="text", layout="TOKENS").
• Adds HF text backend loading (model.tokenizer) and threads attention_mask through forward passes and Captum explainers (including embedding-dim reduction to per-token scores).
• Adds tests, docs, and a contributor config + e2e smoke test for SST-2 text classification with token attribution and HTML reporting.

Reviewed changes

Copilot reviewed 60 out of 61 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
uv.lock	Locks new/updated deps incl. transformers stack
pyproject.toml	Adds text optional extra (transformers)
src/raitap/configs/schema.py	Adds ModelConfig.tokenizer + data.inputs schemas
src/raitap/configs/zen.py	Imports input parser package for registration
src/raitap/data/_parser_factory.py	Shared _target_ resolve/instantiate helper
src/raitap/data/data.py	Loads/tokenizes text; stores attention_mask
src/raitap/data/input_parsers/init.py	Imports built-in text input parsers
src/raitap/data/input_parsers/base.py	Defines InputParser protocol
src/raitap/data/input_parsers/factory.py	Adds input parser factory
src/raitap/data/input_parsers/registration.py	Adds @input_parser family decorator
src/raitap/data/input_parsers/text_csv.py	CSV text-column parser
src/raitap/data/input_parsers/text_dir.py	Directory-of-.txt parser
src/raitap/data/input_parsers/text_jsonl.py	JSONL text-field parser
src/raitap/data/label_parsers/factory.py	Refactors to shared parser factory
src/raitap/data/metadata.py	Infers text kind/layout from modality
src/raitap/data/types.py	Adds InputModality.text + .txt extension
src/raitap/data/tests/test_input_parser_registry.py	Tests Hydra registry integration
src/raitap/data/tests/test_load_text.py	Tests tokenisation helper
src/raitap/data/tests/test_text_input_parsers.py	Tests CSV/JSONL/dir parsers
src/raitap/data/tests/test_text_metadata.py	Tests text metadata inference
src/raitap/data/tests/test_types.py	Updates modality membership test
src/raitap/deps/inference.py	Infers text extra when tokenizer configured
src/raitap/deps/tests/test_inference.py	Tests deps inference for tokenizer models
src/raitap/models/base_backend.py	Widens backend __call__ to accept kwargs
src/raitap/models/model.py	Loads HF backend when model.tokenizer set
src/raitap/models/onnx_backend.py	Updates backend signature for kwargs
src/raitap/models/torch_backend.py	Forwards kwargs; adds HF text backend loader
src/raitap/models/tree_backend.py	Updates backend signature for kwargs
src/raitap/models/tests/test_backend.py	Updates backend protocol test
src/raitap/models/tests/test_backend_kwargs.py	Tests kwargs forwarding to backend
src/raitap/models/tests/test_hf_text_backend.py	Tests HF backend forward w/ mask
src/raitap/pipeline/orchestrator.py	Passes attention_mask into forward pass
src/raitap/pipeline/phases/forward_pass.py	Adds forward_kwargs threading via context
src/raitap/pipeline/phases/tests/test_forward_pass.py	Updates forward-pass test backend signature
src/raitap/pipeline/tests/test_text_classification_e2e.py	Adds SST-2 end-to-end smoke test
src/raitap/robustness/tests/test_factory.py	Updates test backend signature
src/raitap/robustness/tests/test_e2e_statistical_sampling.py	Updates test backend signature
src/raitap/robustness/tests/test_e2e_robustness_matrix.py	Updates test backend signature
src/raitap/robustness/tests/test_e2e_real_data.py	Updates test backend signature
src/raitap/task_families/classification.py	Threads mask into forward + explainer kwargs
src/raitap/task_families/tests/test_classification_forward_mask.py	Tests mask reaches forward
src/raitap/transparency/explainers/captum_explainer.py	Pops mask into Captum additional_forward_args; wraps HF outputs
src/raitap/transparency/semantics.py	Treats text Layer* attributions as TOKEN_SEQUENCE space
src/raitap/transparency/tests/test_assess_transparency_detection.py	Updates test backend signature
src/raitap/transparency/tests/test_captum_text_attribution.py	Adds Captum text attribution mechanism tests
src/raitap/transparency/tests/test_factory.py	Updates test backend signature
src/raitap/transparency/visualisers/captum_visualisers.py	Accepts batched token layouts; renders batch panels
src/raitap/transparency/visualisers/tests/test_visualisers.py	Adds validation test for batched token sequence
docs/contributor/adding/index.md	Links new input-parser contributor doc
docs/contributor/adding/adding-an-input-parser.md	New “adding an input parser” guide
docs/contributor/adding/adding-a-task-family.md	Updates data-layer boundary notes for text
docs/contributor/architecture.md	Notes data module includes text now
docs/contributor/modules/data.md	Documents text modality + inputs registry seam
docs/modules/data/configuration.md	Documents data.inputs variants for text
docs/modules/data/own-vs-built-in.md	Adds user-facing text input docs
docs/modules/model/configuration.md	Documents model.tokenizer behavior
docs/modules/model/own-vs-built-in.md	Adds HF hub id + tokenizer guidance
docs/modules/transparency/visualisers.md	Updates CaptumTextVisualiser batch behavior
contributor-configs/text-classification-sst2/README.md	Documents runnable SST-2 example
contributor-configs/text-classification-sst2/build_data.py	Generates gitignored SST-2 fixtures
contributor-configs/text-classification-sst2/assessment.yaml	Adds full SST-2 assessment config

[stanlrt]

Triage of the automated review comments:

Fixed

• Copilot — batched token_labels mislabel (captum_visualisers.py): CaptumTextVisualiser.visualise now raises a clear ValueError when token_labels is passed with a (B, T) batch (a single label list would be reused across samples). Per-sample labels remain follow-on work in Add batch-aware text attribution visualisation #99. Added tests. (b9ec9da)
• Copilot — bare ImportError for missing text extra (data.py): the lazy transformers import now raises a clear "install the 'text' extra" message. (86005d0)

False positives (enforced tooling — ruff + pyright — is green on all)

• Code Quality — base.py:16 "statement has no effect": the ... is a Protocol method body, matching the existing label_parsers/base.py convention.
• Code Quality — registration.py:13 "unused import InputParser": used as the TypeVar(bound="InputParser") forward-ref under TYPE_CHECKING (mirrors label_parsers/registration.py).
• Code Quality — "unused Any/AppConfig" in the two test files: both are referenced from string-literal cast("Any", ...) / cast("AppConfig", ...) targets, which need the imports for type resolution. ruff check passes.

Deferred (valid, low-impact, follow-up)

• Copilot — tokenizer loaded twice (data.py + load_hf_text_backend): both use the same model.tokenizer id, so the second from_pretrained hits the HF cache (no re-download) and cannot mismatch. Sharing a single instance is a clean-up worth a follow-up, not a correctness issue.
• Copilot — text sample_ids are sequential ["0","1",...]: valid; returning None for text would drop the spurious label-parser warning. Bundling with the related data.source/data.inputs.source clean-up in a follow-up to avoid touching label alignment in this PR.

CI: the Quality Gate pyright failure was the text extra not being installed in the gate env (so transformers was unresolved); fixed by adding --extra text to the quality-gate + package-smoke syncs (1a4affa).

[Copilot]

Pull request overview

Copilot reviewed 63 out of 64 changed files in this pull request and generated 2 comments.

[Copilot]

Pull request overview

Copilot reviewed 63 out of 64 changed files in this pull request and generated 2 comments.