[textures] Agents cannot reliably select sources from flat path mappings

[jamiesun]

Context Boundary

• Repo/module: talkincode/scorekit, texture profile schema and agent-driven scene authoring
• Version: scorekit 0.5.0
• Runtime: scorekit CLI
• OS/arch: Darwin arm64
• Relevant commands: scorekit schema --texture-profile, scorekit --help
• Observed external usage: ScoreData/profiles/textures/scoredata.yaml is intended to bind reusable local texture sources but currently exposes only 7 curated keys.

Expectation vs. Reality

Expected

An agent-driven composition workflow should be able to enumerate available texture sources, distinguish them by meaningful sound characteristics, and verify that an exact source exists before writing textures[].source into a scene.

The discovery contract should remain usable when a profile exposes hundreds of ambience, Foley, transition, impact, tonal, industrial, organic, and sound-design recordings. Agents should be able to determine that no suitable source exists instead of silently choosing an unrelated approximation.

Actual

scorekit schema --texture-profile exposes only:

• name
• description
• root
• sources, whose values are path strings

The schema has additionalProperties: false, so it provides no supported location for source-level metadata such as semantic tags, family, character, recommended loop/one_shot mode, loopability, duration, intensity, or scene use cases.

scorekit --help also exposes no texture inventory, inspection, or search command. An authoring agent must already know the portable key or infer suitability from logical keys and raw filenames. Small curated profiles therefore leave scene intents uncovered, while large exhaustive profiles become difficult to select from reliably.

Reproduction Path

1. Run:

   scorekit schema --texture-profile
   

2. Observe that sources is defined as a flat portable source name -> audio file path mapping and that no source-level metadata is supported by the schema.

3. Run:

   scorekit --help
   

4. Observe that there is no command for enumerating or querying texture sources by semantic properties.

5. Give an authoring agent a scene request requiring a specific texture, such as a glass-break impact, a seamless industrial grinding bed, or a soft liquid ambience, without separately telling it the source key.

6. The agent must inspect and interpret the profile manually, use an out-of-band catalog, substitute an approximate source, or omit the requested layer.

Minimal profile demonstrating the information boundary:

name: example-textures
root: ./recordings
sources:
  chain_grind_loop: industrial/chaingrindLoop.wav
  glass_break_01: impacts/glass_break.wav
  bubbles_soft: water/bubbles.wav

The profile binds files correctly, but it cannot express which source is loopable, its duration or intensity, or which scene intents it matches.

Blast Radius

• Affected users/paths: agent-driven scene authoring that uses optional textures; especially installations with large local sample libraries.
• Frequency: occurs whenever the requested sound intent is not already associated with a known portable source key.
• Severity rationale: medium-to-high capability and output-quality impact. Core MIDI compilation is unaffected, but texture selection can be omitted, semantically incorrect, or blocked by an unknown key.
• Current workaround: manually read the full profile and source filenames, keep a separate undocumented metadata catalog, and manually verify paths before building.

Acceptance Boundary

A complete result should make texture sources machine-discoverable for authoring agents at large inventory sizes. In particular, an agent should be able to:

• enumerate the sources exposed by a texture profile;
• retrieve enough semantic information to distinguish materially different candidates;
• identify recommended playback constraints such as loop versus one_shot where known;
• determine explicitly when no exact candidate is available;
• continue using existing flat texture profiles without breaking compatibility.

Canonical configuration format

YAML should remain the canonical, human-maintained source of truth for both texture path bindings and agent-facing metadata. The feature should not require maintainers or agents to combine separately maintained YAML and JSON/JSONL configuration files.

A structured YAML source may look like:

schema_version: 2
name: scoredata-textures
root: ../..
sources:
  industrial_chain_grind_loop_01:
    path: libraries/example/chaingrindLoop.wav
    family: industrial
    tags: [chain, metal, grinding, dark, mechanical]
    playback:
      modes: [loop]
      default_mode: loop
      loopable: true
    audio:
      duration_seconds: 6.4
      intensity: 0.65
      brightness: 0.35
      tonality: atonal
    scenes: [factory, machinery, dungeon, tension]

Existing string-valued source mappings should remain valid for backward compatibility:

sources:
  ocean_drum: recordings/ocean-drum.wav

JSON, JSONL, or SQLite indexes may be generated from the canonical YAML for CLI output or search performance, but they should be treated as derived artifacts rather than independently maintained configuration.

This issue does not otherwise prescribe whether discovery is implemented through an extended profile loader, a generated index, or additional CLI commands.

Evidence

Current schema excerpt:

{
  "sources": {
    "description": "Portable source name -> audio file path (WAV, FLAC, OGG, etc.).",
    "type": "object",
    "additionalProperties": { "type": "string" }
  },
  "additionalProperties": false
}

In the observed ScoreData corpus, VSCO 2 CE Miscellania Raw alone contains 158 WAV files, while the current external texture profile exposes 7 sources. VCSL contains additional bowed, scraped, glass, chime, ocean-drum, siren, and other sound-design candidates. This makes the limitation visible well before the inventory reaches thousands of files.