feat(mappers): support explicit property schemas

[AG0708]

Summary

• allow mapped properties to define an expr plus arbitrary JSON Schema keywords
• validate explicit schemas and surface configuration errors early
• extend the built-in stream_maps config schema so validate_config=True accepts typed mappings
• preserve existing string/null mapping compatibility and avoid mutating user config

Validation

• uv run pytest -q — 855 passed, 388 deselected, 1 expected xfail, 21 subtests passed
• changed-file pre-commit — passed
• mypy and ty on both changed Python files — passed
• Sphinx build with warnings treated as errors — passed
• focused post-review regression suite — 5 passed

CI note

The API Changes job reports the intentional value change to the public STREAM_MAPS_CONFIG schema. No API is removed; extending that schema is required so SDK plugins using validate_config=True can accept the new typed mapping form. Comparable additive configuration-schema changes in this repository have produced the same diagnostic.

Closes #2010

[sourcery-ai]

Reviewer's Guide

Adds support for explicitly-typed mapped properties in stream maps by allowing dict-based property definitions with JSON Schema, validating them at config-load time, and extending the stream_maps capability schema and documentation, while preserving existing string/null mapping behavior and avoiding user config mutation.

File-Level Changes

Change	Details	Files
Allow mapped properties to be defined as typed objects combining an expression with arbitrary JSON Schema keywords, and validate these definitions eagerly.	Extend _init_functions_and_schema to accept both string and dict property definitions, treating dicts as typed property mappings. Extract and validate the 'expr' field from typed property definitions, raising StreamMapConfigError for missing/invalid expressions or invalid JSON Schemas. Populate transformed_schema properties directly from validated typed definitions and reuse expression parsing for both string and typed mappings. Adjust handling of null and 'NULL' mappings to use explicit equality checks and keep behavior unchanged.	singer_sdk/mapper.py
Extend configuration/schema capabilities to describe and accept typed stream map property definitions.	Add AnyType to capabilities imports and use it as additional_properties in the stream_maps ObjectType. Update the stream_maps capability schema to include an ObjectType alternative with a required 'expr' string and arbitrary extra JSON Schema keywords for mapped properties.	singer_sdk/helpers/capabilities.py
Document and test the new typed mapped property behavior, including config validation and error handling.	Add tests that verify typed property transforms produce expected output and schemas while leaving the original config objects unmutated. Add tests that ensure typed property definitions pass plugin config validation when validate_config=True is used. Add parametrized tests that assert invalid typed property definitions raise StreamMapConfigError with appropriate messages. Update stream_maps documentation to describe object-based mappings with expr plus JSON Schema, give YAML examples, and clarify that string/null mappings remain supported.	tests/core/test_mapper.py docs/stream_maps.md

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[read-the-docs-community]

Documentation build overview

📚 Meltano SDK | 🛠️ Build #34022182 | 📁 Comparing 55885a4 against latest (c399afe)

  🔍 Preview build  

1 file changed

± stream_maps.html

[sourcery-ai]

Hey - I've found 1 issue, and left some high level feedback:

• In the capabilities config schema, Property("expr", StringType, ...) likely needs StringType() to be consistent with the other type helpers and avoid passing the class instead of an instance.
• For typed property definitions, you rely on DEFAULT_JSONSCHEMA_VALIDATOR.check_schema at runtime; if you want to keep config errors as deterministic as possible, consider constraining additional_properties in the ObjectType schema so obviously invalid keys are caught earlier by config validation.

Prompt for AI Agents

Please address the comments from this code review:

## Overall Comments
- In the capabilities config schema, `Property("expr", StringType, ...)` likely needs `StringType()` to be consistent with the other type helpers and avoid passing the class instead of an instance.
- For typed property definitions, you rely on `DEFAULT_JSONSCHEMA_VALIDATOR.check_schema` at runtime; if you want to keep config errors as deterministic as possible, consider constraining `additional_properties` in the `ObjectType` schema so obviously invalid keys are caught earlier by config validation.

## Individual Comments

### Comment 1
<location path="singer_sdk/mapper.py" line_range="578-579" />
<code_context>
-            if prop_def in {None, NULL_STRING}:
+            if prop_def is None or prop_def == NULL_STRING:
                 if prop_key in (self.transformed_key_properties or []):
                     msg = (
                         f"Removing key property '{prop_key}' is not permitted in "
</code_context>
<issue_to_address>
**suggestion (bug_risk):** Use `str(ex)` instead of `ex.message` for jsonschema errors to be more robust.

When formatting the error for an invalid typed stream map schema, this relies on `ex.message`, which is deprecated/internal in newer Python/jsonschema versions and may be missing or differently formatted. Prefer `str(ex)` (optionally with the exception type) for a more robust message, e.g. `f"Invalid JSON Schema ...: {ex}"`.

Suggested implementation:

```python
        except jsonschema.exceptions.ValidationError as ex:
            raise ValueError(
                f"Invalid JSON Schema for typed stream map schema: {ex}"
            )

```

```python
        except jsonschema.exceptions.SchemaError as ex:
            self.logger.error(
                "Invalid JSON Schema for typed stream map schema: %s", str(ex)
            )
            raise

```

If there are other uses of `ex.message` for jsonschema-related exceptions in this file (or nearby modules), they should be updated in the same way:
- Replace `ex.message` in f-strings with `{ex}`.
- Replace `ex.message` used as a standalone argument (e.g. logger calls) with `str(ex)`.
This keeps formatting robust across jsonschema/Python versions.
</issue_to_address>

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.35%. Comparing base (c399afe) to head (55885a4).

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #3733   +/-   ##
=======================================
  Coverage   94.35%   94.35%           
=======================================
  Files          74       74           
  Lines        6289     6289           
  Branches      770      770           
=======================================
  Hits         5934     5934           
  Misses        266      266           
  Partials       89       89           

Flag	Coverage Δ
core	82.93% <ø> (ø)
end-to-end	75.62% <ø> (+0.03%)	⬆️
optional-components	45.01% <ø> (+0.03%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[codspeed-hq]

Merging this PR will not alter performance

✅ 14 untouched benchmarks

---

_{Comparing AG0708:codex/2010-typed-stream-map-properties (55885a4) with main (c399afe)}

[AG0708]

I also applied the review-level StringType() consistency suggestion in 55885a4. I am intentionally keeping additional_properties=AnyType(): after expr is removed, the object is a JSON Schema, whose valid vocabulary includes composition keywords, annotations, and extension keywords. Enumerating keys in the plugin config schema would reject valid present or future schemas; DEFAULT_JSONSCHEMA_VALIDATOR.check_schema remains the authoritative validation step. The config layer still requires expr to be a string, and the validate_config=True regression test covers that boundary.