Skip to content

fix(table): preserve null extension-array values in JSON (#10299) - #10326

Open
hsusul wants to merge 1 commit into
marimo-team:mainfrom
hsusul:fix-table-extension-null-serialization
Open

fix(table): preserve null extension-array values in JSON (#10299)#10326
hsusul wants to merge 1 commit into
marimo-team:mainfrom
hsusul:fix-table-extension-null-serialization

Conversation

@hsusul

@hsusul hsusul commented Jul 25, 2026

Copy link
Copy Markdown

This pull request was authored by a coding agent.

📝 Summary

Closes #10299

Problem

When a pandas DataFrame column contains an extension array (such as pint-pandas) with missing values (pd.NA, None, np.nan), PandasTableManager._to_json_str calls _extension_column_needs_stringify(result[col]) and unconditionally performs result[col] = result[col].astype(str). This stringifies missing cells into literal text like "<NA> meter" (or "nan") instead of JSON null.

Fix

Convert valid non-missing cells in extension columns to string (astype(str)), while using .astype(object).where(series.notna(), None) so missing entries remain None and serialize to JSON null as expected.

📋 Pre-Review Checklist

  • For large changes, or changes that affect the public API: this change was discussed or approved through an issue, on Discord, or the community discussions (Please provide a link if applicable).
  • Any AI generated code has been reviewed line-by-line by the human PR author, who stands by it.
  • Video or media evidence is provided for any visual changes (optional).

✅ Merge Checklist

  • I have read the contributor guidelines.
  • Documentation has been updated where applicable, including docstrings for API changes.
  • Tests have been added for the changes made.

Testing

  • uv run --group test pytest tests/_plugins/ui/_impl/tables/test_pandas_table.py — Passed (122 passed).
  • uv run ruff check marimo/ tests/ — Passed with 0 errors.
  • uv run ruff format --check marimo/ tests/ — Passed with 0 issues.

@hsusul

hsusul commented Jul 25, 2026

Copy link
Copy Markdown
Author

I have read the CLA Document and I hereby sign the CLA

@vercel

vercel Bot commented Jul 25, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
marimo-docs Ready Ready Preview, Comment Jul 25, 2026 4:28am

Request Review

@github-actions

github-actions Bot commented Jul 25, 2026

Copy link
Copy Markdown
Contributor

All contributors have signed the CLA ✍️ ✅
Posted by the CLA Assistant Lite bot.

@hsusul

hsusul commented Jul 25, 2026

Copy link
Copy Markdown
Author

I have read the CLA Document and I hereby sign the CLA

I have read the CLA Document and I hereby sign the CLA

@hsusul

hsusul commented Jul 25, 2026

Copy link
Copy Markdown
Author

recheck

@mscolnick
mscolnick marked this pull request as ready for review July 29, 2026 13:38

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (1)

tests/_plugins/ui/_impl/tables/test_pandas_table.py:2373

  • The expected string for the non-missing pint value is hard-coded ("1.0 meter"), which can vary across pint/pint-pandas versions and formatting. To avoid a brittle test, derive the expected string from pint-pandas (like test_to_json_str_pint_pandas_series does) and only hard-code the missing value as None.
        manager = self.factory.create()(df)
        json_data = json.loads(manager.to_json_str())

        assert json_data == [{"length": "1.0 meter"}, {"length": None}]

@kirangadhave kirangadhave added the bug Something isn't working label Jul 29, 2026
expected = [{"value": value} for value in series.astype(str)]
assert json_data == expected

@pytest.mark.requires("pint_pandas")

@kirangadhave kirangadhave Jul 31, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this test will never run, because we don't have pint or pint_pandas listed anywhere in our deps. Same with any other tests marked as requires pint or pint_pandas.

The test should create a new masked extension type within test scope and test against that. None of the new code is pint specific.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Missing pint-pandas values render as text ("<NA> meter") in table JSON instead of null

3 participants