fix: IpynbConverter.accepts() raises UnicodeDecodeError on non-ASCII files (e.g. French PDFs)

[echavet]

Summary

IpynbConverter.accepts() can raise UnicodeDecodeError when processing files with non-ASCII content (e.g. PDFs with French or other accented text). This propagates uncaught through the conversion pipeline and crashes the entire conversion — even though the file has nothing to do with Jupyter notebooks.

Problem

The engine in _markitdown.py only protects accepts() calls against NotImplementedError:

_accepts = False
try:
    _accepts = converter.accepts(file_stream, stream_info, **_kwargs)
except NotImplementedError:
    pass

Any other exception raised by accepts() propagates up and crashes the caller. IpynbConverter.accepts() reads the file stream and decodes it to look for nbformat markers. If the file contains bytes that cannot be decoded (binary content, wrong charset), a UnicodeDecodeError is raised.

Traceback (production environment — Windows Server)

File "markitdown\converters\_ipynb_converter.py", line 36, in accepts
    notebook_content = file_stream.read().decode(encoding)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 43916: ordinal not in range(128)

The affected file was a standard French-language PDF invoice. The byte 0xc3 is the first byte of a multi-byte UTF-8 sequence representing accented characters (é, è, à, etc.).

Root Cause

accepts() is a predicate — per the contract defined in _base_converter.py, it must return bool and never raise. The decode block was not protected against encoding failures:

# Before — unsafe
encoding = stream_info.charset or "utf-8"
notebook_content = file_stream.read().decode(encoding)  # 💥 raises on binary/wrong-charset files
return "nbformat" in notebook_content and "nbformat_minor" in notebook_content

Note: stream_info.charset can be explicitly set to "ascii" by MIME/charset detection for certain file types, making the issue reproducible even when "utf-8" is the fallback.

Fix

Wrap the decode + check block in try/except (UnicodeDecodeError, ValueError) and return False on failure. A file that cannot be decoded is definitively not a Jupyter notebook.

# After — safe
try:
    encoding = stream_info.charset or "utf-8"
    notebook_content = file_stream.read().decode(encoding)
    return (
        "nbformat" in notebook_content
        and "nbformat_minor" in notebook_content
    )
except (UnicodeDecodeError, ValueError):
    # File contains non-decodable bytes — definitely not a notebook
    return False

The finally: file_stream.seek(cur_pos) block is preserved, ensuring the stream position contract is always respected.

Compatibility

• ✅ No behavior change for valid .ipynb files or JSON streams
• ✅ No new dependencies
• ✅ No changes to method signatures
• ✅ Respects the accepts() contract: returns bool, resets stream position

Related

• Fixes issue: bug: IpynbConverter.accepts() raises UnicodeDecodeError on non-ASCII files (French PDFs, etc.) #1894
• This fix is independent of, and compatible with, PR feat: add progress_callback support for real-time conversion tracking #1851 (progress_callback feature)

Type of Change

• Bug fix (non-breaking change — adds error handling only)