Skip to content

Fix Python comprehension filters after each for clause - #1623

Open
amkdown-max wants to merge 2 commits into
lark-parser:masterfrom
amkdown-max:codex/fix-python-comprehension-filters
Open

Fix Python comprehension filters after each for clause#1623
amkdown-max wants to merge 2 commits into
lark-parser:masterfrom
amkdown-max:codex/fix-python-comprehension-filters

Conversation

@amkdown-max

Copy link
Copy Markdown

Summary

  • attach zero or more comp_if clauses to each comp_for
  • allow filters to appear between multiple comprehension for clauses
  • add regression coverage for list, set, dict, generator, and async comprehensions

Why

Python permits every comprehension for clause to have its own sequence of if filters. Lark's grammar instead placed one optional filter after the entire comp_fors sequence, so valid expressions such as:

[(i, j) for i in range(3) if i > 0 for j in range(5) if j % 2 == 0]

failed when the parser reached the second for.

This change mirrors CPython's for_if_clause structure by keeping each clause's filters with that clause.

Fixes #1525.

Validation

  • python3 -m tests TestPythonParser
  • python3 -m tests (1311 tests passed, 174 skipped)
  • pre-commit run --all-files --show-diff-on-failure
  • parsed 51 real Python files from lark/ and tests/
  • verified the regression with both LALR/contextual and Earley/basic parsers
  • verified the synchronous examples are accepted by both CPython and Lark

@amkdown-max
amkdown-max marked this pull request as ready for review July 19, 2026 07:21
@erezsh

erezsh commented Jul 19, 2026

Copy link
Copy Markdown
Member

Looks good! Just one nit, I prefer -

comp_for: [ASYNC] "for" exprlist "in" or_test comp_ifs
comp_ifs: comp_if*

It's more organized and easier to process. What do you think?

@amkdown-max

Copy link
Copy Markdown
Author

Agreed — this is clearer and keeps the rule easier to process. Updated in 085eba1. The full test suite (1,311 tests), mypy, and all pre-commit hooks pass. Thanks!

@amkdown-max

Copy link
Copy Markdown
Author

Hi @erezsh, just a gentle follow-up: the workflow runs for the latest commit (085eba1) are still awaiting approval. When convenient, could you approve them so the checks can run? No rush, and thanks again!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

comp_if should be moved into comp_for to correctly reflect Python's comprehension grammar

3 participants