build: migrate type checker from mypy to basedpyright#1170
Conversation
There was a problem hiding this comment.
Pull request overview
This PR improves static typing for openwpm.utilities.db_utils.query_db and its wrapper get_javascript_entries by using @overload so the return type is determined by the as_tuple argument (rows by default, tuples only when as_tuple=True), without changing runtime behavior.
Changes:
- Add
@overloadsignatures forquery_dbto precisely typeas_tuple=Falsevsas_tuple=True. - Apply the same overload pattern to
get_javascript_entries. - Update typing imports to support
overload/Literal.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #1170 +/- ##
==========================================
+ Coverage 62.19% 68.12% +5.92%
==========================================
Files 40 38 -2
Lines 3902 3576 -326
==========================================
+ Hits 2427 2436 +9
+ Misses 1475 1140 -335 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
ed4a23b to
a7989aa
Compare
a7989aa to
bdb4cbd
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.
Comments suppressed due to low confidence (1)
openwpm/utilities/db_utils.py:68
- Same overload ambiguity as
query_db:as_tuple: Literal[True] = ...makes theTrueoverload appear callable without specifyingas_tuple, overlapping with the default-Falseoverload. Prefer makingas_tuple=Truerequired in this overload (no default) so inference can’t accidentally select the tuple-returning overload for default calls.
@overload
def get_javascript_entries(
db: Path, all_columns: bool = ..., as_tuple: Literal[True] = ...
) -> List[Tuple[Any, ...]]: ...
| @overload | ||
| def query_db( | ||
| db: Path, query: str, params: Any = ..., as_tuple: Literal[True] = ... | ||
| ) -> List[Tuple[Any, ...]]: ... |
Add @overload signatures so the return type (tuples vs sqlite Row) is inferred from the as_tuple keyword argument. Drop the now-redundant isinstance assertions in tests that the refined overloads make unnecessary.
Replace mypy with basedpyright in pyproject.toml, the pre-commit config, and the conda environment files. Add a multiprocess type stub so the untyped library stops producing false positives.
Fix genuine type errors found when switching to basedpyright: a possibly-unbound variable, missing None-checks, an exception passed where a traceback limit was expected, and an incompatible store_blob override in the in-memory provider. Annotate the BrowserManager and StorageController queues with precise payload types and mark the storage base classes as explicitly abstract. Delete the unmaintained build_cookie_table utility, which imported the long-uninstallable netlib package and was therefore already un-importable, along with the now-orphaned cookie module it was the sole importer of. Record the remaining recommended-mode findings in a basedpyright baseline so CI gates only on newly introduced issues.
bdb4cbd to
78fdbe4
Compare
Summary
Migrates OpenWPM's static type checker from mypy to basedpyright and resolves the genuine type errors that surfaced in the process. Organised as three commits.
1.
feat(typing): overload query_db and get_javascript_entriesdb_utils.query_db(and the wrapperget_javascript_entries) returnslist[sqlite3.Row]by default andlist[tuple[Any, ...]]only withas_tuple=True. The old union return type forced every default-mode caller to addassert isinstance(row, sqlite3.Row)or# type: ignore.@overloadsignatures now make the return type follow theas_tuplekeyword:query_db(db, q)list[sqlite3.Row]query_db(db, q, as_tuple=False)list[sqlite3.Row]query_db(db, q, as_tuple=True)list[tuple[Any, ...]]query_db(db, q, as_tuple=some_bool)list[sqlite3.Row] | list[tuple[Any, ...]]as_tupleis now keyword-only — the overloads would otherwise overlap. Redundantisinstanceassertions in the test suite are removed.2.
build: migrate type checker from mypy to basedpyrightpyproject.toml,.pre-commit-config.yaml, and the conda environment filestypeCheckingMode = "recommended"multiprocesstype stub added so the untyped library stops producing false positives3.
fix(typing): resolve type errors surfaced by basedpyrightGenuine bugs and type errors basedpyright caught:
browser_profile_pathinBrowserManagerNone-checks beforeopen()/ screenshot pathstraceback.format_exc()expected a frame limitstore_bloboverride in the in-memory storage provider, realigned to the abstract signatureQueuepayload types on the BrowserManager / StorageController IPC queuesABCAlso deletes the unmaintained
build_cookie_tableutility and thecookiemodule it was the sole importer of — both depended on the long-uninstallablenetlibpackage and were already un-importable.The remaining
recommended-mode findings (untyped third-party libraries, intentional bad-input tests, themanual_test.pydebug script) are frozen in.basedpyright/baseline.json, so CI gates only on newly introduced issues.Test plan
basedpyright— 0 errors, 0 warningspre-commit run --all-filespassespytestsuite — 137 passed, 3 skipped. The lone failure,test_audio_fingerprinting, is the known pre-existing flake (@pytest.mark.skipif(CI), "Flaky on CI") and is unrelated to these changes.