Skip to content

Latest commit

 

History

History
94 lines (54 loc) · 5.69 KB

File metadata and controls

94 lines (54 loc) · 5.69 KB

Architecture

Current documented release: 1.3.3.

cpython-extensions transforms ordinary Python functions while preserving a conservative, fail-closed boundary around CPython 3.13 implementation details. Benchmarks observe the implementation; they do not define semantics.

Source layout

src/python_extensions/
  __init__.py      public API and lazy feature loading
  switch.py        switch syntax recognition, planning, portable/live lowering
  _livegate.c      optional native live lookup + gate write
  _specialize.py   partial evaluation, specialization, adaptive hot paths
  _runtime.py      runtime qualification and diagnostics
  inline.py        inline registry, call-site analysis, lowering, optimization
  goto.py          label/goto lowering and control-flow validation
  compose.py       canonical multi-extension composition
  _core/           CFG, data-flow, verification, and reports

Transformation pipeline

The canonical order is:

switch -> partial -> inline -> goto -> specialize/hotpath -> verify

Switch runs first because it may rebuild source. Partial evaluation can expose constants before inlining. Goto resolves pseudo-labels after static code growth. specialize and hotpath are alternative final dispatch layers. Every accepted transformation is verified before return.

Runtime qualification

Root import runs a bounded package-owned qualification of the CPython 3.13 assumptions required by the core path: wordcode alignment, required opcodes, CodeType.replace, exception-table decoding, the shared verifier, portable switch execution, and goto prerequisites.

Mutation-sensitive or dependency-specific checks remain lazy. runtime_diagnostics(full=True) can additionally qualify live layout/native support and the inline/specialization subsystems.

Qualification state is process-local and synchronized. Failure caches retain detached type/message information rather than traceback-bearing exception objects.

Shared verification state

The verifier uses a bounded identity/weak-reference cache. It avoids CodeType.__hash__ because code constants can contain unhashable values, and weak references prevent cached verification from keeping transformed code alive.

Transformation reports reuse verified CFG/stack information where possible rather than rebuilding the same analysis.

Switch

Portable compiler

Portable mode is the normal architecture. It preserves Python mapping hash/equality semantics and chooses only lowerings whose observable behavior can be justified for the source shape. Plans include compact direct/template routes and balanced/general control-flow routes.

case_key_mode="typed" adds exact runtime type to route identity when Python's normal mapping aliases (for example 1, 1.0, and True) are not desired.

Live compiler

Live modes keep heterogeneous route bodies in-frame and update a verified jump gate at runtime. They are CPython-3.13-specific and have explicit re-entry/concurrency contracts.

mode="auto" does not silently become live. An explicit live_threshold= opt-in is plan-aware: strong portable direct/expression/statement-template plans can veto live selection even above the route threshold.

Native live dispatcher

The optional _livegate extension fuses route lookup and gate update. live_engine="auto" uses it only after runtime self-tests succeed; otherwise the live path can fall back to ctypes. Free-threaded CPython does not use live dispatch.

Specialization and partial evaluation

partial() removes explicitly bound parameters from the effective signature and performs conservative simplification.

specialize() maintains guarded variants with a generic fallback. hotpath() adds bounded runtime shape discovery before promotion. Guards avoid arbitrary user equality where it could change observable behavior, and adaptive profiling has finite shape/call budgets.

Eligible ordinary monomorphic hot paths can use sys.monitoring during warm-up and later move to a verified in-frame dispatcher; unsupported shapes remain on wrapper/generic paths.

Inline

Inlining is registry-driven. The transformer identifies registered direct-call shapes, proves binding/argument semantics, clones eligible bytecode, and then runs conservative local/CFG/data-flow optimizations.

binding="frozen" is snapshot semantics. binding="guarded" validates the relevant callable state and deoptimizes to an ordinary call when the target no longer matches.

Large repeated helpers can use shared appended regions to reduce code duplication. Expansion, growth, and final code-size limits bound pathological generated input.

Goto

enable_goto() recognizes label .name and goto .name pseudo-statements. Strict mode verifies synthetic edges against the final CFG and CPython exception-region signatures. Missing labels, duplicate labels, invalid stack edges, and unsafe protected-region transitions fail explicitly.

Unsafe mode is an explicit escape hatch and is not the default contract.

Concurrency and lifecycle

Portable transformed functions have ordinary Python call concurrency subject to their own application logic. Registry/qualification mutation is synchronized separately from runtime hot paths.

Live switch modes have stronger restrictions because they modify a dispatch gate. Use the documented mode that matches the required re-entry/concurrency model rather than treating fast as a generic performance setting.

Compatibility boundary

This project intentionally depends on CPython 3.13 bytecode and exception-table behavior. A source-level API does not make the implementation interpreter-portable. Support is defined by Compatibility and runtime qualification, not by successful import on an untested interpreter.