|
| 1 | +<!-- Last reviewed: 2026-03 --> |
| 2 | + |
| 3 | +## Project |
| 4 | + |
| 5 | +Synapse Python Client — official Python SDK and CLI for Synapse (synapse.org), a collaborative science platform by Sage Bionetworks. Provides programmatic access to entities (projects, files, folders, tables, views), metadata, permissions, evaluations, and data curation workflows. Published to PyPI as `synapseclient`. |
| 6 | + |
| 7 | +## Stack |
| 8 | + |
| 9 | +- Python 3.10–3.14 (`setup.cfg`: `python_requires = >=3.10, <3.15`) |
| 10 | +- HTTP: httpx (async), requests (sync/legacy) |
| 11 | +- Models: stdlib dataclasses (NOT Pydantic) |
| 12 | +- Tests: pytest 8.2, pytest-asyncio, pytest-socket, pytest-xdist |
| 13 | +- Docs: MkDocs with Material theme, mkdocstrings |
| 14 | +- Linting: ruff, black (line-length 88), isort (profile=black), bandit, flake8 |
| 15 | +- CI: GitHub Actions → SonarCloud, PyPI deploy on release |
| 16 | + |
| 17 | +## Commands |
| 18 | + |
| 19 | +```bash |
| 20 | +# Install for development |
| 21 | +pip install -e ".[boto3,pandas,pysftp,tests,curator,dev]" |
| 22 | + |
| 23 | +# Unit tests |
| 24 | +pytest -sv tests/unit |
| 25 | + |
| 26 | +# Integration tests (requires Synapse credentials, runs in parallel) |
| 27 | +pytest -sv --reruns 3 tests/integration -n 8 --dist loadscope |
| 28 | + |
| 29 | +# Pre-commit checks (ruff, black, isort, bandit) |
| 30 | +pre-commit run --all-files |
| 31 | + |
| 32 | +# Build docs locally |
| 33 | +pip install -e ".[docs]" && mkdocs serve |
| 34 | +``` |
| 35 | + |
| 36 | +## Conventions |
| 37 | + |
| 38 | +### Async-first with generated sync wrappers |
| 39 | +All new methods must be async with `_async` suffix. The `@async_to_sync` class decorator (`core/async_utils.py`) auto-generates sync counterparts at class definition time. Never write sync methods manually on model classes — the decorator handles it. |
| 40 | + |
| 41 | +### Protocol classes for sync type hints |
| 42 | +Each model in `models/` has a corresponding protocol in `models/protocols/` defining the sync method signatures. When adding a new async method to a model, add its sync signature to the protocol class so IDE type hints work. |
| 43 | + |
| 44 | +### Dataclass models with `fill_from_dict()` |
| 45 | +Models are `@dataclass` classes, NOT Pydantic. REST responses are deserialized via `fill_from_dict()` methods on each model. New models must follow this pattern. |
| 46 | + |
| 47 | +### Concrete types are Java class names |
| 48 | +`core/constants/concrete_types.py` maps Java class names (e.g., `org.sagebionetworks.repo.model.FileEntity`) for polymorphic entity deserialization. When adding new entity types, register the concrete type string here. |
| 49 | + |
| 50 | +### Mixin composition for shared behavior |
| 51 | +Shared functionality lives in `models/mixins/` (AccessControllable, StorableContainer, AsynchronousJob, etc.). Prefer adding to existing mixins over duplicating logic across models. |
| 52 | + |
| 53 | +### `synapse_client` parameter pattern |
| 54 | +Most functions accept an optional `synapse_client` parameter. If omitted, `Synapse.get_client()` returns the cached singleton. Never pass `None` explicitly — omit the argument instead. |
| 55 | + |
| 56 | +### Branch naming |
| 57 | +Use `SYNPY-{issue_number}` or `synpy-{issue_number}` prefix for feature branches. PR titles follow `[SYNPY-XXXX] Description` format. |
| 58 | + |
| 59 | +## Architecture |
| 60 | + |
| 61 | +``` |
| 62 | +synapseclient/ |
| 63 | +├── client.py # Synapse class — public entry point, REST methods, auth |
| 64 | +├── api/ # REST API layer — one file per resource type |
| 65 | +├── models/ # Dataclass entities (Project, File, Table, etc.) |
| 66 | +│ ├── protocols/ # Sync method type signatures for IDE hints |
| 67 | +│ ├── mixins/ # Shared behavior (ACL, containers, async jobs) |
| 68 | +│ └── services/ # Model-level business logic |
| 69 | +├── operations/ # High-level CRUD: get(), store(), delete() |
| 70 | +├── core/ # Infrastructure: upload/download, retry, cache, creds, OTel |
| 71 | +│ ├── upload/ # Multipart upload (sync + async) |
| 72 | +│ ├── download/ # File download (sync + async) |
| 73 | +│ ├── credentials/ # Auth chain (PAT, OAuth, env vars, config file) |
| 74 | +│ └── constants/ # Concrete types, config keys, limits |
| 75 | +├── extensions/ # Optional modules (curator) |
| 76 | +└── entity.py, table.py, ... # Legacy classes (pre-OOP rewrite) |
| 77 | +``` |
| 78 | + |
| 79 | +Data flows: Client REST methods → API service functions → Models with `fill_from_dict()` → returned to caller. The `operations/` layer provides a simpler interface over this chain. |
| 80 | + |
| 81 | +## Constraints |
| 82 | + |
| 83 | +- Do not use Pydantic for models — the codebase uses stdlib dataclasses with custom serialization. Mixing would break the `@async_to_sync` decorator and `fill_from_dict()` pattern. |
| 84 | +- Do not write synchronous test files — write async tests only. The `@async_to_sync` decorator is validated by a dedicated smoke test. Duplicate sync tests were removed to cut CI cost. |
| 85 | +- Unit tests must not make network calls — `pytest-socket` blocks all sockets. Use `pytest-mock` for HTTP mocking. |
| 86 | +- `develop` is the default/main branch, not `main` or `master`. PRs target `develop`. |
| 87 | +- Legacy classes in root `synapseclient/` (entity.py, table.py, etc.) are kept for backwards compatibility. New features go in `models/` using the dataclass pattern. |
| 88 | + |
| 89 | +## Testing |
| 90 | + |
| 91 | +- `asyncio_mode = auto` in pytest.ini — no need for `@pytest.mark.asyncio` |
| 92 | +- `asyncio_default_fixture_loop_scope = session` — all async tests share one event loop |
| 93 | +- Unit test client fixture: session-scoped, `skip_checks=True`, `cache_client=False` |
| 94 | +- Integration tests use `--reruns 3` for flaky retries and `-n 8 --dist loadscope` for parallelism |
| 95 | +- Integration fixtures create per-worker Synapse projects; use `schedule_for_cleanup()` for teardown |
0 commit comments