Feature Request: Claude has spoken, anyone interested in this proposal?

[kundeng]

If there are enough interests, I can vibe code this.

Evolving Sacred: Dependencies, Grid Search, External Config — Marimo-Friendly

1. How Sacred Actually Works (A Walkthrough)

Let's trace what happens when you run the simplest Sacred experiment. This isn't the "conceptual model" — it's the actual code path.

The Setup Code

from sacred import Experiment

ex = Experiment("my_exp")

@ex.config
def cfg():
    lr = 0.01
    epochs = 10

def train_step(lr):
    return lr * 2

@ex.main
def my_main(lr, epochs):       # Sacred injects lr=0.01, epochs=10 from config
    return train_step(lr)      # just a normal function call with the injected value

ex.run()

What Experiment("my_exp") Does

Experiment is a subclass of Ingredient. When you call Experiment("my_exp"), it:

1. Grabs the caller's __file__ via inspect.stack() to know where the experiment lives on disk
2. Scans that file for imports to record package dependencies (numpy 1.24, torch 2.1, etc.)
3. Records the file's content and git hash so the exact code can be reproduced later
4. Initializes empty lists: configurations, captured_functions, commands, observers

That's it. The Experiment is basically a container that collects things.

What @ex.config Does

When you write @ex.config def cfg(): lr = 0.01; epochs = 10, Sacred does something unusual. It doesn't call cfg(). It doesn't even look at lr and epochs yet. Instead:

1. It passes cfg to ConfigScope(cfg)
2. ConfigScope.__init__ uses inspect.getsourcelines(cfg) to get the function's source code as text
3. It parses the source with Python's ast module to extract the function body (everything after def cfg():)
4. It compile()s that body into a standalone code object
5. It stores this compiled code for later

The config function body is essentially a little program that Sacred will eval() later to produce a dictionary. That's why you can write things like:

@ex.config
def cfg():
    lr = 0.01
    epochs = 10
    total_steps = epochs * 100  # Computed values work!

The local variables after eval become the config entries: {"lr": 0.01, "epochs": 10, "total_steps": 1000}.

What @ex.capture Does

@ex.capture wraps your function so that any parameters you don't explicitly pass get filled in from the config. Here's the actual mechanism:

1. create_captured_function(train_step) is called
2. It creates a Signature object from the function using inspect.signature(), recording the parameter names (["lr"]) and any defaults
3. It attaches attributes to the function: .config = {}, .logger = None, .run = None
4. It wraps the function with a @wrapt.decorator that intercepts every call

When the wrapped function is later called (e.g., train_step()), the wrapper:

1. Builds an options dict that combines the function's .config dict with special values _config, _log, _run
2. Calls signature.construct_arguments(args, kwargs, options) which does the key trick: for each parameter that wasn't explicitly passed, look it up by name in options and fill it in
3. Calls the real function with the filled-in arguments

So train_step() with no arguments becomes train_step(lr=0.01) because "lr" is in the config dict.

What @ex.main Does

@ex.main is just @ex.capture plus registering the function as the default command. Nothing more.

What ex.run() Does — The Big One

This is where everything comes together. ex.run() calls create_run() in initialize.py, which is the most complex part of Sacred. Here's the sequence:

Step 1: Build the Scaffolding

Sacred supports nested sub-experiments called Ingredients. Even if you don't use them, your Experiment is treated as a single-ingredient tree. create_run():

1. Walks the ingredient tree (depth-first) and sorts ingredients deepest-first
2. Creates a Scaffold object for each ingredient. A Scaffold is a temporary workspace that holds the config-in-progress, the list of captured functions belonging to that ingredient, and references to child Scaffolds.

Think of Scaffolding as the "build environment" — it exists only during create_run() to wire everything up, then gets thrown away.

Step 2: Resolve the Config (4 phases)

This is where Sacred computes the final config dictionary that all captured functions will share.

Phase 1 — CLI/programmatic overrides: If you called ex.run(config_updates={"lr": 0.5}), those updates get routed to the right Scaffold based on dotted-path matching. "lr" goes to the root; "data.path" goes to the "data" ingredient's Scaffold.

Phase 2 — Named configs: If you called ex.run(named_configs=["fast"]), each named config is evaluated (same eval() trick as @ex.config) and its results get merged in.

Phase 3 — Normal config scopes: Each Scaffold evaluates its @ex.config functions. The eval() happens inside a DogmaticDict — a special dict that tracks what was modified, what was added, and what type-changed. The "dogmatic" part: if a value was already set by an override (Phase 1), the DogmaticDict refuses to overwrite it and records that a conflict happened.

Phase 4 — Seeding: Sacred generates a random seed (or uses one from config) and hierarchically distributes seeds to sub-ingredients.

After this, you have one flat config dict: {"lr": 0.5, "epochs": 10, "seed": 123456}.

Step 3: Wire Up Captured Functions

For each captured function in every Scaffold, Sacred:

1. Sets func.config to the relevant slice of the config (the whole config, or just a sub-dict if the function has a prefix)
2. Sets func.logger to a child logger
3. Sets func.rnd to a per-function random state
4. Sets func.run to the Run object (so functions can access _run)
5. Makes the config read-only (ReadOnlyDict) to prevent accidental mutation

Step 4: Create and Execute the Run

A Run object is created holding: the final config, the main function, the observers list, and pre/post hooks. When Run.__call__() fires:

_emit_started()           → tells all observers "run started"
start heartbeat timer     → periodic observer updates
pre_run_hooks()           → any @ex.pre_run_hook functions
main_function()           → YOUR code runs here
post_run_hooks()          → any @ex.post_run_hook functions
_emit_completed(result)   → tells all observers "run done, here's the result"

If your code throws, _emit_failed() fires instead. If you Ctrl-C, _emit_interrupted().

The Observer Protocol

Observers are notified at lifecycle events via a simple interface. All methods are optional (default no-op):

class RunObserver:
    def started_event(self, ex_info, command, host_info, start_time, config, meta_info, _id): ...
    def heartbeat_event(self, info, captured_out, beat_time, result): ...
    def completed_event(self, stop_time, result): ...
    def failed_event(self, fail_time, fail_trace): ...
    def interrupted_event(self, interrupt_time, status): ...
    def resource_event(self, filename): ...
    def artifact_event(self, name, filename, metadata, content_type): ...
    def log_metrics(self, metrics_by_name, info): ...

FileStorageObserver creates numbered directories (1/, 2/, ...) with config.json, run.json, cout.txt, metrics.json. MongoObserver writes to MongoDB. They're completely decoupled from execution — they just listen.

Ingredients (Sub-Experiments)

An Ingredient is a reusable bundle of config + captured functions with its own namespace:

data_ing = Ingredient("data")

@data_ing.config
def data_cfg():
    path = "train.csv"

@data_ing.capture
def load(path):
    return pd.read_csv(path)

ex = Experiment("main", ingredients=[data_ing])

The resulting config is nested: {"lr": 0.01, "data": {"path": "train.csv"}}. You override with config_updates={"data": {"path": "test.csv"}}.

The limitation: ingredients can't be instantiated multiple times (you can't have two "data" ingredients with different paths), and they can't produce outputs that other ingredients consume. They're just config namespaces with attached functions.

Marimo Compatibility (Tested)

Sacred works inside marimo today. I ran 15 tests. Everything passes — config scopes, observers, metrics, named configs, captured functions, config overrides — with one friction point: marimo rewrites _-prefixed function names (e.g., _fast → _cell_Hbol_fast). Since Sacred registers named configs by func.__name__, an @ex.named_config def _fast() becomes unreachable by its intended name. Fix: don't use underscore prefixes on Sacred-decorated functions in marimo.

---

2. What We Want to Add

Three capabilities, each independent but composable:

1. Step dependencies — let captured functions declare that they need the output of other captured functions, so Sacred can run them in the right order
2. Config composition / external config — Hydra-style config groups, file-based configs, better CLI overrides
3. Grid search / sweeps — run the same experiment across many config combinations

All three should preserve Sacred's existing programming model and observer protocol.

---

3. Step Dependencies (Replacing Hamilton)

The Problem

Today, Sacred has no idea that train() needs the output of load_data(). You call load_data() yourself inside train() or wire things up manually in your @ex.main:

@ex.main
def my_main():
    data = load_data()      # You manage the call order
    model = build_model()
    result = train(data, model)  # You pass outputs manually
    return result

Hamilton solves this by matching function parameter names to other functions' names — if you have def train(load_data: pd.DataFrame), Hamilton infers that train depends on a function named load_data. This is clever but creates real problems: accidental name collisions, confusing error messages when names don't match, and no way to reuse a function under a different name.

The Proposal: Explicit @ex.step with Dep() Annotations

Instead of magic name-matching, dependencies are declared explicitly using typing.Annotated:

from sacred import Experiment, Dep
from typing import Annotated

ex = Experiment("my_exp")

@ex.config
def cfg():
    data_path = "train.csv"
    lr = 0.01

@ex.step
def load_data(data_path: str) -> pd.DataFrame:
    """data_path comes from config (normal Sacred behavior).
    Returns a DataFrame that other steps can depend on."""
    return pd.read_csv(data_path)

@ex.step
def build_model() -> nn.Module:
    return MyModel()

@ex.step
def train(
    data: Annotated[pd.DataFrame, Dep("load_data")],
    model: Annotated[nn.Module, Dep("build_model")],
    lr: float,  # No Dep annotation → comes from config
) -> dict:
    """'data' will receive the return value of load_data.
    'model' will receive the return value of build_model.
    'lr' comes from config, same as any captured function."""
    # ... training loop ...
    return {"accuracy": 0.95}

Why Dep() annotations instead of name-matching:

• Explicit is better than implicit. You can see exactly where each input comes from by reading the signature. No need to guess whether data refers to a step, a config value, or a local variable.
• Names are decoupled. The parameter name (data) doesn't have to match the step name (load_data). You can name your parameters for readability.
• Refactoring is safe. Rename a step → update the Dep("...") strings. Your IDE can find-and-replace. With name-matching, renaming a function silently breaks downstream functions.
• Multiple outputs work naturally. If a step returns a NamedTuple, you can depend on a specific field: Dep("train.accuracy").

How @ex.step Works Under the Hood

@ex.step is built on top of @ex.capture — it's a captured function with extra metadata:

# On Ingredient class:
def step(self, function):
    """Register a function as a DAG step. It's also a captured function."""
    captured = self.capture(function)

    # Extract Dep annotations from the signature
    deps = {}
    for param_name, param in inspect.signature(function).parameters.items():
        annotation = param.annotation
        if get_origin(annotation) is Annotated:
            for meta in annotation.__metadata__:
                if isinstance(meta, Dep):
                    deps[param_name] = meta.step_name

    captured._step_deps = deps
    captured._step_name = function.__name__
    captured._is_step = True
    self._steps.append(captured)
    return captured

The Dep class itself is trivial:

class Dep:
    """Annotation marking a parameter as depending on another step's output."""
    def __init__(self, step_name: str):
        self.step_name = step_name

Why This Is AST-Free (and Why That Matters)

Sacred's ConfigScope (@ex.config) is the fragile part of the system. It calls
inspect.getsourcelines → ast.parse → compile → eval to extract variable
assignments from a function body. This breaks in REPLs, notebooks, and marimo
because getsourcelines can't always find the source.

The @ex.step + Dep() design avoids AST entirely. Everything happens through
runtime type introspection:

# Registration (decorator time) — no AST:
get_type_hints(func, include_extras=True)   # extracts Dep annotations
inspect.signature(func)                      # parameter names + defaults

# Execution (call time) — no AST:
Signature.construct_arguments(...)           # Sacred's existing config injection

The key enabler is ParamSpec. A ParamSpec-preserving wrapper can add behavior
(caching, observer events, dep injection) around the original function while
keeping its full signature visible to both type checkers and inspect.signature:

P = ParamSpec("P")
R = TypeVar("R")

def step_wrapper(func: Callable[P, R]) -> Callable[P, R]:
    @functools.wraps(func)
    def inner(*args: P.args, **kwargs: P.kwargs) -> R:
        # inject deps from DAG outputs, fill config from Sacred
        return func(*args, **kwargs)
    return inner

Sacred's Signature.construct_arguments is fully compatible with Annotated types —
it sees all parameters normally and fills them from the config dict without caring
about the Dep() metadata. So dep params get filled by the DAG executor, config
params get filled by Sacred's existing system, and neither path touches AST.

This also means the entire step layer works in marimo, Jupyter, IPython, and
exec() — anywhere inspect.getsourcelines would fail.

The answer is yes — @ex.config should also move away from AST. See section 4.0
("Killing ConfigScope") for the full design: replace ConfigScope with ConfigFunction,
a plain callable that takes overrides as kwargs and returns a dict. Computed values
recompute naturally, no DogmaticDict or eval() needed. The entire system becomes
AST-free.

DAG Construction and Execution

A new StepDAG class handles ordering and execution:

class StepDAG:
    def __init__(self, steps):
        self.steps = {s._step_name: s for s in steps}
        self.deps = {s._step_name: s._step_deps for s in steps}

    def topological_order(self):
        """Standard topological sort — nothing fancy."""
        # Returns list of step names in valid execution order
        ...

    def execute(self, config, run=None):
        """Run all steps in order, passing outputs to dependents."""
        outputs = {}
        for step_name in self.topological_order():
            step_func = self.steps[step_name]
            # Collect inputs from upstream outputs
            step_inputs = {}
            for param_name, dep_name in self.deps[step_name].items():
                step_inputs[param_name] = outputs[dep_name]
            # Call the captured function — Sacred fills in config params,
            # we additionally pass step outputs as explicit kwargs
            outputs[step_name] = step_func(**step_inputs)
        return outputs

    def visualize(self):
        """Render the DAG as a graph (for marimo or terminal)."""
        ...

How It Fits Into Sacred's Run

The modification to Run.__call__() is minimal:

class Run:
    def __call__(self, *args):
        # ... existing setup (emit_started, heartbeat, pre_run_hooks) ...

        if self.step_dag is not None:
            # NEW: DAG mode
            self.result = self.step_dag.execute(self.config, run=self)
        else:
            # EXISTING: single main function
            self.result = self.main_function(*args)

        # ... existing teardown (post_run_hooks, emit_completed) ...

If you don't use @ex.step, nothing changes. If you do, the DAG replaces @ex.main.

Alternative: @ex.step(deps=...) Without Annotations

If you prefer not to use typing.Annotated, the same information can go in the decorator:

@ex.step(deps={"data": "load_data", "model": "build_model"})
def train(data, model, lr: float) -> dict:
    ...

Or as a dict-of-steps:

@ex.step(after=["load_data", "build_model"])
def train(load_data_result, build_model_result, lr: float) -> dict:
    ...

The Dep() annotation approach and the decorator-kwarg approach can coexist — they're two ways to declare the same information. The annotation style is more self-documenting in the signature; the decorator style is more compact. Sacred can support both.

Caching

Steps naturally support caching since they have defined inputs and outputs:

@ex.step(cache=True)
def load_data(data_path: str) -> pd.DataFrame:
    return pd.read_csv(data_path)

But caching is just one thing the dep graph unlocks. The graph gives us step identity, and step identity gives us caching, expiry, parallelism, per-step artifacts, and cross-run versioning. The rest of this section works through each of those.

---

3b. What the Dep Graph Unlocks

Step Identity: The Fingerprint

Every step has a fingerprint — a hash that answers "has anything about this computation changed?" A fingerprint is built from three things:

fingerprint = hash(
    source_hash,            # hash of the function's source code
    config_inputs,          # the config values this step actually uses
    upstream_fingerprints,  # fingerprints of the steps this step depends on
)

This is recursive. If featurize depends on load_data, then featurize's fingerprint includes load_data's fingerprint. If load_data's source code changes, featurize's fingerprint changes too — even if featurize itself didn't change.

Critically, a step's fingerprint only includes the config values that step actually reads. The @ex.step decorator already knows which parameters have Dep() annotations (come from upstream) and which don't (come from config). So:

@ex.step
def train(
    features: Annotated[np.ndarray, Dep("featurize")],  # not in fingerprint's config_inputs
    lr: float = 0.01,    # IS in fingerprint's config_inputs
    epochs: int = 10,     # IS in fingerprint's config_inputs
) -> dict:
    ...

If you change data_path (used by load_data, not by train), train's direct config inputs don't change — but its upstream fingerprints do, because load_data's fingerprint changed. So train's fingerprint changes too. The change correctly propagates through the graph.

If you change lr, only train's fingerprint changes. load_data and featurize are unaffected and can be served from cache.

I prototyped this and verified it works:

config = {data_path: "train.csv", n_features: 100, lr: 0.01, ...}

load_data:   ae8800d4  inputs={data_path, normalize}
build_model: 690c5875  inputs={architecture}
featurize:   82349ac6  inputs={n_features}  upstream={load_data: ae8800d4}
train:       1596ee1e  inputs={lr, epochs}  upstream={featurize: 82349ac6, build_model: 690c5875}

--- After changing only lr=0.1 ---
load_data:   ae8800d4  changed=False  ← cache hit
build_model: 690c5875  changed=False  ← cache hit
featurize:   82349ac6  changed=False  ← cache hit
train:       3784fb7a  changed=True   ← must re-run

Caching

The StepRecord

When a step executes, we store a StepRecord alongside its output:

@dataclass
class StepRecord:
    step_name: str
    fingerprint: str              # the fingerprint that produced this output
    source_hash: str              # hash of function source alone
    config_inputs: dict           # config values this step consumed
    upstream_fingerprints: dict   # {dep_name: fingerprint}
    created_at: float             # timestamp
    elapsed_seconds: float        # how long the step took
    output_path: str              # path to serialized output
    output_hash: str              # hash of the serialized output
    run_id: str                   # Sacred run ID
    ttl_seconds: float = 0        # time-to-live (0 = no time expiry)
    tags: dict = field(default_factory=dict)

This record is both the cache metadata and the audit trail. It tells you: what ran, when, with what inputs, how long it took, and where the output lives.

Cache Lookup

Before executing a step, check the cache:

def should_execute(step, current_fingerprint, cache):
    cached_output, record = cache.get(step.name, current_fingerprint)
    if cached_output is not None:
        return False, cached_output   # cache hit
    return True, None                  # cache miss

cache.get() checks three things in order:

1. Does a record exist for this step name?
2. Does the stored fingerprint match the current fingerprint? (source, config, or upstream changed → miss)
3. Has the TTL expired? (timed expiry → miss)

If all three pass, return the cached output. Otherwise, re-execute.

Storage

The cache store is a directory next to Sacred's observer storage:

runs/
  1/                        # Sacred run (FileStorageObserver)
    config.json
    run.json
    ...
  _cache/                   # Step cache (new)
    load_data/
      record.json           # StepRecord
      ae8800d4.pkl          # serialized output, keyed by fingerprint
    featurize/
      record.json
      82349ac6.pkl
    train/
      record.json
      1596ee1e.pkl

Outputs are serialized with pickle (or a configurable serializer — joblib, cloudpickle, etc. for objects pickle can't handle). Each fingerprint gets its own file, so old outputs stick around until explicitly pruned.

Expiry

Three kinds of expiry, composable:

1. Fingerprint expiry (automatic)

This is the default. If the fingerprint changes (source code edited, config value changed, upstream changed), the cache is invalid. No configuration needed — it's inherent in the fingerprint design.

@ex.step(cache=True)
def featurize(...):
    ...
# Edit featurize's source code → fingerprint changes → cache miss

2. Timed expiry (TTL)

For steps that read external data that might change:

@ex.step(cache=True, ttl=3600)  # invalidate after 1 hour
def load_data(data_url: str):
    return pd.read_csv(data_url)

@ex.step(cache=True, ttl="24h")  # human-readable
def fetch_from_api(endpoint: str):
    return requests.get(endpoint).json()

Implementation is simple — StepRecord.created_at + ttl < now() means expired.

3. Conditional expiry (custom predicate)

For when expiry depends on something outside the config, like a file's modification time or a database timestamp:

@ex.step(cache=True, expire_if=lambda: os.path.getmtime("data.csv") > last_cache_time)
def load_data():
    return pd.read_csv("data.csv")

Or more practically, a built-in file-hash check:

@ex.step(cache=True, watch_files=["data.csv", "labels.csv"])
def load_data():
    ...

watch_files adds the file hashes (or mtimes) to the fingerprint. If data.csv changes on disk, the fingerprint changes, the cache misses.

4. Manual invalidation

cache = ex.get_cache()
cache.invalidate("load_data")              # invalidate one step
cache.invalidate_downstream("load_data")   # invalidate it + everything that depends on it
cache.clear()                               # nuke everything

Parallel Execution

The DAG tells us which steps can run concurrently — any steps whose dependency sets don't overlap can execute at the same time.

load_data ──→ featurize ──→ train ──→ evaluate
build_model ─────────────↗

In this graph, load_data and build_model have no dependencies on each other. They can run in parallel. featurize waits for load_data. train waits for both featurize and build_model. The executor:

def execute_parallel(dag, config, max_workers=4):
    outputs = {}
    remaining = set(dag.steps.keys())

    with ThreadPoolExecutor(max_workers=max_workers) as pool:
        futures = {}
        while remaining:
            # Find steps whose deps are all satisfied
            ready = [name for name in remaining
                     if all(d in outputs for d in dag.upstream(name))]

            for name in ready:
                if name not in futures.values():
                    f = pool.submit(run_step, name, dag, outputs, config)
                    futures[f] = name

            # Wait for at least one to complete
            for f in as_completed(futures):
                name = futures.pop(f)
                outputs[name] = f.result()
                remaining.discard(name)

    return outputs

I prototyped this and measured: sequential = 1.72s, parallel = 1.42s for the graph above. The speedup grows with wider graphs (more independent branches).

Parallel + cache compose: if load_data is cached and build_model isn't, build_model starts immediately while load_data returns from cache instantly. featurize then starts as soon as load_data's cached output is ready, potentially overlapping with build_model's execution.

Thread vs process: ThreadPoolExecutor works for I/O-bound steps (data loading, API calls). For CPU-bound steps (training), ProcessPoolExecutor avoids the GIL. The executor is configurable:

result = ex.run(parallel=True, executor="process", max_workers=4)

Per-Step Artifacts and Observer Integration

New Observer Events

The DAG execution emits per-step events that observers can listen to:

class RunObserver:
    # Existing events (unchanged):
    def started_event(self, ...): ...
    def completed_event(self, ...): ...
    # ...

    # New step-level events (default no-op, backward compatible):

    def step_started_event(self, step_name, step_deps, config_inputs):
        """A step is about to execute."""
        pass

    def step_completed_event(self, step_name, elapsed, fingerprint, from_cache, output_hash):
        """A step finished (or was served from cache)."""
        pass

    def step_failed_event(self, step_name, elapsed, fail_trace):
        """A step raised an exception."""
        pass

    def dag_resolved_event(self, dag_spec):
        """The full DAG structure, emitted once before execution begins.
        dag_spec is a dict: {step_name: {deps: [...], config_inputs: [...], source_hash: ...}}"""
        pass

Existing observers (FileStorageObserver, MongoObserver) ignore these — their default is no-op. New step-aware observers can record detailed per-step info.

FileStorageObserver Extension

A StepFileStorageObserver (subclass) that writes per-step data:

runs/
  42/
    config.json              # existing
    run.json                 # existing, now includes dag_spec
    cout.txt                 # existing
    steps/                   # NEW
      load_data/
        record.json          # StepRecord (fingerprint, timing, etc.)
        output.pkl           # serialized output (optional)
      featurize/
        record.json
        output.pkl
      train/
        record.json
        output.pkl
    dag.json                 # NEW: full graph structure
    dag.dot                  # NEW: graphviz rendering

The dag.json is the key artifact for versioning:

{
  "steps": {
    "load_data": {
      "source_hash": "ae8800d4",
      "config_inputs": {"data_path": "train.csv"},
      "deps": [],
      "fingerprint": "ae8800d4d287",
      "elapsed": 2.3,
      "from_cache": false
    },
    "featurize": {
      "source_hash": "f3a1b2c3",
      "config_inputs": {"n_features": 100},
      "deps": ["load_data"],
      "fingerprint": "82349ac69ea1",
      "elapsed": 0.0,
      "from_cache": true
    }
  },
  "edges": [
    ["load_data", "featurize"],
    ["featurize", "train"],
    ["build_model", "train"],
    ["train", "evaluate"]
  ]
}

Cross-Run Versioning and Diffing

With dag.json stored per run, we can answer questions across runs:

"What changed between run 42 and run 43?"

from sacred.step_diff import diff_runs

diff = diff_runs("runs/42/dag.json", "runs/43/dag.json")
print(diff)

Output:

Step changes:
  load_data:   unchanged (fingerprint ae8800d4 in both)
  featurize:   unchanged (fingerprint 82349ac6 in both)
  build_model: SOURCE CHANGED (690c5875 → a1b2c3d4)
  train:       CONFIG CHANGED (lr: 0.01 → 0.1)
  evaluate:    UPSTREAM CHANGED (train fingerprint changed)

Config diff:
  lr: 0.01 → 0.1

Steps re-executed: build_model, train, evaluate
Steps from cache:  load_data, featurize

This falls out naturally from comparing fingerprints and their components.

"Which runs used the same load_data?"

from sacred.step_query import find_runs_by_step

runs = find_runs_by_step("runs/", step_name="load_data", fingerprint="ae8800d4")
# Returns: [42, 43, 44, 47]  — all runs where load_data had this fingerprint

This is just a scan of dag.json files. With a database observer (Mongo, SQL), it's a query.

"Show me the lineage of this result"

from sacred.step_lineage import trace_lineage

lineage = trace_lineage("runs/43/dag.json", "evaluate")
print(lineage)

Output:

evaluate (fingerprint: d4e5f6)
  ← train (fingerprint: 3784fb, config: {lr: 0.1, epochs: 10})
    ← featurize (fingerprint: 82349a, config: {n_features: 100}, CACHED)
      ← load_data (fingerprint: ae8800, config: {data_path: "train.csv"}, CACHED)
    ← build_model (fingerprint: a1b2c3, config: {architecture: "resnet"})

Every result has a complete provenance chain back to raw data.

Run-to-Run Artifact Comparison

Since step outputs are stored with their fingerprints, you can pull outputs from different runs and compare:

run42_train_output = cache.get_by_run("42", "train")
run43_train_output = cache.get_by_run("43", "train")

# Same load_data, same featurize, different lr → compare training results directly
# You know the ONLY difference is lr because the fingerprints tell you so

How Sweeps Interact With All of This

The cache makes sweeps much more efficient. In a sweep over lr=[0.001, 0.01, 0.1]:

Trial 1 (lr=0.001):
  load_data:   EXECUTE (2.3s)   → cache with fingerprint ae8800
  build_model: EXECUTE (0.5s)   → cache with fingerprint 690c58
  featurize:   EXECUTE (1.2s)   → cache with fingerprint 82349a
  train:       EXECUTE (45.0s)  → fingerprint includes lr=0.001
  evaluate:    EXECUTE (3.0s)

Trial 2 (lr=0.01):
  load_data:   CACHE HIT (0.01s)  ← same fingerprint
  build_model: CACHE HIT (0.01s)  ← same fingerprint
  featurize:   CACHE HIT (0.01s)  ← same fingerprint
  train:       EXECUTE (44.0s)    ← different fingerprint (lr changed)
  evaluate:    EXECUTE (3.0s)

Trial 3 (lr=0.1):
  load_data:   CACHE HIT
  build_model: CACHE HIT
  featurize:   CACHE HIT
  train:       EXECUTE (43.0s)
  evaluate:    EXECUTE (3.0s)

Without cache: 3 × (2.3 + 0.5 + 1.2 + ~44 + 3.0) = ~153s
With cache:    (2.3 + 0.5 + 1.2) + 3 × (~44 + 3.0) = ~145s (saves upstream)

The savings grow with more expensive upstream steps and more sweep trials. For a 100-trial sweep where only lr varies, you run load_data and featurize once instead of 100 times.

The sweep's SweepResult can also include per-trial DAG info:

results = ex.run_sweep(Sweep(lr=[0.001, 0.01, 0.1]))
for run in results.runs:
    print(f"lr={run.config['lr']}")
    for step_name, record in run.step_records.items():
        print(f"  {step_name}: {'CACHED' if record.from_cache else f'{record.elapsed:.1f}s'}")

Putting It Together: The Full Step Execution Flow

Here's what happens when Run.__call__() fires with a DAG:

1. Run.__call__() starts
2. _emit_started()
3. _emit_dag_resolved(dag_spec)           ← NEW: full graph structure sent to observers

4. For each step in topological order (or parallel):
   a. Compute fingerprint for this step
   b. Check cache:
      - fingerprint match? TTL ok? watch_files unchanged? expire_if predicate?
      - HIT:  _emit_step_completed(name, elapsed=0, from_cache=True)
              Use cached output
      - MISS: _emit_step_started(name)
              Execute the step (Sacred fills config params, DAG fills upstream outputs)
              Store output + StepRecord in cache
              _emit_step_completed(name, elapsed, from_cache=False)
              OR _emit_step_failed(name) if exception

5. Collect final outputs
6. _emit_completed(result)

Observers see every step's lifecycle. The cache is transparent — downstream steps don't know or care whether their inputs came from cache or fresh execution. The fingerprint system guarantees correctness: if the cache says it's a hit, the output is identical to what fresh execution would produce (assuming deterministic steps — stochastic steps should set cache=False or use seeds in the fingerprint, which Sacred already generates).

---

4. Config Composition & External Config (Replacing Hydra)

What Sacred Already Has

Sacred already handles most of what Hydra does, just with different names:

Hydra Concept	Sacred Equivalent	Gap
Config file (config.yaml)	ex.add_config("file.yaml")	None — works today
Config group (model/resnet.yaml)	@ex.named_config	Need file-based discovery
CLI overrides (model.lr=0.01)	python train.py with lr=0.01	Need dotted path for nested
Composition (select one per group)	ex.run(named_configs=["fast"])	Need multi-group selection
Multirun / sweep	Nothing	Need to build

So the gaps are actually small. Here's how to fill them.

But first — there's a bigger opportunity here: dropping the AST entirely.

4.0 Killing ConfigScope (Dropping AST Compilation)

ConfigScope is the only part of Sacred that uses AST compilation. When you write:

@ex.config
def cfg():
    input_size = 784
    hidden_size = input_size * 2

Sacred calls inspect.getsourcelines(cfg) → ast.parse → compile → eval inside a
DogmaticDict. The DogmaticDict silently blocks writes to "fixed" keys (CLI overrides),
so input_size gets pinned and hidden_size recomputes off the pinned value.

This is clever, but it's also the single thing that breaks in marimo, Jupyter, IPython,
and exec(). It makes Sacred source-location-dependent. We should kill it.

The replacement: config as a plain function that returns a dict.

@ex.config
def cfg(input_size=784, use_gpu=True):
    hidden_size = input_size * 2
    batch_size = 64 if use_gpu else 8
    device = "cuda" if use_gpu else "cpu"
    return dict(
        input_size=input_size,
        hidden_size=hidden_size,
        batch_size=batch_size,
        device=device,
        use_gpu=use_gpu,
    )

The new @ex.config wraps this in a ConfigFunction (not ConfigScope):

class ConfigFunction:
    """Replaces ConfigScope. No AST, no getsourcelines, no eval."""

    def __init__(self, func):
        self._func = func
        self._sig = inspect.signature(func)

    def __call__(self, fixed=None, preset=None, fallback=None):
        fixed = fixed or {}
        preset = preset or {}

        # Split overrides into function params vs post-hoc overrides
        param_names = set(self._sig.parameters.keys())
        param_overrides = {k: v for k, v in fixed.items() if k in param_names}
        post_overrides = {k: v for k, v in fixed.items() if k not in param_names}

        # Merge preset into param overrides (same priority as ConfigScope)
        for k, v in preset.items():
            if k in param_names and k not in param_overrides:
                param_overrides[k] = v

        # Call the function — computed values recompute naturally
        result = self._func(**param_overrides)

        # Pin directly-overridden computed values
        result.update(post_overrides)

        # Build ConfigSummary (same interface as before)
        defaults = self._func()   # call with no overrides to get baseline
        added = {k for k in result if k not in defaults}
        modified = {k for k in result if k in defaults and result[k] != defaults[k]}

        return ConfigSummary(added, modified, ...)

What changes, what doesn't:

Behavior	ConfigScope (AST)	ConfigFunction (no AST)	Verdict
Simple key-value config	auto-captures locals	explicit return dict	explicit is better
Computed values recompute on override	DogmaticDict blocks writes, re-evals	function params as kwargs, re-calls	identical result, cleaner mechanism
Override a computed value directly	DogmaticDict silently wins	post_overrides dict.update()	more predictable
Conditionals (if/else in config)	re-evals with pinned values	function re-runs with new args	identical
Inline comments as docs	AST extracts # comments	lost — use docstring or Field()	rarely used, acceptable loss
Works in marimo/Jupyter/REPL	breaks (getsourcelines fails)	works everywhere	the whole point
Named configs	also uses ConfigScope	same ConfigFunction pattern	works

The one behavioral difference: ConfigScope auto-captures every local variable.
ConfigFunction requires an explicit return dict(...). This is strictly better — it's
the same reason Python 3 removed exec leaking locals. You know exactly what's in
your config by reading the return statement.

Migration path: Keep ConfigScope working for backward compat behind a flag.
New code uses ConfigFunction. The @ex.config decorator auto-detects: if the
function has a return statement annotation or returns something, use ConfigFunction;
if it doesn't, fall back to ConfigScope with a deprecation warning.

4.1 Config Groups from Directories

Add one method to Ingredient:

def add_config_group(self, group_name, directory):
    """Auto-discover YAML/JSON files in a directory as named configs."""
    for filename in sorted(os.listdir(directory)):
        if filename.endswith(('.yaml', '.yml', '.json')):
            name = os.path.splitext(filename)[0]
            full_name = f"{group_name}.{name}"
            self.add_named_config(full_name, os.path.join(directory, filename))

Directory layout:

configs/
  optimizer/
    adam.yaml       → named config "optimizer.adam"
    sgd.yaml        → named config "optimizer.sgd"
  model/
    resnet.yaml     → named config "model.resnet"
    vit.yaml        → named config "model.vit"

Usage:

ex.add_config_group("optimizer", "configs/optimizer")
ex.add_config_group("model", "configs/model")

# Use exactly like existing named configs:
ex.run(named_configs=["optimizer.sgd", "model.resnet"])

# Or from CLI:
# python train.py with optimizer.sgd model.resnet lr=0.05

This uses Sacred's existing named config machinery. create_run() already handles named configs in Phase 2 — it evaluates them and merges the results. Zero changes to the config pipeline.

4.2 Typed Config with Dataclasses (Optional)

For users who want type checking, a thin wrapper around dataclasses:

from dataclasses import dataclass, asdict

@dataclass
class TrainConfig:
    lr: float = 0.001
    epochs: int = 100
    batch_size: int = 32

# Just convert to dict — Sacred's existing add_config handles the rest
ex.add_config(asdict(TrainConfig()))

# Reconstruct in captured functions if you want the typed object back:
@ex.capture
def train(_config):
    cfg = TrainConfig(**{k: _config[k] for k in TrainConfig.__dataclass_fields__
                         if k in _config})
    # cfg.lr, cfg.epochs, cfg.batch_size are all typed

A convenience utility:

# sacred/utils.py
def from_config(config_dict, dataclass_type):
    """Reconstruct a dataclass from a Sacred config dict."""
    fields = dataclass_type.__dataclass_fields__
    return dataclass_type(**{k: config_dict[k] for k in fields if k in config_dict})

# Usage:
@ex.capture
def train(_config):
    cfg = from_config(_config, TrainConfig)

No changes to Sacred's config engine needed. Dataclasses are just a nicer way to define and consume config — Sacred still sees plain dicts internally.

4.3 Config Defaults File

For projects that want a single "base config" file (like Hydra's config.yaml):

# Load defaults from a file, then layer on named configs and CLI overrides
ex.add_config("configs/defaults.yaml")

# Sacred's existing priority: CLI overrides > named configs > add_config > @ex.config
# This already works correctly.

---

5. Grid Search / Sweeps

5.1 The Simple Version

A sweep is just a loop over ex.run() with different config_updates. This already works today:

for lr in [0.001, 0.01, 0.1]:
    for batch_size in [32, 64]:
        run = ex.run(config_updates={"lr": lr, "batch_size": batch_size})
        print(f"lr={lr}, bs={batch_size} → {run.result}")

Each call to ex.run() creates a fully independent Sacred Run with its own observer events. FileStorageObserver logs each as a separate numbered directory. Mongo logs each as a separate document. No changes needed.

5.2 A Nicer API

Wrap the loop in a class:

from itertools import product

class Sweep:
    def __init__(self, **param_grids):
        """Each kwarg is a param name → list of values to try."""
        self.param_grids = param_grids

    def configs(self):
        """Yield config_update dicts for every combination."""
        keys = list(self.param_grids.keys())
        for combo in product(*self.param_grids.values()):
            yield dict(zip(keys, combo))

    def __len__(self):
        result = 1
        for v in self.param_grids.values():
            result *= len(v)
        return result

Add a method on Experiment:

class Experiment(Ingredient):
    def run_sweep(self, sweep, named_configs=(), workers=1):
        """Run all configs from a Sweep. Returns SweepResult."""
        runs = []
        for config_updates in sweep.configs():
            run = self.run(config_updates=config_updates,
                          named_configs=named_configs)
            runs.append(run)
        return SweepResult(runs)

Usage:

sweep = Sweep(
    lr=[0.001, 0.01, 0.1],
    batch_size=[32, 64, 128],
)

results = ex.run_sweep(sweep)
print(results.best("accuracy"))      # Config with highest accuracy
print(results.to_dataframe())        # All runs as a table

5.3 SweepResult

class SweepResult:
    def __init__(self, runs):
        self.runs = runs

    def best(self, metric, mode="max"):
        """Return the Run with the best metric value."""
        key = (max if mode == "max" else min)
        return key(self.runs, key=lambda r: r.result.get(metric, float('-inf')))

    def to_dataframe(self):
        """Config + results as a pandas DataFrame."""
        import pandas as pd
        rows = []
        for run in self.runs:
            row = dict(run.config)
            if isinstance(run.result, dict):
                row.update(run.result)
            else:
                row["result"] = run.result
            rows.append(row)
        return pd.DataFrame(rows)

5.4 DAG-Aware Caching for Sweeps

If the experiment uses @ex.step and the sweep only varies downstream parameters, upstream steps don't need to re-run:

# If sweeping over lr, and load_data doesn't depend on lr,
# load_data runs once, train runs 3 times
sweep = Sweep(lr=[0.001, 0.01, 0.1])
results = ex.run_sweep(sweep)  # With step caching, load_data is automatic

This works automatically if steps have cache=True — the cache key includes only the step's actual inputs, not all config values.

5.5 Parallel Sweeps

from concurrent.futures import ProcessPoolExecutor

class Experiment(Ingredient):
    def run_sweep(self, sweep, named_configs=(), workers=1):
        if workers == 1:
            return SweepResult([
                self.run(config_updates=cu, named_configs=named_configs)
                for cu in sweep.configs()
            ])
        else:
            with ProcessPoolExecutor(max_workers=workers) as pool:
                futures = [
                    pool.submit(self.run, config_updates=cu, named_configs=named_configs)
                    for cu in sweep.configs()
                ]
                return SweepResult([f.result() for f in futures])

5.6 Observer Protocol for Sweeps

Optional new events (backward compatible — existing observers ignore them):

class RunObserver:
    # All existing events stay exactly the same...

    # New optional events:
    def sweep_started_event(self, sweep_spec, n_trials):
        """Called before a sweep. Default: no-op."""
        pass

    def sweep_completed_event(self, results_summary):
        """Called after all sweep trials. Default: no-op."""
        pass

Individual trial Runs emit the standard started_event/completed_event as always.

---

6. Marimo Integration

6.1 What Already Works

All of Sacred's core features work inside marimo cells today (tested). The usage pattern:

import marimo as mo

app = mo.App()

@app.cell
def setup():
    from sacred import Experiment
    from sacred.observers import FileStorageObserver
    ex = Experiment("my_exp", interactive=True)
    ex.observers.append(FileStorageObserver("runs/"))
    @ex.config
    def my_config():       # Don't use underscore prefix!
        lr = 0.01
        epochs = 10
    @ex.main
    def train(lr, epochs):
        return {"loss": 1.0 / (lr * epochs)}
    return ex,

@app.cell
def run_it(ex):
    result = ex.run()
    print(result.result)
    return result,

6.2 Config UI from Sacred Config

A small bridge module that generates marimo widgets from Sacred's config:

# sacred/contrib/marimo.py
import marimo as mo

def config_ui(experiment):
    """Generate interactive UI elements from a Sacred experiment's config."""
    # Do a dry run to get the resolved config
    run = experiment._create_run(options={"--unobserved": True})
    config = run.config

    elements = {}
    for key, value in config.items():
        if key == "seed":
            continue
        if isinstance(value, bool):
            elements[key] = mo.ui.switch(value=value, label=key)
        elif isinstance(value, float):
            elements[key] = mo.ui.number(value=value, label=key, step=value/10 or 0.001)
        elif isinstance(value, int):
            elements[key] = mo.ui.number(value=value, label=key, step=1)
        elif isinstance(value, str):
            elements[key] = mo.ui.text(value=value, label=key)
        elif isinstance(value, list):
            elements[key] = mo.ui.text(value=str(value), label=key)
    return mo.ui.dictionary(elements)

Usage in marimo:

@app.cell
def config_cell(ex):
    from sacred.contrib.marimo import config_ui
    ui = config_ui(ex)
    mo.output.replace(ui)
    return ui,

@app.cell
def run_cell(ex, ui):
    # ui.value is a dict — feeds directly into Sacred's config_updates
    result = ex.run(config_updates=ui.value)
    return result,

Slide a slider → marimo re-runs run_cell → new Sacred Run with new config. Each run is fully logged by observers.

6.3 Sweep Explorer

@app.cell
def sweep_cell(ex):
    from sacred import Sweep
    sweep = Sweep(lr=[0.001, 0.01, 0.1], epochs=[10, 50, 100])
    results = ex.run_sweep(sweep)
    df = results.to_dataframe()
    mo.ui.table(df)
    return results,

6.4 The Name-Mangling Pitfall

Marimo rewrites _-prefixed function names to be cell-private. Sacred registers things by func.__name__. So:

# BAD in marimo:
@ex.named_config
def _fast():         # Becomes "_cell_Xxxx_fast" — unreachable as "fast"
    lr = 0.1

# GOOD in marimo:
@ex.named_config
def fast():          # Stays "fast"
    lr = 0.1

A future fix: add an optional name= parameter to Sacred's decorators:

@ex.named_config(name="fast")
def _whatever():     # Name doesn't matter — Sacred uses "fast"
    lr = 0.1

This is a small change to Ingredient.named_config() — use the explicit name if provided, otherwise fall back to func.__name__.

---

7. Summary: What Changes Where

Feature	New Code	Modified Code	Observer Impact
@ex.step + Dep()	sacred/step.py	ingredient.py	None yet
StepDAG execution	sacred/dag.py	run.py, initialize.py	step_started/completed/failed, dag_resolved
StepCache + fingerprints	sacred/cache.py	dag.py	step_completed gets from_cache
Parallel DAG execution	sacred/dag.py	None	None
Per-step artifacts	sacred/observers/step_storage.py	FileStorageObserver (subclass)	steps/ subdirectory
Cross-run versioning	sacred/step_diff.py	None	dag.json stored per run
ConfigFunction (kill AST)	sacred/config/config_function.py	ingredient.py, config_scope.py (deprecated)	None
Config groups	None (new method)	ingredient.py	None
Dataclass config util	sacred/utils.py	None	None
Sweep + SweepResult	sacred/sweep.py	experiment.py	sweep_started/completed
Marimo bridge	sacred/contrib/marimo.py	None	None
Name-mangling fix	None	ingredient.py decorators	None

Everything is additive. Existing experiments with @ex.main + @ex.capture work identically. The observer protocol gains optional events that default to no-ops, so existing observers don't break.

---

8. Open Questions

1. Step return types: Should steps be required to have return type annotations? It helps with DAG visualization and type-checking between connected steps, but Sacred has never required type hints before.

2. Step vs main coexistence: If someone has both @ex.main and @ex.step, what wins? Proposal: @ex.main takes precedence with a warning that steps are being ignored.

3. Cache serialization: Pickle is the default, but some objects aren't picklable (database connections, generators). Support pluggable serializers (joblib, cloudpickle, custom)? Or just skip caching for unpicklable outputs with a warning?

4. Cache sharing across experiments: Should two different experiments be able to share a cache if they have identical steps? The fingerprint system already enables this (same source + same inputs = same fingerprint), but sharing needs a shared cache directory.

5. Stochastic steps and caching: Sacred auto-generates seeds. If a step uses _seed or _rnd, should the seed be part of the fingerprint? Probably yes — otherwise cached results from a different seed would be incorrectly reused. This means @ex.step(cache=True) on a stochastic step caches per-seed, which is correct but uses more cache space.

6. Sweep observer grouping: Should sweep trials share a group ID? FileStorageObserver could use sweep_1/trial_1/, sweep_1/trial_2/ etc. This needs a sweep_id field on Run.meta_info.

7. Parallel execution and observer thread safety: Sacred's observers weren't designed for concurrent step_completed_event calls from multiple threads. Options: serialize observer calls behind a lock, use a queue, or document that step-aware observers must be thread-safe.

8. Config validation: With dataclass configs, should Sacred validate types at run time? Currently it only warns about type changes (e.g., int → str). Full validation would catch bugs earlier but is a behavior change.

9. Graph visualization in marimo: The DAG can render as graphviz DOT, mermaid, or an interactive d3 graph. Which format should be the default for marimo cells? Mermaid is probably simplest since marimo renders it natively.

[vnmabus]

My 50 cents: I do not use Sacred very often since I finished my PhD, but if I knew that this project has any vibe-coded part (specially given how much human-made magic is already in it) I would not trust it to handle my experiments. The only way I trusted this project (which was hard for me to fully understand) is because I assumed it to be written and reviewed thoroughly by (smart) humans.

Also, from the emotional side: Sacred was the first project in which I contributed a PR, and thanks to that I can now proudly say that I am an open-source developer. It is already hard to see that it is a bit abandoned, but I hope than the maintainers resist the pressure to let "vibe-coded" AI slop into it.