Skip to content

Commit 62096fc

Browse files
author
DhanashreePetare
committed
docs: document manifest replay for download, delete, and deploy
1 parent 0abf185 commit 62096fc

3 files changed

Lines changed: 94 additions & 3 deletions

File tree

‎README.md‎

Lines changed: 73 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@ Command-line and Python client for downloading and deploying datasets on DBpedia
1919
- [Deploy](#cli-deploy)
2020
- [Delete](#cli-delete)
2121
- [Manifest](#cli-manifest)
22+
- [Replay](#cli-manifest-replay)
2223
- [Module Usage](#module-usage)
2324
- [Deploy](#module-deploy)
2425
- [Development & Contributing](#development--contributing)
@@ -149,8 +150,10 @@ Options:
149150
--help Show this message and exit.
150151

151152
Commands:
153+
delete Delete a dataset from the databus.
152154
deploy Flexible deploy to Databus command supporting three modes:
153155
download Download datasets from databus, optionally using vault access...
156+
manifest Manifest utilities.
154157
```
155158

156159
<a id="cli-download"></a>
@@ -609,6 +612,76 @@ The manifest records input parameters, per-file URLs, checksums, byte sizes, tim
609612
610613
Refer [examples/reproducible-download.md](examples/reproducible-download.md) for a full walkthrough.
611614
615+
<a id="cli-manifest-replay"></a>
616+
#### Replay
617+
618+
Any manifest written with `--manifest` can be replayed later using `databusclient manifest replay <path>`. Replay re-executes the original operation using the parameters recorded in the manifest — you don't need to remember or retype the original command.
619+
620+
```bash
621+
# Python
622+
databusclient manifest replay [OPTIONS] MANIFEST_PATH
623+
# Docker
624+
docker run --rm -v $(pwd):/data dbpedia/databus-python-client manifest replay [OPTIONS] MANIFEST_PATH
625+
```
626+
627+
**Important:** credentials are never stored in the manifest and must always be supplied fresh at replay time — `--vault-token`, `--databus-key`, and `--apikey` behave exactly as they do on the original commands.
628+
629+
```bash
630+
databusclient manifest replay --help
631+
632+
# Output:
633+
Usage: databusclient manifest replay [OPTIONS] MANIFEST_PATH
634+
635+
Replay a previously recorded manifest operation.
636+
637+
Currently supports replay of download, delete, and deploy manifests.
638+
For delete manifests, an interactive confirmation is required by
639+
default -- use --force to skip it for scripted/unattended use, or
640+
--dry-run to preview without prompting or deleting.
641+
642+
Options:
643+
--localdir TEXT Override local output directory for download replay.
644+
--databus TEXT Override Databus endpoint for replay.
645+
--vault-token TEXT Vault token file path, required if manifest auth
646+
method is vault_token.
647+
--databus-key TEXT Databus API key, required if manifest auth method is
648+
databus_key. Also required for delete replay.
649+
--apikey TEXT Databus API key, required for deploy replay.
650+
--force For delete replay: skip the interactive confirmation
651+
prompt. Required for unattended/scripted replay.
652+
--dry-run For delete replay: force a dry-run preview even if
653+
the original operation wasn't one.
654+
--help Show this message and exit.
655+
```
656+
657+
**Replaying a download:**
658+
```bash
659+
databusclient manifest replay ./manifests/download-run.jsonld --localdir ./replayed-data
660+
```
661+
If `--localdir` is omitted, replay falls back to the same auto-computed folder structure a fresh download would use — this is not necessarily the same folder the original download used, since the original folder location itself is never stored in the manifest.
662+
663+
**Replaying a delete:** by default, replay asks for confirmation before deleting, exactly like a normal `delete` call:
664+
```bash
665+
databusclient manifest replay ./manifests/delete-run.jsonld --databus-key YOUR_API_KEY
666+
# About to replay a DELETE operation for the following 1 URI(s):
667+
# - https://databus.dbpedia.org/...
668+
# This is irreversible. Proceed? [y/N]:
669+
```
670+
For unattended/scripted use (e.g. CI/CD), skip the prompt with `--force`:
671+
```bash
672+
databusclient manifest replay ./manifests/delete-run.jsonld --databus-key YOUR_API_KEY --force
673+
```
674+
If the original delete was run with `--dry-run --manifest ...`, replay automatically previews without deleting — no flag needed. You can also force a preview on a manifest that wasn't originally a dry run:
675+
```bash
676+
databusclient manifest replay ./manifests/delete-run.jsonld --databus-key YOUR_API_KEY --dry-run
677+
```
678+
679+
**Replaying a deploy:** supported for classic (distributions-as-arguments) and metadata-file deploys. The manifest stores fully-resolved deployment metadata (checksums, sizes, formats already computed), so replay never re-downloads or re-hashes the original files:
680+
```bash
681+
databusclient manifest replay ./manifests/deploy-run.jsonld --apikey YOUR_API_KEY
682+
```
683+
Replaying redeploys the same version — if it already exists on Databus, it is updated. WebDAV/Nextcloud deploys cannot be replayed, since the originally uploaded local files may no longer exist at their original paths by the time replay runs.
684+
612685
## Module Usage
613686
614687
<a id="module-deploy"></a>

‎databusclient/cli.py‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -514,10 +514,11 @@ def manifest_replay(manifest_path, localdir, databus, vault_token, databus_key,
514514
"""
515515
Replay a previously recorded manifest operation.
516516
517-
Currently supports replay of download and delete manifests.
517+
Currently supports replay of download, delete, and deploy manifests.
518518
For delete manifests, an interactive confirmation is required by
519519
default -- use --force to skip it for scripted/unattended use, or
520-
--dry-run to preview without prompting or deleting.
520+
--dry-run to preview without prompting or deleting. Deploy replay
521+
supports classic and metadata-file modes only (not WebDAV).
521522
"""
522523
overrides = {
523524
"localDir": localdir,

‎examples/reproducible-download.md‎

Lines changed: 18 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -36,4 +36,21 @@ Key fields:
3636
Six months later, a colleague can verify the same data was
3737
downloaded by checking the checksums in the manifest against
3838
the files on disk, or by inspecting the `dbus:replayParams`
39-
to understand exactly what was fetched and when.
39+
to understand exactly what was fetched and when.
40+
41+
## Actually reproducing it: replay
42+
43+
Rather than manually re-typing the original command from the
44+
`dbus:replayParams` you inspected above, replay it directly:
45+
46+
```bash
47+
databusclient manifest replay ./manifests/labels-download.jsonld --localdir ./data-replayed
48+
```
49+
50+
This re-runs the download using the exact same parameters that
51+
were recorded — compression, format conversion, checksum
52+
validation, and so on — without needing to remember or
53+
reconstruct the original command by hand. Credentials
54+
(`--vault-token`/`--databus-key`) are never stored in the
55+
manifest and, if the original download needed them, must be
56+
supplied again here.

0 commit comments

Comments
 (0)