Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
83 changes: 83 additions & 0 deletions .github/workflows/launcher-reproducible.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
name: launcher reproducible build

# Reproducibility is a release gate, not a per-commit check: a full run
# builds the TD image twice (privileged mkosi in a pinned container) and
# asserts every artifact is byte-identical. That is minutes-long and
# Linux/Docker-only, so it runs only when a release is published.
#
# Precondition: at release cut, launcher/build/Dockerfile.builder must be
# pinned to a real Debian digest (Pinning protocol in launcher/build/README.md).
# build.sh refuses to run while the PLACEHOLDER digest is present, so an
# unpinned (non-reproducible) release fails this job by design — we do NOT
# set ALLOW_UNPINNED_BUILDER here.
on:
release:
types: [published]
workflow_dispatch: {}

jobs:
reproducible:
# ubuntu-latest can run privileged Docker; if the hosted runner cannot
# complete the mkosi image build, move this to a self-hosted Linux runner.
runs-on: ubuntu-latest
steps:
- name: Check out the repo
uses: actions/checkout@v4

- name: Install Go
uses: actions/setup-go@v6
with:
Comment thread
0xHansLee marked this conversation as resolved.
Comment thread
0xHansLee marked this conversation as resolved.
Comment thread
0xHansLee marked this conversation as resolved.
go-version: '1.24'

- name: Cache OpenSSL static build
id: cache-openssl
uses: actions/cache@v4
with:
path: /usr/local/lib64
key: openssl-3.0.13-static-${{ runner.os }}

- name: Install build dependencies
run: |
sudo apt-get update
sudo apt-get install -y build-essential cmake libssl-dev

- name: Build static OpenSSL (if not cached)
if: steps.cache-openssl.outputs.cache-hit != 'true'
run: |
cd /tmp
wget -q https://www.openssl.org/source/openssl-3.0.13.tar.gz
tar xzf openssl-3.0.13.tar.gz
cd openssl-3.0.13
./config --prefix=/usr/local --openssldir=/usr/local/openssl no-shared
make -j"$(nproc)"
sudo make install_sw

- name: Verify reproducible build (build twice, diff every artifact)
run: ./launcher/build/verify-reproducible.sh

# The custom TDX kernel must be reproducible AND keyless (CONFIG_MODULE_SIG=n),
# so anyone can rebuild it byte-identically with no private signing key and
# independently rederive platform_commitment. Heavy (~full kernel build) — same
# release gate; move to a self-hosted runner if the hosted one is too slow.
kernel:
runs-on: ubuntu-latest
steps:
- name: Check out the repo
uses: actions/checkout@v4

- name: Install kernel build dependencies
run: |
sudo apt-get update
sudo apt-get install -y build-essential bc bison flex libelf-dev \
libssl-dev dpkg-dev xz-utils kmod cpio

- name: Build kernel twice (keyless assertion + byte-identical determinism)
run: |
set -e
KERNEL_WORKDIR=/tmp/kbuild-a ./launcher/kernel/build-kernel.sh /tmp/kernel-a.deb
KERNEL_WORKDIR=/tmp/kbuild-b ./launcher/kernel/build-kernel.sh /tmp/kernel-b.deb
a=$(sha256sum /tmp/kernel-a.deb | awk '{print $1}')
b=$(sha256sum /tmp/kernel-b.deb | awk '{print $1}')
echo "build A: $a"; echo "build B: $b"
[ "$a" = "$b" ] || { echo "kernel build is NOT reproducible ($a != $b)"; exit 1; }
echo "kernel is reproducible + keyless (MODULE_SIG=n asserted in build-kernel.sh)"
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@

# Build
build/
# launcher/build/ is the reproducible-image build pipeline (Dockerfile,
# build.sh, etc.), not a build *output* — un-ignore it explicitly.
!launcher/build/

# cb-mpc C++ library (auto-downloaded)
.cbmpc/
Expand Down
6 changes: 3 additions & 3 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -41,17 +41,17 @@ setup-cbmpc:
# Produces a binary that fail-closes on every TEE operation. Do NOT use for
# devnet, mainnet, or any environment that performs DKG.
build: setup-cbmpc
CGO_LDFLAGS_ALLOW=".*" ./scripts/go_with_cpp.sh $(CBMPC_PATH) $(GO) build -mod=readonly $(BUILD_FLAGS) -ldflags="-buildid= $(BUILDINFO_FLAGS) -extldflags=-Wl,-w" -o $(OUT_DIR)/$(BIN_NAME) ./
CGO_LDFLAGS_ALLOW=".*" ./scripts/go_with_cpp.sh $(CBMPC_PATH) $(GO) build -mod=readonly $(BUILD_FLAGS) -ldflags="-buildid= $(BUILDINFO_FLAGS)" -o $(OUT_DIR)/$(BIN_NAME) ./

# build-sgx — production SGX/Gramine build with cb-mpc C++ library.
# mr_enclave-stable across rebuilds.
build-sgx: setup-cbmpc
CGO_LDFLAGS_ALLOW=".*" ./scripts/go_with_cpp.sh $(CBMPC_PATH) $(GO) build -mod=readonly -tags "sgx $(build_tags)" -ldflags="-buildid= $(BUILDINFO_FLAGS) -extldflags=-Wl,-w" -o $(OUT_DIR)/$(BIN_NAME) ./
CGO_LDFLAGS_ALLOW=".*" ./scripts/go_with_cpp.sh $(CBMPC_PATH) $(GO) build -mod=readonly -tags "sgx $(build_tags)" -ldflags="-buildid= $(BUILDINFO_FLAGS)" -o $(OUT_DIR)/$(BIN_NAME) ./

# build-tdx — production TDX build with cb-mpc C++ library. CGO/cb-mpc
# setup is identical to the SGX target.
build-tdx: setup-cbmpc
CGO_LDFLAGS_ALLOW=".*" ./scripts/go_with_cpp.sh $(CBMPC_PATH) $(GO) build -mod=readonly -tags "tdx $(build_tags)" -ldflags="-buildid= $(BUILDINFO_FLAGS) -extldflags=-Wl,-w" -o $(OUT_DIR)/$(BIN_NAME) ./
CGO_LDFLAGS_ALLOW=".*" ./scripts/go_with_cpp.sh $(CBMPC_PATH) $(GO) build -mod=readonly -tags "tdx $(build_tags)" -ldflags="-buildid= $(BUILDINFO_FLAGS)" -o $(OUT_DIR)/$(BIN_NAME) ./

# Run standard (non-SGX) binary
run:
Expand Down
13 changes: 8 additions & 5 deletions enclave/tdx/providers.go
Original file line number Diff line number Diff line change
Expand Up @@ -45,11 +45,14 @@ type ProviderPolicy struct {

// supportedProviders defines the PolicyOR branches used for sealing.
//
// IMPORTANT: PolicyOR digest computation is order-sensitive. Reordering or
// removing entries invalidates ALL existing sealed blobs that were written
// against the prior order; affected operators must re-seal. New entries
// appended at the end are forward-compatible (existing blobs continue to
// unseal as long as their original entries remain present in the same order).
// IMPORTANT: the sealed object's authPolicy is fixed at seal time — a
// single-provider blob binds a bare PolicyPCR digest, a multi-provider blob a
// PolicyOR-over-PolicyPCR digest. Unseal reconstructs the policy from the
// CURRENT supportedProviders, so ANY change to this slice — including appending
// a new entry — changes the reconstructed digest and makes every previously
// sealed blob fail to unseal (fail-closed, but a data-loss trap). Appending is
// NOT forward-compatible: adding a firmware vintage requires re-sealing, or a
// wire-format bump with multi-policy unseal support.
//
// supportedProviders is the canonical list of PCR-extension states under
// which we will agree to unseal data. ANY change requires a code edit and a
Expand Down
35 changes: 31 additions & 4 deletions enclave/tee.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
"errors"
"fmt"
"os"
"path/filepath"
"sync/atomic"

cmtdb "github.com/cometbft/cometbft-db"
Expand Down Expand Up @@ -155,16 +156,42 @@
}

// SealToFile seals data with the running enclave's sealing key and writes the
// sealed blob to filePath with mode 0600. The write is NOT atomic; concurrent
// readers may observe a partially written file during the write window.
// sealed blob to filePath with mode 0600. The write is atomic: the ciphertext
// is written to a temp file in the same directory and renamed into place, so a
// concurrent reader never observes a partially written file and a crash mid-
// write leaves the previous blob intact.
func SealToFile(data []byte, filePath string) error {
sealed, err := Default().Seal(data)
if err != nil {
return fmt.Errorf("failed to seal data: %w", err)
}

if err := os.WriteFile(filePath, sealed, 0600); err != nil {
return fmt.Errorf("failed to write %s: %w", filePath, err)
dir := filepath.Dir(filePath)
tmp, err := os.CreateTemp(dir, "."+filepath.Base(filePath)+".tmp-*")
if err != nil {
return fmt.Errorf("failed to create temp file in %s: %w", dir, err)
}
tmpName := tmp.Name()
// Best-effort cleanup if we bail before the rename succeeds.
defer func() { _ = os.Remove(tmpName) }()

if err := tmp.Chmod(0600); err != nil {
_ = tmp.Close()
return fmt.Errorf("failed to chmod temp file: %w", err)
}
if _, err := tmp.Write(sealed); err != nil {
_ = tmp.Close()
return fmt.Errorf("failed to write temp file: %w", err)
}
if err := tmp.Sync(); err != nil {
_ = tmp.Close()
return fmt.Errorf("failed to sync temp file: %w", err)
}
if err := tmp.Close(); err != nil {
return fmt.Errorf("failed to close temp file: %w", err)
}
if err := os.Rename(tmpName, filePath); err != nil {
return fmt.Errorf("failed to rename temp file to %s: %w", filePath, err)

Check warning on line 194 in enclave/tee.go

View check run for this annotation

Jinn Agent / Jinn PR Review

enclave/tee.go#L194

After `os.Rename(tmpName, filePath)` succeeds, the deferred `os.Remove(tmpName)` runs against a path that no longer exists (the rename moved it) and silently returns an error. This is harmless in practice, but the log line `log.Infof("%s is sealed and written successfully", filePath)` is printed on the *success* path only, after the rename — so the semantics are correct. Just confirming: the deferred Remove is a benign no-op on success.
}
log.Infof("%s is sealed and written successfully", filePath)

Expand Down
15 changes: 10 additions & 5 deletions enclave/tee_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -211,19 +211,24 @@ func TestSealToFile_SealError(t *testing.T) {
require.False(t, exists, "SealToFile must not create the destination file when Seal fails")
}

// TestSealToFile_WriteError covers the os.WriteFile-failure branch. Asking
// the OS to write into a path under a non-existent directory triggers a real
// write error that we can assert is propagated as a wrapped error.
// TestSealToFile_WriteError covers the write-failure branch. The atomic write
// creates a temp file in the destination's directory first; a non-existent
// directory makes that step fail with ENOENT, which must be propagated as a
// wrapped error and must not leave any file behind.
func TestSealToFile_WriteError(t *testing.T) {
restore := enclave.SwapDefault(fakeBackend{name: "fake"})
defer restore()

// Path under a non-existent subdirectory; os.WriteFile returns ENOENT.
// Path under a non-existent subdirectory; the temp-file create returns ENOENT.
dst := filepath.Join(t.TempDir(), "no-such-subdir", "out.bin")

err := enclave.SealToFile([]byte("payload"), dst)
require.Error(t, err)
require.Contains(t, err.Error(), "failed to write")
require.ErrorIs(t, err, os.ErrNotExist)

exists, statFailed := dirEntryExists(dst)
require.False(t, statFailed, "stat returned a non-ENOENT error")
require.False(t, exists, "SealToFile must not leave the destination file on failure")
}

// TestUnsealFromFile_UnsealError covers the Unseal-failure branch of
Expand Down
86 changes: 86 additions & 0 deletions launcher/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
# story-kernel-launcher

Reproducible, hardened Trust Domain (TD) image build pipeline and runtime
environment for story-kernel validators. Everything required to deliver,
verify, and operate a production-grade TDX node lives under this directory.

## What it provides

- **Reproducible image build** via `mkosi` — same source produces a
byte-identical rootfs (and deterministic `code_commitment`) across
machines and time
- **Keyless reproducible kernel** — the custom TDX kernel is built with
`CONFIG_MODULE_SIG=n` (`kernel/`), so anyone can rebuild it byte-identically
with **no private signing key** and independently verify `platform_commitment`
- **Boot-bound sealing** — key shares are sealed on the GCP confidential-VM
vTPM (`/dev/tpmrm0`) under `PolicyPCR(PCR 7, 11, 12)`, tying them to the exact
Secure-Boot policy, dm-verity rootfs, and story-kernel ELF; they unseal only in
that measured boot (see `docs/operator-guide.md` → Measured-boot chain and sealing)
- **`dm-verity` rootfs** — block-level integrity for the booted image;
any tampering after boot crashes the kernel
- **OS hardening** — no SSH, root locked, all getty masked, ptrace
disabled, `/dev/mem` unavailable, no shell after handoff
- **RTMR3 self-extend** — story-kernel binary identity is baked into
the TDX attestation chain before any DKG work begins
- **External config injection** — the light-client config (which chain to
follow) is fetched at boot from cloud instance metadata into tmpfs, not
baked into the rootfs, so `code_commitment`/`platform_commitment` stay
stable across chains and resets (see `docs/operator-guide.md` → Node config)
- **Platform & code commitment extraction tooling** — operators and
auditors derive both on-chain governance values from a real quote
(`attestation/verify-platform.sh`, `attestation/verify-rtmr3.sh`)

## Threat model

The launcher defends threats (A) host memory read, (B) root-in-TD code
tampering, (B') validator share extraction, (C) fork-chain light-client
spoofing, and (D) Sybil committee joins. The full matrix — one row per
threat, one column per component — is in
[`docs/threat-model.md`](docs/threat-model.md).

This launcher is **not** a confidentiality replacement. The cryptographic
protocol (DKG + threshold decryption) provides confidentiality. The
launcher exists so the protocol's assumptions about validator code
integrity actually hold in deployed environments.

## Layout

| Subdirectory | Purpose |
|---|---|
| `kernel/` | Keyless reproducible custom TDX kernel: pinned `.config` + `build-kernel.sh` |
| `mkosi/` | Reproducible image build configuration |
| `initrd/` | Custom initrd: early-load modules + dm-verity rootfs setup |
| `boot/` | Boot artifacts: kernel cmdline, dm-verity setup |
| `attestation/` | RTMR3 self-extend flow at boot |
| `hardening/` | OS hardening policies (sysctl, modprobe blacklist) |
| `build/` | Reproducible build pipeline (containerized) |
| `docs/` | Operator + auditor + threat-model documentation |

## Related code outside launcher/

- `enclave/tdx/` — TDX backend Go code (attestation, sealing client)
- `enclave/tdx/README.md` — TDX backend overview

## Building

```sh
# Build the reproducible TD image (containerized, deterministic)
./build/build.sh

# Verify a rebuild is byte-identical to a published reference
./build/verify-reproducible.sh
```

After a TD boots the image, extract the on-chain governance values from a
fresh quote with `attestation/verify-platform.sh` (platform_commitment) and
`attestation/verify-rtmr3.sh` (code_commitment). End-to-end deploy steps —
download, verify, launch, register, upgrade, halt-recovery — are in
[`docs/operator-guide.md`](docs/operator-guide.md).

## Why this is its own component

story-kernel is the Go binary that performs DKG work. The launcher is the
*substrate* around it — image, boot, attestation, sealing, hardening — that
makes the binary's security guarantees real. Keeping them in the same repo
but in distinct trees lets the binary and its substrate be versioned and
audited together while staying organizationally separate.
54 changes: 54 additions & 0 deletions launcher/attestation/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
# attestation/

RTMR3 self-extend flow. Before story-kernel runs any DKG work, RTMR3
is extended with the SHA-384 of its ELF, binding its identity to the
TDX attestation chain.

## Status

Implemented. The boot-time extender lives in
`../mkosi/mkosi.skeleton/usr/local/lib/story-kernel/extend-rtmr3.sh`
and is driven by `story-kernel-rtmr3-extend.service`. This directory
holds the *verification* tooling.

## Commitments

The boot chain (firmware, bootloader, kernel, cmdline) extends
RTMR0..RTMR2, which form the **platform_commitment**:

```
platform_commitment = keccak256(MRTD || RTMR0 || RTMR1 || RTMR2)
```

RTMR3 is extended once at startup with the kernel ELF SHA-384, giving the
**code_commitment**:

```
code_commitment = keccak256(RTMR3)
```

This lets governance approve a platform (firmware vintage, boot chain)
independently from a specific story-kernel binary.

## Contents

This directory holds the operator/auditor **verification** tooling. The
boot-time *extenders* run inside the image: `extend-rtmr3.sh` (RTMR3) and
`measure-binary.sh` (PCR 12) are copied verbatim into the rootfs from
`mkosi.skeleton/`. The `measure-binary.sh` here is a standalone reference
that the `test/` harness exercises.

| File | Purpose |
|---|---|
| `verify-rtmr3.sh` | Post-boot check: extracts RTMR3 from a fresh quote → `code_commitment = keccak256(RTMR3)` |
| `verify-platform.sh` | Extracts MRTD/RTMR0..2 → `platform_commitment = keccak256(MRTD‖RTMR0‖RTMR1‖RTMR2)` |
| `verify-pcr12.sh` | Asserts vTPM PCR 12 == `SHA-256(0x00*32 ‖ SHA-256(ELF))` — the sealing bind |
| `measure-binary.sh` | PCR 12 extend helper (reference copy; the in-image one runs at boot) |
| `test/` | Docker harness: runs measure-binary.sh + verify-pcr12.sh against swtpm, asserts tamper detection |
| `README.md` | This file |

## Reference

`contracts/src/protocol/TDXValidationHook.sol` consumes both
commitments via the `approvedPlatform[platform_commitment]` and
`approvedBinary[code_commitment]` mappings.
Loading