Skip to content

Boot, updates, and rollback

SpaceOS verifies each release before activation and retains a known-good state for rollback. Workload rollout and system updates use separate mechanisms.

Deploy and watch a release

Inspect the complete release before sending it:

Terminal window
space deploy payload:release --target sat-42 --plan

The plan names the composition digest, target profile, boot files, signing identities, security versions, and expected image digest. Deploy the release and save the returned ID:

Terminal window
space deploy payload:release --target sat-42
space status <deploy-id>

The status checklist advances through published, approved, queued, uplinking, installed, activating, and activated. The node signs the installed and activated acknowledgements.

Boot verification

Boot alternates between authority checks (a signature proves who authorized the content) and content checks (a digest proves the bytes are the ones authorized). On the Raspberry Pi CM5 reference target the chain is:

StageVerification
Board ROM and firmwareNative board secure boot verifies the customer-signed boot image.
Boot release manifestAn Ed25519 signature binds the kernel, runner, applications, and rollback version together.
Root activationdm-verity verifies every filesystem block on read.
space-init handoffThe measured launch plan is checked against what is actually on disk, and boot stops on any mismatch.
Workload admissionTUF authorizes signed workload digests, with no network needed.

Each layer verifies its own object. Successful verification at one layer does not satisfy another. The native board signature proves firmware may boot an object, not that a workload may launch. On the development VM the chain starts at the host hypervisor instead of board firmware, and the root of trust is the development signing identity pinned on first use; the dm-verity and TUF layers are the same.

Update mechanisms

SpaceOS uses different mechanisms for workloads and for the boot-critical system:

MechanismChangesActivation
Workload blue/greenIndependently deployable services and their policyOld and new run side by side; promote after health probes
System releaseKernel, initial RAM filesystem, control plane, verity rootAppend to the content store; trial reboot; confirm after boot health

A workload update does not reboot the node. A system release is appended to the content store and activated through a trial reboot.

The two mechanisms also map onto the two support levels: blue/green workload rollout, signed admission, and the runtime’s own signed self-update apply everywhere, including daemon installs on a vendor Linux. System releases and the hardware rollback floor apply to full-OS targets, where SpaceOS owns the boot path.

Workload blue/green rollout

A workload generation rolls out without a reboot:

  1. Preflight. SpaceOS checks the candidate image digest, signature, permitted resources, security version, and available memory. If the old and new copies cannot run together, the update stops and the old copy keeps running.
  2. Launch green. Candidates start in dependency order while the old (blue) instances remain authoritative.
  3. Probe. Each service must pass its declared healthcheck within its deadline. Health requires the declared probe to succeed.
  4. Promote. Routes switch atomically to the healthy generation, the commit is persisted, and the node emits a signed healthy acknowledgement. Only then do the blue instances stop.

A candidate that fails before promotion is stopped and blue stays authoritative. A multi-service generation reports either a committed result or a rolled-back result.

System release rollout

SpaceOS does not use fixed A/B partitions. System images live in a content store: one partition holding any number of dm-verity filesystem images indexed by their root hash, the way a git object store holds objects. An update appends the new image (sharing unchanged content with the old), and a small, tamper-evident boot record holds three pointers: the confirmed release, the release on trial, and a last known good to fall back to. The boot record selects releases by content identity. Retention policy and free space determine how many rollback targets remain available.

A tamper-evident boot record points to confirmed, trial, and last-known-good releases in the content store. Confirmation advances the confirmed pointer only after health checks pass.

A boot-critical change is staged into the store, verified, read back, and armed as the trial release with a power-loss-safe record before the node reboots. The trial boot must pass native secure boot, the release manifest, dm-verity activation, state recovery, and the configured probes. Only then does the confirmed pointer move; failure or boot-attempt exhaustion falls back automatically to the last healthy release.

Every phase has a durable recovery rule: power loss during receive resumes the transfer, during staging leaves the confirmed release untouched, during confirmation retries idempotently. The confirmed release remains bootable and verified throughout the update.

Anti-rollback floor

The security version number (SVN) is distinct from the release generation: it changes only when policy intends to prohibit older security states, not on every rebuild. After a system release confirms, the node ratchets a hardware-backed floor: on the locked CM5 profile, a counter in the TPM security chip that only ever counts up. Any release whose SVN is below the floor is invalid even with an otherwise valid older signature. Independently updated workloads have their own per-component SVNs in an authenticated rollback map, so each ratchet applies only to the updated workload.

Routine rollback stays possible within that boundary: space rollback creates a new authorized operation that re-delivers the target’s prior activated release, admissible only while its SVN still satisfies the committed floor.

Terminal window
space rollback --target sat-42
space status <rollback-deploy-id>

Delta transfer

The desired release is always identified by its full content digest; how the bytes get there is a delivery optimization. Delivery reuses verified content the node already holds, and a changed large object may travel as a delta: only the chunks that differ, with chunk boundaries chosen by content so that an insertion early in a file does not invalidate everything after it. A delta names its exact base and target digests; a missing or mismatched base falls back to an authorized full object. The node accepts only the named base digest. Ground-side inventory is a transfer hint and carries no authorization authority.

Over a kilobit-class uplink, a routine patch can transfer as a small CFDP delta instead of a full image.

Deployment states

A deploy is a durable record that survives restarts, missed contacts, and redelivery:

The states, in order, are published, approved, queued, uplinking, installed, activating, and activated.

Two acknowledgements from the node, both signed, mark the transitions: a staged acknowledgement (bytes written, verified, read back, not booted) moves the deploy to installed, and a healthy acknowledgement (probes passed, floor ratcheted) moves it to activated. Ground-side delivery evidence records delivery only. The staged acknowledgement records installation, and the healthy acknowledgement records success. space status <deploy-id> renders the pipeline as a checklist, one row per stage with its evidence; after activation the same deployment appears under space ps with runtime state.