Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
838a6d9
Initial edits to move from roles to capabilities
ajcraig Jul 14, 2026
1799871
Further improvements to device capabilities and requirements markdown…
ajcraig Jul 21, 2026
6cb68d3
Further refinement removing content referring to the older device roles.
ajcraig Jul 23, 2026
8844848
Update system-design/specification/margo-devices/device-requirements.md
ajcraig Jul 31, 2026
443a8c4
Update system-design/specification/margo-devices/device-requirements.md
ajcraig Jul 31, 2026
6522aec
Update system-design/specification/margo-management-interface/device-…
ajcraig Jul 31, 2026
a251792
Update system-design/specification/observability/collecting-workload-…
ajcraig Jul 31, 2026
856d13e
Update system-design/specification/observability/collecting-workload-…
ajcraig Jul 31, 2026
4a5cfee
Update system-design/specification/margo-management-interface/device-…
ajcraig Jul 31, 2026
0b1cc56
Update system-design/specification/margo-management-interface/device-…
ajcraig Jul 31, 2026
18ad80e
Update system-design/specification/margo-management-interface/device-…
ajcraig Jul 31, 2026
8788172
Address feedback on PR.
ajcraig Jul 31, 2026
0c20e3c
Update to device capabilities regarding required reporting of device …
ajcraig Jul 31, 2026
7abff0f
updated note based on review fb
ajcraig Jul 31, 2026
e1a0a1f
Update system-design/specification/margo-management-interface/device-…
ajcraig Aug 3, 2026
fa9cc8d
Update system-design/specification/margo-devices/device-requirements.md
ajcraig Aug 3, 2026
65f3b7e
Update system-design/specification/margo-devices/device-requirements.md
ajcraig Aug 3, 2026
4170e66
Update system-design/specification/margo-devices/device-requirements.md
ajcraig Aug 3, 2026
544c56e
Update system-design/specification/margo-devices/device-requirements.md
ajcraig Aug 3, 2026
0740786
Update system-design/specification/margo-devices/device-requirements.md
ajcraig Aug 3, 2026
abfc313
Update system-design/specification/margo-devices/device-requirements.md
ajcraig Aug 3, 2026
f3abecf
Update system-design/specification/margo-devices/device-requirements.md
ajcraig Aug 3, 2026
e442dd6
Update system-design/specification/margo-management-interface/device-…
ajcraig Aug 3, 2026
8c179bb
Update system-design/specification/margo-management-interface/device-…
ajcraig Aug 3, 2026
be1bfdc
Update system-design/specification/margo-management-interface/device-…
ajcraig Aug 3, 2026
00ab1b4
Merge remote-tracking branch 'origin/pre-draft' into ajc/roles-to-cap…
ajcraig Aug 3, 2026
36ab428
Address feedback from Matthias' review.
ajcraig Aug 3, 2026
dd179da
Update system-design/specification/margo-devices/device-requirements.md
ajcraig Aug 4, 2026
42fdee2
Update system-design/specification/margo-management-interface/device-…
ajcraig Aug 4, 2026
ae30f4c
Update system-design/specification/margo-management-interface/device-…
ajcraig Aug 4, 2026
c3a7e3d
Update system-design/specification/margo-management-interface/workloa…
ajcraig Aug 4, 2026
17b1a4f
Clear up hard requirement requiring all devices to host collectors.
ajcraig Aug 4, 2026
8e972a9
Update system-design/specification/margo-management-interface/device-…
ajcraig Aug 4, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -344,13 +344,13 @@ spec:
{%- endif %}
{%- endfor %}

### Example: Cluster Enabled Application Deployment Specification
### Example: Helm Deployment Specification

```yaml
{% include 'examples/valid/DesiredState-001.yaml' %}
```

### Example: Standalone Device Application Deployment Specification
### Example: Compose Deployment Specification

```yaml
{% include 'examples/valid/DesiredState-002.yaml' %}
Expand Down
28 changes: 18 additions & 10 deletions system-design/specification/margo-devices/device-requirements.md
Original file line number Diff line number Diff line change
@@ -1,19 +1,27 @@
# Device Requirements

All current device roles MUST meet the following requirements to be considered Margo compliant. These requirements ensure devices can effectively host and manage Margo compliant workloads. A device MUST support onboarding and management from only a single workload fleet manager.
A device MUST meet a baseline set of requirements to be considered Margo-conformant. These requirements ensure devices can effectively manage Margo-conformant workload and/or host those workloads. A device MUST support onboarding and management from only a single workload fleet manager. A device reports what it can provide to the ecosystem through its [device capabilities](../margo-management-interface/device-capabilities.md).

All Margo-conformant devices MUST provide the following components:

All devices MUST provide the following components:

- A workload fleet management client
- An OCI container runtime

> Note: This is not required for a device behind a see-thru gateway.

All workload hosting devices MUST provide the following capabilities:

- At least one supported deployment type
- At least one supported workload runtime
- An OTEL collector

## Standalone Cluster Role Requirements
> Note: See [device capabilities](../margo-management-interface/device-capabilities.md) for the supported deployment types and runtimes.

Devices filling the standalone cluster role MUST provide the following additional components:
## Helm-enabled Device Requirements

Devices supporting the Helm deployment type, MUST provide the following additional components:

- Kubernetes orchestration platform
- Capabilities for deploying Helm charts
- Kubernetes orchestration platform

### Helm deployment approaches

Expand All @@ -28,12 +36,12 @@ Margo does not dictate how devices deploy workloads packaged as Helm charts. A d
If a device vendor chooses a deployment approach that does not interact with the Kubernetes API, the list of Kuberentes APIs available in the cluster MUST be provided out-of-band. This is supported by a variety of tooling such as the Helm GO SDK, `helm template` command, Kustomization, and ArgoCD.


## Standalone Device Role Details
## Compose-enabled Device Requirements

Devices filling the standalone device role MUST provide the following additional components:
Devices supporting the Compose deployment type, MUST provide the following additional components:

- Compose compliant software as the orchestration platform
- Capabilities for deploying Compose applications
- An OCI container runtime, for example Docker or Podman


## Future Requirements
Expand Down

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
Expand Up @@ -485,7 +485,13 @@ components:
enum: [DeviceCapabilitiesManifest]
properties:
type: object
required: [id, vendor, modelNumber, serialNumber, roles]
required: [id, vendor, modelNumber, serialNumber]
# Only identity fields are required. A device that hosts workloads reports cpus, memory,
# storage, peripherals, interfaces, otelCollector (true), supportedRuntimes (>=1), and
# supportedDeploymentTypes (>=1). A device that does not host workloads (e.g. a see-thru
# gateway that only relays the devices behind it) omits those fields.
# The WFM infers it is non-hosting from their absence and infers a
# gateway from the parent/child deviceId hierarchy.
properties:
id:
$ref: '#/components/schemas/DeviceId'
Expand All @@ -495,36 +501,43 @@ components:
type: string
serialNumber:
type: string
roles:
cpus:
type: array
items:
type: object
required: [cores]
properties:
cores:
type: number
Comment thread
matlec marked this conversation as resolved.
architecture:
type: string
enum: [amd64, arm64, arm]
memory:
type: string
storage:
type: string
peripherals:
type: array
items:
$ref: '#/components/schemas/DevicePeripheral'
interfaces:
type: array
items:
$ref: '#/components/schemas/DeviceCommunicationInterface'
otelCollector:
type: boolean
supportedRuntimes:
type: array
minItems: 1
items:
type: string
enum: [Standalone Cluster, Cluster Leader, Standalone Device, Gateway]
resources:
type: object
required: [cpu, memory, storage, peripherals, interfaces]
properties:
cpu:
type: object
required: [cores]
properties:
cores:
type: number
architecture:
type: string
enum: [amd64, arm64, arm]
memory:
type: string
storage:
type: string
peripherals:
type: array
items:
$ref: '#/components/schemas/DevicePeripheral'
interfaces:
type: array
items:
$ref: '#/components/schemas/DeviceCommunicationInterface'
enum: [oci]
supportedDeploymentTypes:
type: array
minItems: 1
items:
type: string
enum: [helm, compose]
DeviceId:
# format: "{id}[/{id}[/{id}...]]"
# Top-level id is required and must include only unreserved characters as specified in RFC3986.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,32 +1,32 @@
# Collecting Workload Observability Data

The device owner MUST deploy, and configure, an OpenTelemetry collector on their device. The device owner MAY choose the deployment model they wish to follow but MUST use one of the following approaches.
For workload hosting devices there MUST be at least one OpenTelemetry collector deployed to collect the observability data required below. The Device owner MAY choose to deploy multiple OpenTelemetry collectors with each collector receiving different parts of the observability data required below as long as all required observability data is collected.

For standalone and clustered devices there MUST be at least one OpenTelemetry collector deployed to collect the observability data required below. The Device owner MAY choose to deploy multiple OpenTelemetry collectors with each collector receiving different parts of the observability data required below as long as all required observability data is collected.
The device owner MAY choose the deployment model they wish to follow but MUST use one of the following approaches.

![Deployment Model - Multi-Node Deployment](../../figures/System-design-observability-deployment1.drawio.svg)

For multi-node capable clusters the device owner MAY chose to use the DaemonSet deployment model to ensure there is an OpenTelemetry collector running on each node.
For multi-node capable devices the device owner MAY chose to use the DaemonSet deployment model to ensure there is an OpenTelemetry collector running on each node.

![Deployment Model - DaemonSet](../../figures/System-design-observability-deployment2.drawio.svg)

For multi-node capable clusters the device owner MUST ensure the communication between workloads, and collector, from one node to a collector on a different node is secure.
For multi-node capable devices the device owner MUST ensure the communication between workloads, and collector, from one node to a collector on a different node is secure.

The device owner MUST NOT require the use of the sidecar deployment model at this time since this requires the pods/containers to have foreknowledge of this deployment model.

> **Action:** Some more research needs to be done here. If there is a way to do this dynamically without requiring the workload supplier to include special attributes on their pods then it may be allowed.

The device owner MUST NOT pre-configure exporters to send observability data from the device because the end user must control what observability data is exported.

The device owner MUST NOT attempt to inject auto-instrumentation (by using the [OpenTelemetry operator](https://github.com/open-telemetry/opentelemetry-operator#opentelemetry-auto-instrumentation-injection) for example) into any compliant workloads running on the device that are not owned by the device owner.
The device owner MUST NOT attempt to inject auto-instrumentation (by using the [OpenTelemetry operator](https://github.com/open-telemetry/opentelemetry-operator#opentelemetry-auto-instrumentation-injection) for example) into any Margo-conformant workloads running on the device that are not owned by the device owner.

Device owners are NOT required to provide backends for consuming observability data on their devices.

> **Note:** See the [workload observability overview](../../overview/workload-observability.md) page for more information about workload observability.

## Container Platform Observability Requirements

In order to allow for monitoring the chosen container platform's state the device owner MUST ensure the following observability data is being collected and made available for export from the OpenTelemetry collector(s) on the standalone device or cluster
In order to allow for monitoring the chosen container platform's state the device owner MUST ensure the following observability data is being collected and made available for export from the OpenTelemetry collector(s) on the workload hosting device

### Kubernetes

Expand Down Expand Up @@ -58,9 +58,9 @@ For devices running Kubernetes the following is a minimum list of observability

> **Note:** Please see the [information below](#workload-observability-default-telemetry) for the default attributes added by the Kubernetes Attributes Processor.

### Standalone Device Container Platforms
### Compose-based Devices

For devices running non-clustered container platforms such as Docker or Podman the following is a minimum list of observability data that MUST be provided. The device owner MAY choose to provide additional observability data if they wish.
For devices with runtimes supporting compose deployment types, such as Docker or Podman, the following is a minimum list of observability data that MUST be provided. The device owner MAY choose to provide additional observability data if they wish.

- Container observability data MUST be collected.
- It is recommended the Device Owner use the [Docker Stats Receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/dockerstatsreceiver/README.md) or [Podman Stats Receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/podmanreceiver/README.md) with the default configuration to collect this information but using either of these receivers is not required.
Expand Down Expand Up @@ -90,13 +90,13 @@ If the device owner chooses not to deploy the Workload Fleet Management Client a

> **Action:** Need to do research to determine if this makes sense, or not, when the client is not running as a containerized workload. We may have to leave it up to what is covered through device observability for this case. If it is possible, and makes sense, we need to define what should be provided.

In addition to the resource utilization data the Workload Fleet Management Client MUST also send the following minimum set of workload observability data to the open telemetry collector on the standalone device or cluster. The device owner MAY choose to provided additional observability data if they wish.
In addition to the resource utilization data the Workload Fleet Management Client MUST also send the following minimum set of workload observability data to the open telemetry collector on the workload hosting device. The device owner MAY choose to provided additional observability data if they wish.

> **Action:** We need to understand what the WOS/a is going to be doing to determine what this is.

### Connecting to the OpenTelemetry Collector

In order for a workload to publish its observability data to the collector on the standalone device or cluster the device own MUST inject the following environment variables into each container.
In order for a workload to publish its observability data to the collector on the workload hosting device the device owner MUST inject the following environment variables into each container.

|Environment Variable|Description|
|---|---|
Expand All @@ -109,7 +109,7 @@ In order for a workload to publish its observability data to the collector on th

## Exporting Observability Data

End users MUST be able to export observability data from a standalone device or cluster to collectors, or backends, onsite or in the cloud if they wish to make the information available to enable remote monitoring and diagnostics.
End users MUST be able to export observability data from a workload hosting device to collectors, or backends, onsite or in the cloud if they wish to make the information available to enable remote monitoring and diagnostics.

> **Decision Needed:** There is a dependency on the decisions about using OpenTelemetry instead of the management API approach. If OpenTelemetry is chosen then there would be some subset of data that MUST be exported to the workload fleet manager supplier.
>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,6 @@

Workload Fleet Management or observability platform suppliers MAY choose to consume workload observability data exported from the end user's devices to provide valuable services to the end user.

The end user MAY choose to export observability data from Margo compliant devices to other OpenTelemetry collectors or backends within their environment that is not on the device.
The end user MAY choose to export observability data from Margo-conformant devices to other OpenTelemetry collectors or backends within their environment that is not on the device.

> **Note**: See the [workload observability overview](../../overview/workload-observability.md) page for more information about workload observability.
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
# Publishing Workload Observability Data

Compliant workloads MAY choose to expose workload specific observability data by sending their observability data to the Open Telemetry collector on the standalone device or cluster. While this is optional, it is highly recommended in order to support distributed diagnostics.
Margo-conformant workloads MAY choose to expose workload specific observability data by sending their observability data to the Open Telemetry collector on the workload hosting device. While this is optional, it is highly recommended in order to support distributed diagnostics.

Workload suppliers choosing to expose metrics, traces or logs for consumption with OpenTelemetry MUST send the data to the OpenTelemetry collector using OTLP.

The information required to communicate with the device's OTEL Collector is injected into each container using [environment variables](../observability/collecting-workload-observability-data.md#connecting-to-the-opentelemetry-collector).

Workload suppliers SHOULD NOT expect their workloads to be auto-instrumented by anything outside of their control (by the [OpenTelemetry operator](https://github.com/open-telemetry/opentelemetry-operator#opentelemetry-auto-instrumentation-injection) for example).

A workload supplier MAY choose an observability framework other than OpenTelemetry but it MUST be self-contained within the deployment of their workload. If an alternative approach is taken, it is NOT recommended workload suppliers publish their observability data outside the device/cluster by using any other means other than the Open Telemetry collector. If the workload supplier chooses to export data without using the OpenTelemetry collector they MUST NOT do this without the end user's approval.
A workload supplier MAY choose an observability framework other than OpenTelemetry but it MUST be self-contained within the deployment of their workload. If an alternative approach is taken, it is NOT recommended workload suppliers publish their observability data outside the device by using any other means other than the Open Telemetry collector. If the workload supplier chooses to export data without using the OpenTelemetry collector they MUST NOT do this without the end user's approval.

> **Note:** See the [workload observability overview](../../overview/workload-observability.md) page for more information about workload observablity.
>
Expand Down
Loading