Repository navigation
[yamcs-recorder] Export YAMCS events to Loki - #303
Merged
Merged
Conversation
- Add the Go bindings the proto-go-yamcs script generates from events.proto and events_service.proto
- Add SubscribeEvents, which subscribes to an instance's new events - Test subscribing when YAMCS_GRPC_ADDRESS is set, raising one event with CreateEvent - Describe the event subscription test in the README
FarkasJoseph
force-pushed
the
yamcs/events-otel
branch
from
October 8, 2026 21:29
1878b91 to
e70a3ff
Compare
Grafana plugin dev buildA development build of the Grafana datasource plugin is available for this PR (updated on every push).
To try it, set |
- Add LogRecord, which maps an event's times, severity, message and fields onto a log record - Number each YAMCS severity in its OpenTelemetry band, with CRITICAL and SEVERE in FATAL, and leave the severity text unset - Put YAMCS's own fields, including the severity name, in yamcs.<field> attributes, leaving out an empty type or createdBy - Map WARNING_NEW to WARNING and ERROR to SEVERE in yamcs.severity - Add each extra entry as a yamcs.extra.<key> attribute, in key order - Test the conversion against archived fprime-project events
- Add a loki service using config/loki.yml, publishing 3100 and keeping its data in a loki-data volume - Add an otel-collector service using config/otel-collector.yml, publishing OTLP on 4317 (gRPC) and 4318 (HTTP) - Start Loki with Grafana, which provisions a Loki datasource
- Add --otlp, the collector's OTLP gRPC address, and log at startup whether events are exported - Exit at startup when --otlp is not host:port - Subscribe to the instance's events and emit each as a log record from a second goroutine - Batch and export records with service.name set to the instance, and flush them on exit - Cap gRPC's wait between reconnects to the collector at 5 seconds - Return from run when either subscription ends, after stopping the other - Log OpenTelemetry SDK errors through the recorder's logger - Add an events count to the stats line - Describe event export, its log record mapping, the events Loki drops and Loki queries in the package doc and README - Point the datasource plugin README's Events note at Loki
FarkasJoseph
force-pushed
the
yamcs/events-otel
branch
from
October 8, 2026 22:05
e70a3ff to
92728e1
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Grafana can't show YAMCS events yet.
yamcs-recordercopies only telemetry from YAMCS into TimescaleDB, so the Hermes datasource hides its Events query.What
--otlp host:port,yamcs-recorderalso sends each new YAMCS event to Loki, Grafana's log database, through the OpenTelemetry collector at that address.WATCH.yamcs_<field>, and each entry of itsextramap, such as an F Prime argument, becomesyamcs_extra_<key>.Testing
Notes
config/loki.yml, Loki rejects an event more than an hour older than the newest for its instance, more than 7 days old, or more than 10 minutes in the future, so a late downlink can leave gaps. Editing that config can raise those limits. Events raised while the recorder or collector is down are lost too.str()text, so structs aren't JSON, bools readTrueand opcodes are plain numbers. That's fixable upstream.