[Discussion] Refurbished approach to telemetry

[oscgonfer]

I am adding this issue to agree on different aspects that could be done on Telemetry that have been discussed in various areas but probably haven't put together yet formally. Hopefully this list will grow with more ideas.

Tagging @NomDeTom and @caveman99 directly as I'd love to hear your opinions. Feel free to add others.

Decouple reading and publication interval

This idea has been discussed for a while and it's partially linked to the Telemetry Database we have talked about although not only that.

Currently, we read data periodically, but we are semaphored by AirTime / ChUtil after the fact. I'd suggest having two intervals instead: one for reading data at a predefined rate, and one for publishing it over the mesh, which is semaphored if needed. Internally, we could group readings in smart ways to send them over the mesh as a joint packet that contains either:

• a history of readings (for sensor role)
• aggregated metrics over the latest publication (for everyone else)

This would effectively act as a queue for readings that are not sent over, but that are read no matter what. Sensor role could have an additional feature to store this persistently on flash or sd-card if available.

The phone API will behave the same way as it is right now. Below for the MQTT API.

Linked: #9519 and meshtastic/protobufs#854 (although far from complete).

Always publish to MQTT if enabled

Considering the use case for a node that has Wi-Fi/ MQTT enabled, for any role, it could always read data and upload it via mqtt, no matter if the mesh is restricted at that particular moment.

Remove sensor readings from payload

Remove sensor readings that are not important, but that are enabled by default - these could have a toggle for removal by admin message (?)

Linked: #9712

All values in telemetry as scaled ints

What if we made all values in telemetry s scaled ints, (documented in protobuf comments) to save space, similar to location.

Identifiable sensor readings

Nesting telemetry data in protobuf would help us have traceability of who is sending data, now that we have overlapping types of sensors.

Linked: meshtastic/protobufs#852

MQTT bridge and decoupling

I have not tested MQTT scripts lately, but transforming data for subscribers to JSON and decoupling it on topics seems like a reasonable service to provide, but not on the node itself.

Linked: #8665 and #3798

[ndoo]

I don’t know if this belongs here, but I was also thinking of adding a generic Modbus/SDI-12 sensor type, all sorts of industrial sensors just expect you to read 8/16/32/64-bit values out of registers.

To me it sounds useful to have some kind of generic sensor type that gives a Protobuf list of register addresses and their values. That way we can support an infinite number of industrial sensors just by telling the Telemetry module which registers we want polled, and the width of data to retrieve from each address.

E.g.

A config would have a list of Register16 and a list of Register32, the RS-485/SDI-12 address to poll, and then telemetry would return this generic sensor reply

	Register16	Value
0	0x12	0xFFFF
1	0x34	0xDDDD

	Register32	Value
0	0x12	0xFEFEFEFE
1	0x34	0xDEDEDEDE

[NomDeTom]

Just to add a couple of the other things that we discussed but haven't been captured:

We have a lot of portnums available, to help differentiate older traffic and different types of traffic within a psk channel. This also helps internal prioritisation of the packet for e.g. processing, deduplication, etc.

The "compressed" telemetry data packet idea was based on utilising the zigzag/continuity-bit encoding property of sint that small signed numbers are smaller than bigger ones, so if you transmit deltas rather than raw readings you will get a net saving for numbers that don't change very much between readings. You still have to contend with packet length and communication of the content.

Finally, we did discuss having a much grander list of data types, perhaps similar to the other IoT specs, that would allow duplicate readings of a type, and perhaps designating a few standard locations to those types.