fix: an unreported meter reading is absent, not zero - #167
Merged
Conversation
Energy and power fields were filled with 0.0 whenever the property behind them carried no value. A retained-topic replay delivers `$description` before the values it declares, so on every connect — and again after every reconnect, since the adapter rebuilds from a clean accumulator — every circuit exists, is described, and has reported nothing. Throughout that window the snapshot asserted each was drawing no power and had accumulated no energy since installation. A consumer cannot tell that from a meter that genuinely reads zero, and zero is exactly what a lifetime counter drops to when firmware resets it. So a consumer compensating for counter resets books the whole counter as an offset and does it again on the next replay: SpanPanel/span#259, where that reached 8.18 MWh on one circuit and ~10 MWh of fictional energy across a panel. The rule is a discrimination rather than a new default — unreported is None, a reported `0` stays 0.0 — and it is per-property, so a circuit that has published half its meter reports that half. A synthesised `unmapped_tab_*` entry still reads zero, because an unoccupied breaker position genuinely draws nothing. Two consequences of the same rule. The panel-level fields were the worst case: both lugs devices declare one type and are told apart by the `info/direction` they publish, so until that property arrives neither role resolves and all six — the site's whole import and export — were fabricated together. And `dsm_state` stops inferring islanding from silence; its heuristic read "no power crossing the service entrance" out of "nothing has reported yet" and now answers UNKNOWN, which it already returns when it cannot tell. ADAPTER_CONTRACT_VERSION does not move: it guards the calling convention, and both adapters ship this with the bootstrap in one unpublished release.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes the library half of SpanPanel/span#259.
The defect
Every energy and power field on
SpanCircuitSnapshot, and the six panel-level ones read off the lugs, were filled with0.0whenever the property behind them carried no value.A retained-topic replay hands a subscriber
$descriptionbefore the values it declares. So there is a window — on every connect, and again after every broker reconnect, because the adapter is rebuilt from a clean accumulator — in which every circuit on the panel exists, is described, and has reported nothing. Throughout that window the snapshot asserted that each of them was drawing no power and had accumulated no energy since it was installed.Reproduced against the shipped parser: feed an accumulator only
$descriptionand$state=ready, then build a snapshot.Why it is destructive rather than cosmetic
A consumer cannot tell the fabricated zero from a meter that genuinely reads zero — and zero is exactly what a lifetime counter drops to when firmware resets it. A consumer compensating for counter resets books the entire counter as an offset, and does it again on the next replay.
On real hardware that reached 8.18 MWh of fictional energy on a single circuit, roughly 10 MWh across one panel, written into Home Assistant long-term statistics one restart at a time. The compensated value does not move on the tick the zero arrives, so the spike surfaces one sample later and reads as genuine consumption.
The change
The rule is a discrimination, not a new default: an unreported reading is
None, a reported0is0.0, and the two no longer collapse into each other. It is per-property, so a circuit that has published half its meter reports the half it has.A synthesised
unmapped_tab_*entry still reads0.0— an unoccupied breaker position genuinely draws nothing, which is an assertion the adapter is entitled to make rather than a reading it failed to receive.Both adapters: schema-1's
or 0.0, and a new_reading()in schema-0 kept separate from_parse_floatbecause most flat properties legitimately want a substituted default.Two consequences that fell out
The panel-level fields were the worst case, not an edge case. Both lugs devices declare the same type and are told apart by the
info/directionvalue they publish. Until that one property arrives neither role resolves, and all six fields — the whole site's import and export — were fabricated together.This is also why a readiness gate was considered and rejected. Gating on "every property the parser reads has a value" cannot see this:
_lugs_metadatareturns{}whiledirectionis unset, so the lugs rows are not in the map to be required, and the gate would pass while the panel-level channel stayed open. It would validate itself against the state it was meant to check. It also deadlocks — 23 declared properties in the reference capture have no retained value, includingconnection/feeds-device-statuson ordinary circuits, which is a mapped field.dsm_stateno longer infers islanding from silence. Its fallback heuristic asks whether power is crossing the service entrance, and read "no power" out of "nothing has reported yet" — declaring a site off-grid on the strength of a measurement nobody had made. With neither grid signal reported it now answersUNKNOWN, which the same function already returns when it cannot tell.mypysurfaced this; it was invisible before.Not addressed here: schema-1 never populates
energy_accum_update_time_s/instant_power_update_time_s, so a v1.0-tree consumer has no in-band signal of data age. The flat adapter sets both. That is worth fixing, but it is a separate concern from the absent/zero distinction and does not belong in this change — filing it separately.Breaking
float→float | Noneon the readings, documented in the changelog under Changed as BREAKING FOR CONSUMERS. Consumer-facing rather than adapter-facing, so it does not collide with the three adapter breaks already in 3.1.0.ADAPTER_CONTRACT_VERSIONdoes not move. It guards the bootstrap-to-adapter calling convention, and both adapters ship this with the bootstrap in the same unpublished release, so no adapter carrying the old behaviour is reachable. The published 1.0.0 adapters are already refused at discovery on the existing floor.Coalescing with
or 0is rarely the right downstream repair — it reintroduces the fabrication one layer out. The Home Assistant integration's adaptation propagatesNoneinstead, which reaches its unknown-value path and, on aTOTAL_INCREASINGsensor, skips dip compensation entirely so the absent reading never becomes a baseline.Verification
tests/test_absent_readings_are_not_zero.py, driven from the reference capturemypy --strictclean across 41 source filesbuild-checkassertions run locally: all six artifacts build,twine checkpasses, every wheel shipspy.typedand no reference payloads, and the bootstrap installs and fails by name with no adapter present