# Missing sensor readings or a real drop?

Verified 18 September 2026. A deterministic public-data illustration with two
explicitly constructed scenarios. No model, agent, detector, product performance,
customer outcome, or physical outage is evaluated.

## What was changed

Use the first two hours of the first complete day in the source: timestamps from
2016-01-12 00:00:00 inclusive to 02:00:00 exclusive. These are 12 readings at a
10-minute cadence, on physical CSV lines 44–55 (header is line 1). The last
reading is timestamped 01:50. This illustrative window was not selected to
estimate how often an operational problem occurs.

Keep the timestamp grid and all eight values outside its middle third unchanged.
For the four timestamps 00:40, 00:50, 01:00 and 01:10, construct two alternatives:

- **Missing:** replace each value with a blank CSV field / JSON `null`.
- **Zero:** replace each value with the numeric value `0` and mark it
  `constructed_zero`. These are synthetic values, not measured zeroes.

The original four values are 310, 380, 380 and 370 Wh, totaling 1,440 Wh. The
source file is never altered. `source-window.csv` contains the original selected
observations; `missing.csv` and `zero.csv` contain the labeled transformations.
Source line references in a transformed row identify its original counterpart,
not evidence that its altered value was observed.

## Results and denominators

| Scenario | Available / expected | Available sum | Mean of available values | Full-window mean |
| --- | ---: | ---: | ---: | ---: |
| Original source | 12 / 12 | 1,860 Wh | 1,860 / 12 = 155 Wh | 155 Wh |
| Constructed missing | 8 / 12 | 420 Wh | 420 / 8 = 52.5 Wh | Unknown from this scenario |
| Constructed zero | 12 / 12 | 420 Wh | 420 / 12 = 35 Wh | 35 Wh in the constructed scenario |

Means have units of **Wh per 10-minute reading**, not watts. Coverage is the
number of available values divided by the 12 expected timestamps: 100%,
66.666…%, and 100% respectively. Coverage describes value availability under an
assumed known cadence; it does not establish uptime or measurement validity.

Silently filling the four blanks with zero also produces 35 Wh. The total of
the available readings remains 420 Wh, but the denominator changes from 8 to
12. Relative to the available-only mean, `(35 - 52.5) / 52.5 = -1/3`: a 33⅓%
reduction created by imputation. The source observation coverage remains 8/12.
This comparison is not a measured decline in the original system.

The missing scenario deliberately omits the high readings in this window.
Missingness is not random. Its 52.5 Wh available-only mean therefore does not
recover the known original full-window mean of 155 Wh. Preserving nulls avoids
inventing observations; it does not make an incomplete average representative.

`results.json` uses schema `reific.missing-data-versus-zero.v1`. Every scenario
includes the coverage numerator and denominator, mean numerator and denominator,
sample statuses, source row numbers, and full-window totals/means. Unavailable
full-window quantities are JSON `null`. Rounding of coverage is presentation
only; exact integer counts remain available.

## Source, license, and provenance

Candanedo, L. (2017). *Appliances Energy Prediction* [Dataset]. UCI Machine
Learning Repository. [DOI: 10.24432/C5VC8G](https://doi.org/10.24432/C5VC8G).

- [Primary source description](https://archive.ics.uci.edu/dataset/374/appliances+energy+prediction)
- [Original UCI archive](https://archive.ics.uci.edu/static/public/374/appliances%2Benergy%2Bprediction.zip)
- License: [Creative Commons Attribution 4.0 International](https://creativecommons.org/licenses/by/4.0/)
- Original file: `energydata_complete.csv`, retrieved 2026-09-15.
- SHA-256: `2820bf712ad0275cb18b85a05250926100d8e65ebb9f4d2d016ca91ea152a25d`
- Existing repository provenance: `public/evidence/source/provenance.json`.

UCI describes 10-minute energy readings and identifies the `Appliances` unit as
Wh. Source timestamps have no explicit timezone, and the source does not specify
whether they label interval starts or ends. This study retains the timestamps
as provided. The complete original window has no missing appliance values.

Reific selected rows/columns, normalized numeric and CSV serialization, computed
summaries, constructed the two labeled alternatives, and drew the figures.
The attributed data and their transformed CSVs retain CC BY 4.0. No affiliation
with or endorsement by the source author or UCI is implied.

## Reproduce the data with the Python standard library

From the website repository root:

```sh
python3 scripts/evidence/missing_data.py --check
```

For an independent replay, download this folder's `replay.py`, `source-window.csv`,
`missing.csv`, `zero.csv`, and `results.json`, plus the unmodified source CSV
from the original UCI archive. Save them together in an empty directory, then:

```sh
python3 replay.py --source energydata_complete.csv --output . --check
```

No package installation, network request, account, private repository, or model
call is needed to execute the replay. It verifies the entire original file's
SHA-256 before selecting the window. Without `--check`, it writes the five
deterministic data/script artifacts; with `--check`, it verifies their exact
bytes without overwriting them. `--self-test` runs the independent fixtures
without requiring a source file. Python 3.10 or newer is supported.

The six fixture tests check a separately known sum and denominator, zero as a
valid numeric value, empty/all-missing input, original-window sums, preservation
of all unchanged observations, and blank-versus-zero CSV serialization. The
replay also checks source hashes, expected timestamps, known original values,
and physical source line references. These are artifact-integrity checks, not
detector or agent success rates.

## Reproduce the figures separately

`render.py` reads `results.json`; it does not recompute or change the evidence.
It exports `figure.svg`, `figure.png` (1600 × 900), and `share.png` (1200 × 630).
The same y-axis scale is used in all three scenarios. Missing values produce
gaps; no line bridges them. The shaded middle third marks the same selected
slots in each scenario. Hollow markers distinguish synthetic zeroes.

```sh
python3 -m venv /tmp/reific-missing-data-plot
/tmp/reific-missing-data-plot/bin/pip install -r requirements-chart.txt
/tmp/reific-missing-data-plot/bin/python render.py
```

The figure renderer has a separate Matplotlib dependency; the numeric replay
remains standard-library only. Figures use bundled DejaVu fonts, fixed SVG IDs,
and no timestamp metadata. Exact raster bytes can vary across plotting-library
dependencies/platforms; `--check` above verifies the five data/script artifacts,
not the PNG/SVG rendering. Each figure includes source and construction labels.

## What this does not establish

A missing reading does not say whether energy use continued, stopped, or changed.
A numeric zero in a real system also needs measurement-health and operating
context before it can establish a physical event. These constructed values
cannot diagnose a device, cause, or outage. There is no evaluation of Reific's
production models or monitoring and no claim of advantage over ordinary code.