When the Model Tells You Where to Look
Table of Contents
You have fitted a Bayesian Linear Regression model on ten measurements of your Hall effect sensor. The posterior gives you a mean prediction at every point in the operating range — and, crucially, a standard deviation. That standard deviation is not decorative. It is actionable intelligence. This article explains what to do with it.
This is Part 1 of a two-part series. It covers the theory behind active learning for iterative sensor calibration — five algorithms, two uncertainty types, a noise floor you cannot break through, and a state machine that ties it all together. Part 2 grounds every concept in a running Rust demo.
Prerequisite: When Your Sensor Knows What It Doesn’t Know: The Math Behind BLR+ARD. The present article picks up exactly where that one ended — with a fitted posterior $(\boldsymbol{\mu}_\text{post}, \Sigma_\text{post})$ in hand.
1. A Smarter Way to Measure #
The Problem With Uniform Sampling #
You are calibrating a Hall effect position sensor. The sensor’s operating range covers magnetic field strengths from −200 mT to +200 mT. You have collected ten initial measurements at roughly evenly spaced B-field values. You fit a BLR+ARD model. The posterior is coherent — the mean prediction looks reasonable, the standard deviation curve is sensible. The model says: “I’m fairly confident here, fairly confident there, and quite uncertain near the edges and between the two sparse clusters in the middle.”
Now you have time for three more measurements. Where do you point the calibration magnet?
The naive engineer says: “I’ll fill in the gaps — measure at 50 mT, 100 mT, 150 mT.”
That’s not wrong, but it’s not optimal either. If the model is already confident at 100 mT and 150 mT (because a nearby training point happened to constrain those regions well), those measurements are wasted. You’d be paying for information you already have.
The Bayesian engineer says: “The model tells me exactly where it’s uncertain. I’ll measure there.”
This is active learning: a data collection strategy guided by the model’s own uncertainty, with the goal of reducing that uncertainty as efficiently as possible. Instead of collecting data on a fixed schedule, you collect data on demand — exactly where it matters most.
The Posterior Variance Is the Roadmap #
In the BLR+ARD article we derived the posterior predictive distribution for a new input $x_*$:
$$p(y_* \mid x_*, \Phi, \mathbf{y}) = \mathcal{N}(\mu_*, \sigma_*^2)$$with:
$$\mu_* = \boldsymbol{\phi}(x_*)^T \boldsymbol{\mu}_\text{post}$$$$\sigma_*^2 = \underbrace{\beta^{-1}}_{\text{aleatoric}} + \underbrace{\boldsymbol{\phi}(x_*)^T\, \Sigma_\text{post}\, \boldsymbol{\phi}(x_*)}_{\text{epistemic}}$$where:
- $\boldsymbol{\phi}(x_*)$ is the feature vector at the candidate measurement location
- $\boldsymbol{\mu}_\text{post}$ is the posterior mean weight vector
- $\Sigma_\text{post}$ is the posterior covariance matrix ($D \times D$)
- $\beta$ is the learned noise precision ($\beta = 1/\sigma_\text{noise}^2$)
The active learning insight is already present in this formula. The variance $\sigma_*^2$ depends on where $x_*$ is. High variance at a candidate point means: “if you measured here, you would learn a lot.” Low variance means: “you already know what would happen here.” Measure where the variance is highest.
That is the complete algorithm. Everything else in this article is engineering: making that strategy efficient, safe, and production-worthy.
Roadmap #
This article develops five algorithms that turn the active learning insight into a deployable calibration system:
| Algorithm | Question answered |
|---|---|
| 1 — Posterior Variance Grid | Where exactly is the model uncertain? |
| 2 — Acquisition Function | Of all uncertain locations, which three should I measure next? |
| 3 — Precision Assessment | Have I reached the required accuracy? |
| 4 — Noise Floor Detection | Is further measurement futile? |
| 5 — Orchestration State Machine | How do all four previous algorithms combine into a loop? |
Two framing concepts — precision tiers and goal feasibility — complete the picture by turning the abstract loop into a practical deployment tool.
2. Two Kinds of Uncertainty — and Why the Distinction Matters #
The Question That Changes Everything #
The posterior gives you a single number: $\sigma_*^2$ at some candidate point $x_*$. Before you act on it, you need to answer a deeper question: is this uncertainty something you can reduce by taking another measurement, or is it a physical wall you will always hit regardless of how much data you collect?
The answer determines whether your active learning campaign will succeed or fail. And the answer is already encoded in the two-term decomposition of $\sigma_*^2$.
2.1 Aleatoric Uncertainty: The Hardware Limit #
The first term, $\beta^{-1} = \sigma_\text{noise}^2$, is the aleatoric uncertainty — from the Latin alea, meaning dice. It is the irreducible randomness in the measurement process itself.
For a Hall sensor, this includes thermal (Johnson–Nyquist) noise in the readout electronics, quantisation noise from the ADC, electromagnetic interference from nearby circuitry, and mechanical vibration in the sensor mount. No matter how sophisticated your model, no matter how many measurements you collect, a future measurement at the same B-field will still produce a reading that differs from the true value by $\sim \sigma_\text{noise}$.
More precisely: the predictive variance of any single future measurement is bounded below by $\sigma_\text{noise}^2$, even with infinite training data. The noise is i.i.d. per measurement — you can average over repeated measurements at a single point to estimate the mean more precisely, but the predictive distribution for the next reading always carries $\sigma_\text{noise}^2$.
This is the noise floor, and it is a hard physical constant for the given sensor hardware.
Limit behaviour: As $N \to \infty$ (infinite training data), the posterior covariance $\Sigma_\text{post} = (\Lambda + \beta \Phi^T \Phi)^{-1}$ is driven to zero by the growing data-information term $\beta \Phi^T \Phi$. The epistemic term vanishes. What remains is:
$$\lim_{N \to \infty} \sigma^2_*(x_*) = \frac{1}{\beta} = \sigma^2_\text{noise}$$No amount of data can cross this floor.
2.2 Epistemic Uncertainty: The Knowledge Gap #
The second term, $\boldsymbol{\phi}(x_*)^T \Sigma_\text{post} \boldsymbol{\phi}(x_*)$, is the epistemic uncertainty — from the Greek episteme, meaning knowledge. It quantifies how little the model knows about what would happen at $x_*$, given the weight uncertainty encoded in $\Sigma_\text{post}$.
The geometric picture is clean: $\Sigma_\text{post}$ defines an ellipsoid in $D$-dimensional weight space — the region of weight vectors the posterior considers plausible. The quadratic form $\boldsymbol{\phi}(x_*)^T \Sigma_\text{post} \boldsymbol{\phi}(x_*)$ is the projection of that ellipsoid onto the direction $\boldsymbol{\phi}(x_*)$. A long ellipsoid axis pointing in the direction of $\boldsymbol{\phi}(x_*)$ means the model doesn’t know what the output at $x_*$ will be. A short axis in that direction means the model is confident.
The key property: this term is not constant over the input space. At a densely sampled region, $\Sigma_\text{post}$ has been squeezed in the directions corresponding to the feature vectors of those measurements. At sparse regions, it has not. The epistemic uncertainty is high where you haven’t measured — and that is precisely the signal the active learning algorithm exploits.
A related and important consequence: ARD couples directly into the acquisition function. Recall from the BLR+ARD article that a large $\alpha_j$ collapses the posterior variance along the $j$-th feature dimension. If $\alpha_{x^3}$ has grown to $10^6$ due to the cubic feature being irrelevant for the fitting of the current curve, then the ellipsoid is nearly flat in the direction of the cubic polynomial feature. For inputs $x_*$ where the cubic feature is dominant, the epistemic term will be small regardless of how many data you have — because the model already knows with certainty that the cubic coefficient is zero. Active learning will not recommend wasting measurements there.
2.3 The Operational Consequence #
The decomposition is not merely academic. It determines the three outcomes of any calibration campaign:
Outcome 1 — Success: The epistemic term shrinks with each well-placed measurement. Eventually, the 95th-percentile $\sigma_*$ over the entire input range drops below the user’s tolerance. Calibration is complete.
Outcome 2 — Noise floor hit: The epistemic term has been reduced so much that $\sigma_*^2 \approx \beta^{-1}$ everywhere. Adding more measurements barely moves the needle. The remaining uncertainty is aleatoric — irreducible. If the user’s tolerance is below the noise floor, no further effort will help.
Outcome 3 — Goal was infeasible from the start: The user requested a precision tighter than $\sigma_\text{noise}$. This is not discovered at the end of the campaign; it can be identified as soon as noise is estimated from the initial data — before a single active learning iteration is run.
| Property | Aleatoric ($\beta^{-1}$) | Epistemic ($\boldsymbol{\phi}_*^T \Sigma_\text{post} \boldsymbol{\phi}_*$) |
|---|---|---|
| Source | Sensor hardware | Insufficient data coverage |
| Reducible by more data? | No | Yes |
| Uniform over input space? | Yes — same everywhere | No — depends on $\boldsymbol{\phi}(x_*)$ |
| Depends on the model? | No — purely physical | Yes — shrinks as $\Sigma_\text{post}$ tightens |
| Lower bound on $\sigma^2_*$? | Yes — always | No — vanishes with data |
The rest of this article builds the machinery to handle all three outcomes cleanly.
3. Algorithm 1 — Computing Posterior Variance Over a Grid #
Where Do We Evaluate? #
In principle, we could evaluate $\sigma_*(x_*)$ at every point in the continuous input interval $[x_\text{min}, x_\text{max}]$. In practice, we discretise the range into a uniform evaluation grid of $G$ points:
$$\mathcal{G} = \left\{ x_\text{min} + g \cdot \frac{x_\text{max} - x_\text{min}}{G - 1} \;\Big|\; g = 0, 1, \ldots, G-1 \right\}$$A grid of $G = 100$ points over the Hall sensor operating range of $[-200, +200]$ mT gives one evaluation every 4 mT — fine enough to resolve the smooth posterior variance curve with negligible discretisation error.
The Batch Variance Calculation #
For each grid point $x_g \in \mathcal{G}$, the posterior standard deviation is:
$$\sigma_*(x_g) = \sqrt{\frac{1}{\beta} + \boldsymbol{\phi}(x_g)^T \Sigma_\text{post} \boldsymbol{\phi}(x_g)}$$Expanded into matrix operations:
- Compute $\boldsymbol{\phi}_g = \boldsymbol{\phi}(x_g) \in \mathbb{R}^D$ — apply the same feature function used during fitting.
- Compute $\mathbf{v}_g = \Sigma_\text{post} \boldsymbol{\phi}_g \in \mathbb{R}^D$ — one matrix-vector product, $O(D^2)$.
- Compute the quadratic $q_g = \boldsymbol{\phi}_g^T \mathbf{v}_g \in \mathbb{R}$ — one dot product, $O(D)$.
- Compute $\sigma_*(x_g) = \sqrt{\beta^{-1} + q_g}$.
Total cost: $O(G \cdot D^2)$. For $G = 100$ and $D = 6$ (the Hall sensor feature dimension from the BLR+ARD article), this is $100 \times 36 = 3{,}600$ floating-point multiplications — well under one millisecond on any modern processor.
The result is a vector of 100 standard deviations: the posterior std profile over the sensor’s operating range. This profile is the input to every subsequent algorithm.
Numerical Safety #
$\Sigma_\text{post}$ is computed as a matrix inverse during BLR fitting. Floating-point arithmetic can introduce tiny numerical errors that make $\Sigma_\text{post}$ slightly non-positive-semidefinite in practice — meaning that $\boldsymbol{\phi}_g^T \Sigma_\text{post} \boldsymbol{\phi}_g$ can occasionally come out as a small negative number (e.g., $-10^{-12}$). Taking the square root of a negative number is undefined.
The fix is a single guard:
$$\sigma_*(x_g) = \sqrt{\beta^{-1} + \max\!\left(0,\; \boldsymbol{\phi}_g^T \Sigma_\text{post} \boldsymbol{\phi}_g\right)}$$This clamps the epistemic term to zero from below. Physically, the interpretation is: if the numerical error makes the epistemic variance appear negative, it is essentially zero, and the predictive variance is just the noise floor $\beta^{-1}$. No information is lost; only a degenerate numerical artefact is suppressed.
4. Algorithm 2 — The Acquisition Function #
The Question #
We have $G = 100$ posterior standard deviations. We want to take $K = 3$ new measurements. Which three locations maximise the information we gain?
4.1 The Greedy Variance-Maximising Strategy #
The principled answer comes from information theory. The differential entropy of a Gaussian $\mathcal{N}(\mu, \sigma^2)$ is:
$$h\!\left(\mathcal{N}(\mu, \sigma^2)\right) = \frac{1}{2} \ln(2 \pi e\, \sigma^2)$$which is monotonically increasing in $\sigma^2$. A measurement at location $x_*$ provides information proportional to the reduction in entropy it causes. Under a Gaussian posterior, the expected information gain from a measurement at $x_*$ is:
$$\text{IG}(x_*) = h\!\left(\mathcal{N}(\mu_*, \sigma^2_*)\right) - \mathbb{E}_{y_*}\!\left[h\!\left(p(y_* \mid x_*)\right)\right] = \frac{1}{2} \ln\!\left(1 + \beta\, \boldsymbol{\phi}(x_*)^T \Sigma_\text{post} \boldsymbol{\phi}(x_*)\right)$$Since $\ln(1 + u)$ is monotonically increasing in $u$ for $u \geq 0$, maximising expected information gain is equivalent to maximising the epistemic uncertainty $\boldsymbol{\phi}(x_*)^T \Sigma_\text{post} \boldsymbol{\phi}(x_*)$ — which is in turn equivalent to maximising $\sigma_*(x_*)$, the total posterior standard deviation (since $\beta^{-1}$ is constant across locations).
Therefore, the greedy criterion is not merely intuitive — it is the exact maximiser of expected information gain for Gaussian posteriors:
$$x^* = \arg\max_{x \in \mathcal{G}} \sigma_*(x)$$🚧 Exploration vs. Exploitation
Variance-maximisation is a pure exploration strategy: it always asks “where am I most uncertain?”, regardless of whether the model prediction at that location is good or bad. This is the correct choice for calibration, where the goal is uniform coverage and precision across the entire operating range.
A different family of acquisition functions — Expected Improvement (EI) — balances exploration with exploitation, asking “where am I most likely to find a value better than my current best?” EI is appropriate for Bayesian optimisation (find the input that maximises output), not for calibration (model the input-output relationship everywhere). For our use case, pure exploration is correct.
4.2 The Exclusion Radius #
There is a practical complication: without any constraint, the top-$K$ highest-variance locations may all cluster at the same spot — the single region of globally highest uncertainty. You’d be sending the calibration engineer to measure at $x = 47.2 \text{ mT}$, $x = 47.3 \text{ mT}$, and $x = 47.4 \text{ mT}$: three measurements that are almost informationally identical.
The solution is an exclusion radius $r$. After selecting the highest-variance point $x^*_1$, we remove all candidate grid points within distance $r$ of $x^*_1$ before selecting $x^*_2$, and repeat for $x^*_3$.
The complete Algorithm 2:
Input: grid G, posterior_std σ(·), existing_samples S, k, exclusion_radius r
─────────────────────────────────────────────────────────────────────────────
1. Score Sort G descending by σ(·)
2. Filter Remove g ∈ G where ∃ s ∈ S ∪ {x*_1,...,x*_{i-1}} : |g − s| ≤ r
3. Fallback If all candidates filtered, use the full sorted list
(never return an empty recommendation)
4. Take top-k from the filtered list
5. Return ranked RecommendedSample list {input_value, expected_std, rank}
The default exclusion radius is 7% of the input range ($r = 0.07 \times (x_\text{max} - x_\text{min})$). For the Hall sensor with range $[-200, 200]$ mT, this is $r = 28$ mT — far enough to ensure the three recommended points are spread across qualitatively different regions of the operating range.
The fallback in step 3 handles a degenerate edge case that is real in production: if the sensor’s operating range is tiny or the calibration is nearly saturated, every grid point may lie within exclusion range of an existing measurement. Rather than returning nothing (which would crash the caller), the acquisition function falls back to reporting the single highest-variance point without exclusion filtering. The operator sees a recommendation; the system does not panic.
4.3 What a Recommendation Means #
The output of Algorithm 2 is a ranked list of up to $K$ structures:
| Field | Meaning |
|---|---|
input_value | The B-field value (mT) to measure at |
expected_std | The posterior standard deviation at that location (V) |
rank | Priority (1 = highest uncertainty, most valuable) |
Operationally: an operator reads “Rank 1: measure at −130 mT (expected std = 0.021 V)” and configures the magnet accordingly. After collecting the measurement, they feed it back into the session. The model refits. The acquisition function runs again on the updated posterior.
This recommendation maps directly onto the recommendations: list<f64> field inside the iteration-result record in the component’s WIT interface — the same values that flow through the WASM Component boundary to the host runtime.
5. Algorithm 3 — Precision Assessment and the 95th-Percentile Criterion #
The Question #
After refitting BLR+ARD on all available measurements, we have an updated posterior std profile $\sigma_*(x)$ over the grid. The operator specified a precision requirement of, say, $\epsilon = 0.010$ V. Are we done?
This sounds simple — compare $\sigma_*$ to $\epsilon$ — but the question conceals a non-trivial design choice: which summary statistic of the 100-point std profile do you compare?
5.1 Three Statistics and Why Two Are Wrong #
The maximum $\max_g \sigma_*(x_g)$: always dominated by the very edges of the input range ($x_\text{min}$ and $x_\text{max}$), where the evaluation grid extends beyond the calibration data and the posterior naturally widens. Using the max would mean you never declare success until the very boundary of the range is tightly constrained — a condition that may require hundreds of measurements and that is irrelevant for a sensor that never operates at its extreme limits. Overly pessimistic.
The mean $\frac{1}{G}\sum_g \sigma_*(x_g)$: blends well-covered interior regions with poorly-covered edges. A model with excellent coverage of 90% of the range but high uncertainty in a 10% tail could declare success via mean — leaving the operator with a calibration that fails in a specific region. Potentially dangerous.
The 95th percentile $p_{95}(\sigma_*) = \sigma_*(x_{[95]})$ where $x_{[95]}$ is the 95th-order statistic of the sorted std values: the clean middle ground. It says: “95% of the operating range has posterior std below this value.” A small tail (the outermost 5% of the range — typically the very edge grid points) is allowed to remain more uncertain without blocking success. This matches engineering intuition: a sensor calibration that covers 95% of the range at the specified precision is genuinely good.
This is why PrecisionStatus::MetGoal is declared when $p_{95}(\sigma_*) \leq \epsilon$, not $\max(\sigma_*) \leq \epsilon$.
5.2 The Four-State Classifier #
Precision assessment produces one of four states:
| Condition | Status | Operational meaning |
|---|---|---|
| $p_{95}(\sigma_*) \leq \epsilon$ | MetGoal | Calibration complete — stop |
| $p_{95}(\sigma_*) \leq 1.1\,\epsilon$ | Near | Within 10% of goal — one more iteration likely sufficient |
| $p_{95}(\sigma_*) > 1.1\,\epsilon$ | Unmet | More measurements needed — continue |
| Noise floor detected (see §6) | NoiseFloorHit | Physical limit — cannot improve further |
Where $\epsilon$ is the user’s target standard deviation, and the 10% buffer for Near provides an informative intermediate signal (“almost there”) without prematurely triggering success.
5.3 The Gap Metric #
In addition to the four-state classification, precision assessment computes a continuous gap:
$$\text{gap} = \frac{p_{95}(\sigma_*) - \epsilon}{\epsilon}$$When the gap is negative (e.g., $-0.15$), the calibration is 15% better than required. When positive (e.g., $+0.40$), it is 40% worse than required. The gap serves as a diagnostic during active learning: a rapidly shrinking gap indicates fast convergence; a slowly shrinking or stagnant gap is an early warning that the noise floor may soon be the bottleneck.
5.4 The Percentile Calculation #
The 95th percentile is computed by the nearest-rank method: sort the $G$ posterior std values in ascending order, then index at position $\text{round}(0.95 \times (G - 1))$. For $G = 100$:
$$\text{index} = \text{round}(0.95 \times 99) = \text{round}(94.05) = 94$$(0-indexed), meaning the 95th-highest value out of 100. No interpolation is needed; the resolution of 100 grid points is more than adequate for Hall sensor calibration.
6. Algorithm 4 — Noise Floor Detection #
The Scenario #
The operator has set a precision goal of $\epsilon = 0.003$ V. The BLR+ARD fit from the initial ten measurements estimated the sensor noise at $\sigma_\text{noise} = 0.008$ V. That target is below the noise floor — physically impossible. But the operator doesn’t know this. They just see: ten iterations of active learning, thirty additional measurements, and the 95th-percentile std stubbornly hovering around 0.009 V, not budging.
How does the system detect that further effort is futile, without requiring the operator to understand the mathematics of aleatoric uncertainty?
6.1 The Convergence Curve #
As measurements accumulate, $\max_g \sigma_*(x_g)$ follows a characteristic decreasing curve. In the early iterations, dense new measurements reduce the worst-case posterior std quickly — by 20–40% per iteration. As the model fills in the coverage gaps, the returns diminish. Eventually, the max std approaches the noise floor $\sqrt{\beta^{-1}} = \sigma_\text{noise}$ from above and plateaus:
$$\max_g \sigma_*(x_g) \;\xrightarrow{N \to \infty}\; \sigma_\text{noise}$$The plateau is the fingerprint of the noise floor. The active learning algorithm detects it by watching the rate of change of the convergence curve, not its absolute value.
6.2 The Relative-Change Heuristic #
Track the history of pairs $(N_i, \sigma_{\max}^{(i)})$ where $N_i$ is the sample count and $\sigma_{\max}^{(i)} = \max_g \sigma_*(x_g)$ after the $i$-th iteration.
Define the relative improvement between consecutive iterations:
$$\Delta_i = \frac{\left|\sigma_{\max}^{(i-1)} - \sigma_{\max}^{(i)}\right|}{\sigma_{\max}^{(i-1)}}$$Declare the noise floor reached when $\Delta_i < \delta$ for $w$ consecutive iterations:
$$\Delta_{i}, \Delta_{i-1}, \ldots, \Delta_{i-w+2} < \delta \quad \text{(all } w-1 \text{ most recent improvements are small)}$$Default values: $\delta = 0.01$ (1% relative improvement threshold), $w = 3$ (three-iteration window).
Why $w = 3$? A single flat step can happen by chance, even in a model that is still converging — perhaps the last recommended measurement happened to land in an already-certain region. Two consecutive flat steps are suspicious. Three consecutive flat steps are the noise floor.
6.3 Confidence Scoring #
The detection algorithm outputs not just a binary verdict but a confidence score $c \in [0, 1]$:
$$c = \frac{n_\text{plateau}}{w - 1}$$where $n_\text{plateau}$ is the number of consecutive iterations in the window showing $\Delta_i < \delta$. This has a natural interpretation:
| Confidence | Meaning |
|---|---|
| $c = 0.0$ | Fewer than $w + 1$ iterations recorded — insufficient history |
| $0 < c < 1.0$ | Some evidence of plateau — monitor closely |
| $c = 1.0$ | All $w$ required consecutive iterations are flat — noise floor confirmed |
The orchestration loop uses the binary likely_at_floor verdict to trigger NoiseFloorHit. The confidence score is available for logging and diagnostics — a host that wants to warn the operator early (“convergence slowing — may be approaching noise floor”) can act on partial confidence before the binary trigger fires.
6.4 What the Host Receives #
When the noise floor is detected, the orchestration state machine emits:
IterationOutcome::NoiseFloorHit(estimated_floor_std)
where estimated_floor_std is the last observed $\sigma_\text{max}^{(i)}$ — a proxy for $\sigma_\text{noise}$. The operator sees a human-readable message along the lines of: “Further measurements will not improve calibration. Estimated sensor noise floor: ±0.008 V. Your precision goal of ±0.003 V is not achievable with this sensor.”
This is only possible because the Bayesian model explicitly represents aleatoric uncertainty as a separate, named quantity. A least-squares residual doesn’t tell you this. A neural network’s training loss doesn’t tell you this. The two-term variance decomposition makes the noise floor not just theoretically implicit but operationally explicit.
7. Precision Tiers — Framing Realistic Goals #
The Problem With Absolute Targets #
A new operator asks: “I need ±0.01 V precision.” Is that a reasonable request?
Without knowing the sensor, you cannot answer. For a high-precision industrial sensor with $\sigma_\text{noise} = 0.001$ V, ±0.01 V is trivial — the Low tier, achievable with ten initial measurements. For a cost-optimised consumer sensor with $\sigma_\text{noise} = 0.02$ V, ±0.01 V is below the noise floor and physically impossible.
The root cause of the confusion: absolute precision targets are sensor-dependent. A calibration framework that asks users to specify tolerances in volts is forcing them to know their sensor’s noise floor before they’ve even started — which is circular.
7.1 Noise-Relative Targets #
The solution: express precision goals as multiples of the sensor’s own noise floor:
$$\epsilon = f \cdot \sigma_\text{noise}$$where $f$ is a tier factor and $\sigma_\text{noise}$ is estimated from the initial BLR+ARD fit before the active learning loop begins.
Once $\sigma_\text{noise}$ is known, the operator doesn’t choose a voltage — they choose an ambition level. “Give me three times the noise floor” is a sensor-agnostic request that means the same thing on every sensor in the product family.
7.2 The Four Tiers #
Four canonical tiers cover the practical range from rapid commissioning to theoretical maximum precision:
| Tier | Factor $f$ | Estimated samples | Use case |
|---|---|---|---|
| Low | 5.0× noise | ~10 | Rapid field commissioning; coarse sanity check |
| Moderate | 3.0× noise | ~25 | Typical production calibration |
| High | 1.5× noise | ~60 | Demanding applications; quality-critical systems |
| Max | 1.0× noise | ~200 | Theoretical sensor limit; rarely achievable in practice |
Sample estimates use the heuristic $\lceil 50 / f^2 \rceil$, calibrated empirically against typical BLR+ARD active learning convergence on Hall sensor data.
Why does Moderate (3×) match the classical “3-sigma” rule? The 3-sigma rule from classical measurement science says a quantity is confidently measured when your measurement uncertainty is at most one third of the specification tolerance. Equivalently, a calibration uncertainty of $3\sigma_\text{noise}$ means the calibration model contributes only $\sim 10\%$ to the total measurement error budget when uncertainties are combined in quadrature (since $(\sigma_\text{noise})^2 \gg \left({\sigma_\text{noise}/\sqrt{10}}\right)^2$). Moderate is calibrated to exactly hit this classical engineering rule of thumb — it is the “right” choice for most production deployments.
Why is Max (1×) rarely achievable? At $\epsilon = \sigma_\text{noise}$, the epistemic term must be reduced to zero everywhere across the operating range, meaning the active learning loop must saturate the entire input space with measurements. The posterior std asymptotically approaches the noise floor from above but never reaches exactly zero epistemic contribution with finite data. The ~200 sample estimate is a practical ceiling; in many real deployments, the Max tier triggers the noise-floor-hit termination before converging.
8. Goal Feasibility — Before the First Active Learning Iteration #
The Pre-Flight Check #
Before running a single active learning iteration, the operator should know whether their precision goal is achievable in principle. This check requires only two inputs: the chosen tier (equivalently, the tolerance $\epsilon$) and the estimated sensor noise $\sigma_\text{noise}$, which is available after the initial BLR+ARD fit on the bootstrap sample.
Feasibility is classified into three categories:
| Condition | Classification | Meaning |
|---|---|---|
| $f > 1.5$ | Achievable | Comfortable margin above noise floor; convergence reliable |
| $0.9 \leq f \leq 1.5$ | Marginal | Close to ceiling; convergence possible but risky |
| $f < 0.9$ | Unachievable | Below noise floor; physically impossible |
With the four standard tiers:
- Low (5×) and Moderate (3×) → always Achievable.
- High (1.5×) → exactly at the Marginal boundary; calibration will converge with enough measurements, but the operator should expect $\sim 60$ samples and understand that Max may not follow.
- Max (1×) → Marginal; convergence is realistic on low-noise sensors but risky on noisier hardware.
The Unachievable classification cannot be triggered by the four standard tiers — it is reserved for custom targets set below $\sigma_\text{noise}$ directly (a precision more stringent than the physical noise floor).
Why This Check Matters #
Without it, an operator pursuing an infeasible goal proceeds through the active learning loop, collecting more and more measurements, watching the precision metric plateau, growing frustrated — and eventually concluding that the software is broken. The pre-flight check converts that frustrating experience into a direct, actionable message:
“Your goal of ±0.003 V (estimated 0.38× noise) is below the sensor noise floor of ±0.008 V. This precision is physically impossible with this sensor. Please choose a less demanding tier.”
This is only possible because BLR+ARD separates aleatoric and epistemic uncertainty explicitly. A classical least-squares fitter gives you a residual: $\|\mathbf{y} - \Phi\hat{\mathbf{w}}\|$. That residual mixes together fitting error, noise, and model inadequacy. It does not isolate the irreducible noise floor. The Bayesian framework does.
8.1 The Noise Estimation Prerequisite #
The feasibility check requires $\sigma_\text{noise}$, which in turn requires an initial BLR+ARD fit. In the four-phase calibration workflow, this is the bootstrap phase (Phase 0): collect $N_\text{init} \geq 10$ measurements at approximately uniform spacing (a one-time passive sampling step), fit BLR+ARD, extract $\sigma_\text{noise} = 1/\sqrt{\hat{\beta}}$.
This bootstrap phase is deliberate. You cannot run active learning before you have a model; you cannot have a model before you have any data; you cannot estimate the noise floor before you have a model. The ten-point initial sample breaks the circularity and gives active learning a sensible starting point.
Part 2 of this series will show the bootstrap phase explicitly — including how the noise estimate tightens as more initial samples are added, and how to detect when the estimate is stable enough to trust.
9. The Orchestration State Machine #
Putting It All Together #
We now have all the pieces: an uncertainty-aware model, an acquisition function, a precision criterion, a noise floor detector, a tier framework, and a feasibility check. The final algorithm — the orchestration state machine — is the glue that connects them into a calibration session.
9.1 The State Machine #
Each call to next_iteration() runs one step of the following logic:
┌──────────────────────────────────────────────────────────────────────┐
│ CalibrationSession::next_iteration() │
│ │
│ Precondition: ≥1 sample available; iteration < max_iterations │
│ │
│ Step 1 ── Fit BLR+ARD on all samples collected so far │
│ │
│ Step 2 ── Evaluate posterior std over G-point grid (Algorithm 1) │
│ │
│ Step 3 ── Assess precision against target ε (Algorithm 3) │
│ ├─ MetGoal ──────────────────────────────────────────────────→ PrecisionMet
│ │
│ Step 4 ── Detect noise floor (Algorithm 4) │
│ ├─ likely_at_floor = true ───────────────────────────────────→ NoiseFloorHit(σ_floor)
│ │
│ Step 5 ── Check iteration budget │
│ ├─ iteration ≥ max_iterations ───────────────────────────────→ MaxIterationsReached
│ │
│ Step 6 ── Recommend next samples (Algorithm 2) │
│ └────────────────────────────────────────────────────────────→ RecommendNext(K samples)
└──────────────────────────────────────────────────────────────────────┘
↑
add_measurement(x, y) ×K
(host collects measurements at recommended locations)
The checks are ordered deliberately. Precision comes first because MetGoal is the success condition and we want to exit immediately if it is reached, without unnecessary noise floor analysis. Noise floor comes second because it is the “graceful failure” condition — better to detect it early than to keep spinning. The iteration budget check is last because it is a safety net, not an expected exit path.
9.2 The Four Outcomes #
next_iteration() returns one of four outcomes:
| Outcome | Interpretation | What the host should do |
|---|---|---|
PrecisionMet | 95th-percentile std ≤ ε — calibration complete | Export the model; stop collecting |
NoiseFloorHit(σ_floor) | Convergence stalled at the noise limit | Inform operator; consider relaxing the goal |
MaxIterationsReached | Budget exhausted without convergence | Log a warning; inspect precision history |
RecommendNext(samples) | Normal iteration — more data needed | Measure at the recommended locations; call add_measurement() K times; call next_iteration() again |
The dominant path during a healthy calibration is RecommendNext for a few iterations, followed by PrecisionMet. The other three outcomes are either the natural termination (success) or graceful failure modes.
9.3 The Synchronous Design #
The state machine is deliberately synchronous. There is no async/await, no thread pool, no background fitting. Each call to next_iteration() blocks until the BLR+ARD fit, the grid evaluation, and the precision check are complete — then returns.
This design matches the physical reality of calibration: measurements take time. After receiving recommendations, the operator (or automation system) must physically configure the calibration fixture, acquire the sensor reading, and record it. That process is inherently sequential and its duration is dominated by hardware, not software. There is no benefit to fitting the model asynchronously in the background while measurements are being taken, because the next fit cannot start until the new measurements arrive.
The caller owns the measurement acquisition step. The component owns the fitting and recommendation steps. They hand off cleanly via the add_measurement() / next_iteration() interface — a synchronous protocol that is equally at home in a command-line tool, a Jupyter notebook, and a WASM Component host.
9.4 Session State #
The state machine maintains four pieces of mutable state across iterations:
samples: the growing list of all collected measurements (raw input and measured output). This is the training set for every BLR+ARD fit.precision_history: onePrecisionRecordper iteration, tracking the 95th-percentile std, mean std, max std, andgoal_metflag. This is the convergence log.noise_floor_history: pairs of (sample count, max std) — the input to Algorithm 4 across iterations.iteration: a monotone counter. Used both to label history records and to enforce the iteration budget.
At any point during the session, the precision history tells the full convergence story: where you started, how fast you converged, whether you stalled. This history is exported as structured JSON and forms the basis of the calibration certificate in the full sensor-calibration-component.
10. Multi-Sensor Calibration (A Forward Pointer) #
In many industrial applications, you are calibrating not one sensor but an array — a set of Hall sensors for a three-axis position system, a bank of pressure sensors for a hydraulic manifold, a grid of temperature sensors across a furnace. The active learning framework extends naturally to this scenario.
Each sensor maintains its own independent CalibrationSession with its own feature function, input range, BLR+ARD model, and precision target. After every round of measurements, the multi-sensor coordinator reports Vec<MultiSensorRecommendation> — one recommendation list per sensor, each containing the K highest-uncertainty locations for that sensor.
The independence assumption is that sensor errors are uncorrelated: a measurement on Sensor A provides no information about Sensor B’s weights. This is typically valid for spatially separated sensors with independent noise sources. When sensors share common-mode errors (e.g., a shared voltage reference), a joint model would be needed — a natural Phase 2 extension of this framework.
Part 2 of this series will demonstrate multi-sensor calibration using the multi_sensor_workflow.rs example.
11. What Part 2 Will Show #
This article has built the complete theoretical vocabulary:
- Two uncertainty components (aleatoric and epistemic) and why only the second is reducible
- Five algorithms for active learning, precision assessment, noise floor detection, and orchestration
- Precision tiers grounded in the 3-sigma rule of measurement science
- Goal feasibility as a pre-flight check enabled by the Bayesian uncertainty decomposition
Part 2 will ground every concept in a running demonstration using the Hall sensor calibration dataset. Starting from ten initial measurements, it will trace the full active learning loop through three iterations — showing the posterior std profile before and after each new measurement batch, the precision history convergence curve, and the noise floor detection triggering when an over-ambitious goal is attempted.
It will also include the comparison that motivated active learning in the first place: side by side, active sampling (guided by Algorithm 2) versus passive uniform sampling (fixed grid). Same number of measurements, same sensor, same model. The difference in final precision is the empirical argument for everything in this article.
Appendix A — The Posterior Update Shrinks the Epistemic Term #
A natural question: what exactly happens to $\sigma_*^2(x_*)$ after you add a measurement at location $x_\text{new}$?
The posterior covariance update (from the BLR+ARD article, Woodbury form) is:
$$\Sigma_\text{post}^{(N+1)} = \left(\Sigma_\text{post}^{(N)\,-1} + \beta\, \boldsymbol{\phi}(x_\text{new})\, \boldsymbol{\phi}(x_\text{new})^T\right)^{-1}$$The new measurement adds the rank-1 update $\beta\, \boldsymbol{\phi}(x_\text{new})\, \boldsymbol{\phi}(x_\text{new})^T$ to the precision matrix $\Sigma^{-1}$. Precision increases. Covariance decreases.
How much does $\sigma_*^2(x_*)$ decrease at an arbitrary query point $x_*$? By the Sherman–Morrison formula:
$$\boldsymbol{\phi}(x_*)^T \Sigma_\text{post}^{(N+1)} \boldsymbol{\phi}(x_*) = \boldsymbol{\phi}(x_*)^T \Sigma_\text{post}^{(N)} \boldsymbol{\phi}(x_*) - \frac{\beta\, \left[\boldsymbol{\phi}(x_*)^T \Sigma_\text{post}^{(N)} \boldsymbol{\phi}(x_\text{new})\right]^2}{1 + \beta\, \boldsymbol{\phi}(x_\text{new})^T \Sigma_\text{post}^{(N)} \boldsymbol{\phi}(x_\text{new})}$$The reduction is:
$$\Delta \sigma_\text{epistemic}^2(x_*) = \frac{\beta\, \left[\boldsymbol{\phi}(x_*)^T \Sigma_\text{post}^{(N)} \boldsymbol{\phi}(x_\text{new})\right]^2}{1 + \beta\, \sigma_\text{epistemic}^2(x_\text{new})}$$Two observations:
The numerator contains $\boldsymbol{\phi}(x_*)^T \Sigma_\text{post} \boldsymbol{\phi}(x_\text{new})$ — the posterior covariance between the query point $x_*$ and the measurement point $x_\text{new}$. A measurement at $x_\text{new}$ reduces uncertainty at $x_*$ in proportion to how correlated they are under the current posterior. Measuring at an uncorrelated location provides zero benefit at $x_*$.
The denominator contains $1 + \beta\, \sigma_\text{epistemic}^2(x_\text{new})$. A measurement at a point with high epistemic uncertainty contributes more to the denominator, which reduces the gain. Counterintuitively, measuring at the highest-uncertainty point is not always the globally optimal choice for reducing uncertainty everywhere — it is only greedy-optimal for the single point $x_\text{new}$ itself.
This is the formal justification for the greedy approximation: it is not globally optimal (the exact optimum requires a combinatorial search over all $K$-subsets), but for the calibration use case it is close enough — and the exclusion radius ensures that the $K$ greedy choices are spread across informationally independent regions.
References #
Academic foundations:
- MacKay, D. J. C. (1992). “Bayesian Interpolation.” Neural Computation 4(3): 415–447. — Original derivation of the Type-II maximum likelihood update rules for BLR+ARD.
- Tipping, M. E. & Bishop, C. M. (2001). “Sparse Bayesian Learning and the Relevance Vector Machine.” JMLR 1: 211–244. — Full treatment of ARD and its sparsifying properties.
- Settles, B. (2009). “Active Learning Literature Survey.” University of Wisconsin–Madison Computer Sciences Technical Report 1648. — Comprehensive overview of acquisition functions, including variance-based strategies and EI.
- Hennig, P. (University of Tübingen). Probabilistic Machine Learning lecture series. — The Gaussian inference framework and BLR derivation used throughout this series.
This codebase:
crates/blr-active— Implementation of all five algorithms described in this article.crates/blr-core— BLR+ARD fitting, noise estimation, feature engineering.dev/blog/blr-and-ard.md— Prerequisite article: derivation of the BLR+ARD posterior.wit/sensor-calibration.wit— WIT interface encodingrecommendations,iteration-result, andard-hyperparameters.