The staircase from the last post was a picture. This time it became a number — a threshold, a window size, a tolerance — tuned by hand against real ride data until the algorithm could pick stable gears out of the noise on its own. Then the firmware caught up, a "bug" turned out to be a hardware fault, and a magnet that had quietly drifted out of alignment explained two weeks of weirdness.
The viewer grows up
The browser-based data explorer from the last post was good enough to spot the staircase. It wasn't good enough to build an algorithm around — for that, every outlier, every double-trigger, every weird transition point needed to be picked out, labelled, and tested against candidate thresholds, over and over, across multiple rides.
So gearsense-viewer.html grew up. It now has three synced charts (cadence/wheel Hz, gear ratio, cumulative pulses) with pan and zoom that survives re-renders, an undo stack 50 deep, and named saved views — export/import as JSON — so a given ride can have several competing "cleanups" on file at once.
The single most useful addition was ignore mode: shift-drag a marquee that selects points by time AND value — not just a time range. That distinction mattered because outlier points often share a timestamp with a perfectly valid point at a completely different ratio. Ignored points become null placeholders with the line bridging smoothly across the gap — except for the cumulative pulse counters, which are monotonic and never get touched, ignored-ratio-sample or not.
Fit mode does the other half of the job: drag a marquee on the gear-ratio chart and it draws a linear regression over that range, reporting mean, slope, and n. Multiple fit segments can coexist in different colours, and any segment can be reactivated — shift-click reopens it as an editable marquee at the current zoom level, handy for tightening a boundary after zooming in further.
Manually cleaning Ride 1 this way produced the proof of concept: two segments with means 0.864 (n=18) and 1.102 (n=17), both essentially flat. Genuinely stable gears, sitting right there in the data, once the noise around them was set aside.
From "clean by hand" to "clean automatically"
The manual cleanup gave a target. The next question was whether a simple rule could find the same segments without a human dragging marquees. It turns out it can — with the right validity filter and a remarkably small adaptive window.
-
01
Validity filter Strip out anything that can't represent a real gear ratio — sensor noise plateaus, standing-still periods, coasting with no pedalling. What's left is the signal the rest of the algorithm operates on.
-
02
Adaptive stable-run detector Grow a run sample-by-sample as long as each new ratio stays within
BANDof the run's running mean. If the run reachesMIN_RUNsamples before breaking, record it as a stable segment — mean, sample count, start and end time. -
03
Cross-ride gear ladder clustering Cluster segment means across rides with a tolerance of 0.06–0.08. Segments that land close together are the same gear seen multiple times; segments seen only once stay "tentative" until corroborated by another ride.
Tuning BAND and MIN_RUN was pure "feed the monkey" — one variable at a time against real data, not theorised in advance. BAND=0.06 was too tight and missed real gears; BAND=0.15 was too loose and merged adjacent gears into one. BAND=0.10 with MIN_RUN=10 landed on 20 stable segments in Ride 1, two of which matched the hand-picked segments to within 0.012–0.025 — close enough to trust the automated version.
Most of a gear ladder, from one ride
Clustering those 20 segments by mean ratio (tolerance 0.06–0.08) produced four gears with enough support to call confirmed — seen in three to six segments each — plus three more seen only once, held as tentative:
Plus tentative single-segment gears at 0.426, 1.252, and 1.438 — seven ratios total, from a single 13.6-minute commute, mostly unassisted.
That result also forced a useful reframing of what GearSense is actually trying to do.
GearSense doesn't need to know "this is 3rd gear" — it needs to say "stable ratio X detected" versus "ratio changed to Y." A lower ratio means lower speed for a given cadence, regardless of what number is printed on the shifter. Even partial gear discrimination is useful on day one, and the ladder only gets more complete as more rides come in.
Firmware catches up: v0.3.0 and v0.4.0
With the algorithm validated offline, it moved onto the ESP32. v0.3.0 added gear_process_sample() — a streaming version of the same stable-run detector — emitting a # GEAR line with the segment's start/end time, mean ratio, and sample count whenever a run completes. The dashboard grew a "Stable" row: n/MIN_RUN while a run is building, then OK (n) once it lands.
v0.4.0 went further with a live, in-session gear ladder (LADDER_MAX_SIZE=10, LADDER_TOL=0.07). The deliberate choice here: a run's mean is folded into the ladder as soon as it first reaches MIN_RUN — not when it ends — so the big green gear-number box on the dashboard updates live, mid-ride, the first time a gear is seen. The number shown is the 1-based rank by ascending ratio among gears seen so far, which means it can shift as new lower gears get discovered. That's expected: it's a live "gears seen so far" view, not a fixed lookup table.
v0.4.0 also added a 7th CSV field, gear_events — a running count of stable segments detected since boot, independent of whether the # GEAR line itself made it out over the wire. That diagnostic turned out to matter a lot.
The case of the missing # GEAR lines
Ride 2 (v0.3.0, mostly motor-assisted, logged over BLE) produced a clean CSV — but zero # GEAR lines. Replaying the same log offline through the validated algorithm found 13 stable segments using identical thresholds. Detection logic looked correct; something was eating the # GEAR output specifically, possibly related to calling the output function twice in quick succession within one 250ms tick, or how the BLE terminal app handles back-to-back notifications.
v0.4.0's gear_events counter was built to pin this down: if it increments in the CSV but # GEAR lines still don't appear, that confirms a delivery bug rather than a detection bug.
Ride 3 (v0.4.0, 16.33 minutes) ran with that diagnostic live — and gear_events stayed at zero for the entire ride. Offline analysis confirmed this was the correct output, not a delivery bug this time.
max_wheel_hz sat at roughly 66.6 in every 30-second bucket for the whole ride — including the first 30 seconds, before pedalling even started. That ruled out the original hypothesis (motor EMI under load): the wheel sensor was reading garbage continuously, regardless of riding state. A hardware inspection found the likely cause — the KY-003 connector felt loose when plugged in, and the wheel's spoke magnet had physically drifted to an oblique angle relative to the sensor. This is a Mark II wiring/mounting fault, not an algorithm or firmware problem.
One more small fix landed along the way: gear_detector.py's parsing regex only matched the original 6-field CSV format, so it silently rejected every row of v0.4.0's 7-field output until the regex was made to treat gear_events as optional. A reminder that "no valid data" and "no data" look identical from the outside.
Loading…