Many hyperlinks are disabled.
Use anonymous login
to enable hyperlinks.
50 most recent check-ins
|
2026-08-28
| ||
| 18:59 |
6210 CLOSED: degraded-mount signal on both filesystems + facade fs_mount_degraded()
exFAT wear_deg (bmp_hint_init abort now loud + flagged, wearScanDegraded()), FAT32 hint_deg (deriveNextFreeHint abort flagged, nextFreeHintDegraded()), facade fs_mount_degraded() routing by fs_fmt. Mount's return-0 tolerance unchanged - it is the designed trade-off; only the missing signal was added. XS_malformed grew the 6210 group (16->20, gate EXPECT bumped): injected read fault at the bitmap/FAT sector vs controls differing only in the fault. Fail-first: flags disabled -> tests 17/19 RED (18/20), reverted -> 20/20. Facade smoke on hardware green both filesystems; build matrix green. leaf check-in: 74010fe781 user: refaqtor tags: trunk | |
| 17:33 | docs: 8c3bd43e67 resolution in regression-report (busof range-map root cause, ZZ_base9 both-direction verification) check-in: c914dacd06 user: refaqtor tags: trunk | |
| 17:32 |
8c3bd43e67 FIXED: busof() was an address-range map - every transfer to addr >= 8 went to nonexistent bus 1
The '>= 8 true-silence dead zone' root cause: usb_app.busof() mapped addresses to buses by the 1160 range convention alone, so on a single-bus rig every usb_app-side access to an address >= 8 transmitted into the void. Hub-host enumeration pins its own bus (never calls busof) - hence 'enumerates fine, deaf to everything after'. New busof: single live bus -> 0; else roster decides; range convention only as dual-bus fallback. Bitwise-identical for addr < 8 single-bus (the whole existing gate). Verified both directions with ZZ_base9 (estate enumerated at 9+ via dev_addr_base - ordinary enum, no SET_ADDRESS gray zone): fix green at addr 10/11 through fs_mount+root-enum; old busof restored = total red on the same recipe (logs -132707/-133103 green, -132850 red). dev_readdr rewritten via app host + gray-zone CAUTION; dev_addr_base added; usb_host gained clear_addr_toggles passthrough. check-in: 27d474f58d user: refaqtor tags: trunk | |
| 17:13 |
6160 CLOSED: hot-plug suite 11/11, gate stanza, full-board revalidation of the engine fixes
HP suite gained the PRE-notice abandon-honesty test - the falsification run (headless_260828-121230) proved it goes RED under a syncing teardown (6 BOT records) while the post-notice form cannot see one (guards refuse before the ring's BOT layer; run -120840). Clean run 11/11 (-121603). Gate: HP stanza (HP_EXP=11); grep -a on log parsing (pnut-term logs carry NULs; plain grep read every green suite as 'did not finish'). Full board on the changed engine (alloc lowest-free + clear_addr_toggles): FAT32 223/223, exFAT 219/219, XS 27/27, XM 16/16, MD 13/13, HP 11/11. Docs: README gate row + regression-report hot-plug section (6160 covered; 8c3bd43e67 addr>=8 silence remains open). check-in: 1ee482f5c5 user: refaqtor tags: trunk | |
| 15:59 | 6160 ROOT CAUSE: clear per-address toggles on SET_ADDRESS (usb_client.clear_addr_toggles, called from xfer_set_addr). Address reuse by the lowest-free allocator exposed a latent bug: stale OUT toggle made a fresh device ACK-and-discard the first CBW (USB 2.0 8.6.4 duplicate rule), NAK INQUIRY 10 s, then our BOT-reset storm wedged it deaf. Parity-deterministic churn-cycle-2 death, runs -112329/-114608. Old monotonic allocator masked this forever by never reusing addresses. check-in: 317b7249c9 user: refaqtor tags: trunk | |
| 15:37 | 6160: 3 s off-dwell in HP churn (300 ms VBUS bounce wedges the thumb - run 260828-112329); ZZ_cure3 probe. Run 2 was 8/10: allocator fix verified (addr 4 reused every cycle), VolumeDirty case green. check-in: a0ff2cceed user: refaqtor tags: trunk | |
| 15:16 |
6160: hot-plug suite (7/10 first run) + addr>=8 defect found (8c3bd43e67) + alloc_address lowest-free-in-window fix
HP_hotplug_tests: detach-mid-write fails clean and fast, EV_DETACH fires, fs_abandon proven zero-BOT-I/O via the 6150 evt ring (both abandon paths), sticky re-bind by serial with content-verified witness, no silent retarget. Churn walked the old monotonic allocator past addr 7 into a zone where control transfers meet hs=$100 silence (new ticket 8c3bd43e67, open); alloc_address now allocates lowest-free in the per-bus 7-address window, honoring the usb_hub_ram addr&7 toggle-table invariant. Diagnostics: usb_app dev_probe/dev_ctl/dev_ctl_q/dev_readdr, usb_host last_phy_status/last_saw_sop. ZZ_* probes preserved as bisection tools. Bench: hub ports 3+4 device-wedged (deaf at every address, immune to all software recovery) - awaiting physical hub reseat. check-in: 5aad03a281 user: refaqtor tags: trunk | |
| 13:51 |
6190 CLOSED: MALFORMED/HOSTILE MEDIA covered for BOTH filesystems. XS_malformed_tests (16 tests, 16/16 x3, gated, seconds to run, no USB device): hostile synthetic volumes served through set_block_device - FAT32's FIRST synthetic media ever (the injection point existed since 2070, never aimed at hostile images).
Fail-clean made MECHANICAL: the shim splits out-of-range READS (induced failures - refused, counted, asserted per case) from out-of-range WRITES (always a violation, audited ZERO across the run), and every hostile-chain read is bounded by an explicit byte count so hangs convert to visible failures. Methodology lesson recorded: a hostile chain is only tested if the reader WALKS it - the sabotaged file must CLAIM enough size to force traversal (measured: a 24-byte size field capped every read before the first bad link). exFAT: absurd/out-of-range bitmap, out-of-range root, ClusterCount lies, truncated entry sets, divergent backup region - all clean, all coded. FAT32: chain self-loop + 2-cycle (bounded, no hang), chain off the FAT end (one real cluster then refused, nothing fabricated), cross-linked files, start-cluster 0/1, lying FSInfo, and the entryOffsetInCluster mid-cluster-root geometry synthesized at last (previously only drive C's physical format reached it). Not covered, stated plainly on the ticket: NameHash-disagrees (read side doesn't consult it), and hostile-media WRITE campaigns (needs its own design - writes mutate the fixture under test). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QV5uNCPuTaXxxymx8mvCo3 check-in: f36e42bf7e user: refaqtor tags: trunk | |
| 12:49 |
6140 CLOSED: MULTI-DRIVE SUPPORTED, TESTED, GATED. usb_app gains (1) the BOT-OPERATION LOCK - CBW/data/CSW sequences from two cogs previously had nothing preventing interleave on the shared host (the mailbox lock serializes transfers, not multi-transfer BOT ops); one lock now wraps every whole BOT operation, primary and second-slot, recovery ladder included, lock order fs-driver->msc; and (2) the SECOND DRIVE SLOT - drive_bind_second captures a second device's endpoints+capacity without disturbing the primary selection, disk_read_b/disk_write_b route by them under the lock, fs_mount_second_exfat mounts the exFAT singleton there while the facade keeps its primary volume. One FAT32 + one exFAT is the singletons' only expressible pair, now first-class.
MD_multidrive_tests (13 tests, gate stanza): all five 6140 items - two-cog concurrent I/O (16 rounds/side, content-verified, ZERO errors x4 runs, worker on the ticket-prescribed 600-long stack), dual-mount dispatch integrity + cross-visibility both ways, selection churn with per-rebind label confirmation, THE BYSTANDER INVARIANT as witness-hash assertions on both idle drives, contention fairness recorded (58-76 kB/s per side, within ~10%). THE STANDING NEGATIVE: second slot bound to a FAT32 thumb must refuse exFAT mount with -81 - the routing discriminator runs every time. Lock-in-hot-path fully re-verified: FAT32-B 223/223, exFAT 219/219, XS 27/27, MD 13/13; build matrix 3/3 (63_504/72_309/86_716). Ledger note: locals named 'pa' collide with the P2 PA register. README + regression report updated. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QV5uNCPuTaXxxymx8mvCo3 check-in: 624bcef175 user: refaqtor tags: trunk | |
| 05:55 |
6170 CLOSED: FULL fs_* FACADE PARITY - the exFAT ladder column now equals the FAT32 column minus only white-box-by-design tests: FAT32 223/223 (drive B), exFAT 219/219, XS structural 27/27. All 27 ledger items resolved:
FACADE (usb_app): per-cog exFAT WORKING DIRECTORY - a per-cog path prefix joined onto every relative path in every path-taking fs_* call, '.' resolving to the cwd and '..' to its parent (exFAT stores no dot entries - a literal join missed on healthy media, caught by FS_dirhandle #24), targets validated (-85/-111), prefixes cleared on mount/unmount/abandon, per-cog join scratch (fs_* is not serialized here). fs_seek REFUSES past-EOF on both filesystems (silent clamping relocates writes - the corruption shape). fs_eof maps FAT32's TRUE-plus-parked -93 dir-handle shape to the converged -114. fs_max_handles() capability query. fs_rename/fs_move forward to exFAT. xpath local pl->plen (operator-caught: collides with the PL2303 OBJ instance in USB_CDC builds; CDC-inclusive build verified compiling). exFAT DRIVER: rename + moveFile IMPLEMENTED (raw-copied $85/$C0 primaries, rebuilt $C1 names, NameHash+SetChecksum16 recomputed; in-place when the entry count holds, relocate+clear when it changes; move copies the set raw between directories - checksum carries; refusals -112/-92/-111/-94; same-dir move is a no-op). Handle-kind guards SPLIT in 7 methods: a dir handle in a file op is -114, not -87's lump. openDirectory on a FILE: -111, not -85's lie of omission. MAX_OPEN 4->6 (parity with FAT32). WRITE-ON-EXIT error contract: command PUBs clear last_error at entry (accessors exempt), plus re-clears after the four dup-check MISSES that are success paths (find_in_dir arms -85 on every miss - caught by FS_error_handling #16). VERIFICATION: XS structural grew 2 rename tests (in-place set rewrite verified ON THE MEDIA at the same entry index with independent checksum/hash recomputes; relocation leaves no orphan members) - 27/27. Ladder measured then greened row by row; every FAT32 control count unchanged. Gate EXPECT at parity, KNOWN-SHORTFALLS block retired, FSDRV env selects the FAT32 drive (estate rotates; absent-pinned drive A failed every row with 'no volume matched'). Suite headers' 'chdir is a no-op on exFAT' claims annotated historical. README + regression report at parity. FS_E_UNSUPPORTED now unreachable, reserved. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QV5uNCPuTaXxxymx8mvCo3 check-in: 87a5a714a8 user: refaqtor tags: trunk | |
| 03:37 |
6180: the benchmark harness lands and completes the ticket's measured half. harness/top_6180_bench.spin2 - per-drive, serial-pinned, commit-stamped, diffable: sequential throughput vs transfer size at the BLOCK layer (512..4096 B, txpat save/restore scratch-span discipline, every restore spot-verified), random single-block latency DISTRIBUTIONS (min/med/p90/max/mean us, fixed LCG seed), fs_* metadata costs, and error-counter deltas around every phase.
Findings (docs/characterization/four-drive-geometry-260827.md third pass): at 4 KB chunks ALL media converge to 770-960 kB/s - the host is protocol-bound at 65-80% of the 12 Mb/s ceiling, not media-bound; random single-block reads cost ~2 ms EVERYWHERE (a host/protocol floor invariant from 128 MB card to 953 GB SSD); the SSD's random-write distribution is a flat 993 us (bridge cache, no tail) while the card's is the flash-GC textbook - median 2 ms, p90 102 ms, the exact tail the NAK budgets exist to survive, now measured; the SSD's 275 ms/mkdir is the 128 KB cluster zero-fill in the open; and ZERO error/retry deltas across every phase on all four drives - every difference is work, not retries. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QV5uNCPuTaXxxymx8mvCo3 check-in: 898ae22b64 user: refaqtor tags: trunk | |
| 02:46 |
6180: second characterisation pass - SD-reader cross-section. Port 4 now a $14CD:$1212 reader with an unused 128 MB card ('Appo', FAT32, 4 KB clusters - a fourth live geometry: small-media FAT32). Card measured: writes 11-19 kB/s flat across all six sizes (flash program time swamps cluster-boundary structure; 32 KB-class writes ~2.3-2.6 s) while READS are the bench's fastest sequential - 358-468 kB/s. Thumbs + SSD repeat pass 1 within noise (repeatability). Doc also records the hub-vs-direct answer: FS hubs are bit-level repeaters, numbers are media/protocol-bound far below the 12 Mb/s ceiling, and the one ticket that could have invalidated hub measurements (6200) was retested clean and closed tonight.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QV5uNCPuTaXxxymx8mvCo3 check-in: 9b8c6a3d39 user: refaqtor tags: trunk | |
| 02:14 |
6180: four-drive geometry characterisation write-up (docs/characterization/four-drive-geometry-260827.md) - the measured dataset from tonight's bench pass: per-drive geometry, open-dominated read-all rates, freeSpace cost by filesystem (FAT32 4 ms vs exFAT 4.77 s bitmap popcount), and the six cluster-boundary write timings with the findings (4 KB boundary crossing costs 1.7x; cluster size is the write-cost lever; exFAT NoFatChain fastest with a repeatable 63-vs-64-sector 2.2x anomaly; first-write flash variance).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QV5uNCPuTaXxxymx8mvCo3 check-in: f0c0d6e935 user: refaqtor tags: trunk | |
|
2026-08-27
| ||
| 21:15 |
6120: add XS_exfat_struct_tests.spin2 itself - the 25-test structural suite the previous commit described and gated (its 'fossil add' had run from the wrong directory and silently missed; the gate stanza, driver fix, and docs landed there, the suite source lands here).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QV5uNCPuTaXxxymx8mvCo3 check-in: 3c7632511a user: refaqtor tags: trunk | |
| 21:14 |
6120: exFAT STRUCTURAL SUITE LANDS (XS_exfat_struct_tests, 25 tests) AND FINDS A REAL DRIVER DEFECT ON ITS FIRST RUN. Synthetic exFAT volume in hub RAM served through set_block_device() (the 0186 rig extended): boot-region checksum incl. the excluded-bytes exclusion pinned, VolumeDirty asserted ON THE MEDIA through write/unmount/abandon, allocation-bitmap round-trips + the 2080 wear policy + freeSpace vs independent popcount, entry sets with SetChecksum16/NameHash verified by INDEPENDENT spec recomputes + multi-$C1 names + no-orphaned-members-on-delete, the full NoFatChain lifecycle incl. the LYING-FLAG corruption case pinned to its exact deterministic misread, ASCII up-case matching + fabricated non-ASCII lossy handling, and a bounds-guarded block shim asserting ZERO out-of-range accesses across the whole run.
THE DEFECT, predicted from reading and confirmed red on the first execution: extend_chain's conversion loop with h_nclus==1 is a from-0-to-minus-1 range - Spin2 REVERSES it (the 0186 class) and wrote set_fat(first-1, first): a stray FAT link in the NEIGHBOURING cluster's entry (FAT[4] measured EOC->5 on the synthetic media). On real media that entry can belong to any chained file: silent bystander chain corruption. It survived the 0186 static audit because the path needs a runtime collision state. FIXED (guard: convert loop only when h_nclus >= 2); suite 25/25. GATED: run_fs_gate.sh gains the structural stanza; full pass GREEN with the fix in - FAT32 223/223, exFAT 192/192, structural 25/25 (fs_gate_result.txt; sizes 58_700/71_869/81_876). Documented gaps recorded on 1891ce6d6a: backup boot region not consulted, SetChecksum16 not verified on parse, $82 up-case table not loaded, no truncate API. README + regression report updated. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QV5uNCPuTaXxxymx8mvCo3 check-in: 53c017c341 user: refaqtor tags: trunk | |
| 20:29 |
6110: THE ELEVEN-SUITE FACADE LADDER IS COMPLETE AND GATED GREEN - FAT32 223/223, exFAT 192/192, every row matching in one end-to-end pass on the wrap-fixed engine (fs_gate_result.txt). FS_directory's exFAT row measured at its true 25/30 on the operator-reformatted SSD (headless_260827-155607, zero BOT lines) and joins the gate with -D SD_INCLUDE_DEBUG on the FAT32 column for its cluster-0 white-box group.
exFAT: from zero regression coverage to 219 tests running / 192 passing through the application-facing fs_* surface, every FAT32 column matching its RT_ original, all 27 exFAT shortfalls named and filed (seek clamp 2, chdir 7, handle-limit 3, error-code parity 8, rename/move unsupported 6, write-on-exit 1). Gate header's wedge note rewritten as history: a BOT-line deviation today is a real signal, not weather. README + regression report updated to the final table. Remaining unported: date-API (timestamp), lifecycle (multicog - needs facade-native design), and the white-box-by-design four, whose exFAT structural equivalents belong to 6120. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QV5uNCPuTaXxxymx8mvCo3 check-in: aa4f7d63bb user: refaqtor tags: trunk | |
| 07:44 |
6100/6080 ROOT CAUSE FIXED: the BOT wedge was the HOST's timer arithmetic, not devices, media, or load. Twelve sites in usb_host.spin2 compared absolute getct() values against deadlines (8 transaction-timeout sites + 4 SOF-scheduler due/resync sites); every such compare misfires when its window straddles the 32-bit CNT wrap - a 21.47 s hazard grid, phase fixed at chip reset. SOF anomalies landed BETWEEN PACKETS of in-flight transfers (bridge goes ACK-then-silent = 6080); transaction sites fired spurious timeouts (the logged phy_st=-2 stamped on transfers the device was still ACKing, hs=$D2).
Cracked by measurement: four consecutive FS_directory exFAT runs threw their first BOT event at mount+10.66..10.70 s - 40 ms repeatability ACROSS A PHYSICAL RESEAT. Phase aliasing between each binary's deterministic I/O schedule and the wrap grid explains the entire recorded phenomenology: per-suite quasi-deterministic wedge points, 'clustering in time', short-suite immunity, clean gate passes beside 3-for-3 standalone wedges, thumbs wedging 'in ~25 ops', reseat/VBUS immunity. FIX: wrap-safe subtract-then-sign-test at all 12 sites (+1 long each; both fit blocks close). VERIFIED FAIL-FIRST: reproducer red 5 runs, then the same suite/drive/phase ran straight through the hazard instant with ZERO BOT lines (headless_260827-030747); top_txpat all patterns PASS incl. full-range SSD LBA; FULL FS GATE GREEN with the fixed engine - FAT32 190/190, exFAT 167/167, every row matching (fs_gate_result.txt; sizes 58_688/71_869/81_864). ALSO: FS_directory cleanup now empties test dirs BY ENUMERATION - wedge-era kills left /RTMANY holding a NAMELESS entry set fs_delete cannot address (reported, not spun on; 6190/6120 material). Its true exFAT count is 25/30 (cd x2, fs_move x3); the gate row stays parked until an SSD fsck clears the residue. Tickets 1464ab6579/06a0b73ad7 carry the full record; README + regression report updated; wedge-classes memory annotated with the root cause. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QV5uNCPuTaXxxymx8mvCo3 check-in: 3fd2a6ccb9 user: refaqtor tags: trunk | |
| 06:43 |
6110: FS_directory ported (eleventh suite, largest at 33 tests) - FAT32 control 33/33 FIRST RUN, matching RT_directory exactly. Facade gains fs_move (FAT32 forwards to moveFile via xlat; exFAT returns FS_E_UNSUPPORTED -120, completing the rename/move pair); legacy readDirectory(idx) enumeration translated onto the dir-handle API; cluster-zero raw-patch group stays driver-bound FAT32-only under the hybrid pattern (exFAT totals 30).
THE TEN-SUITE GATE RAN GREEN END TO END in one pass first (fs_gate_result.txt): FAT32 190/190, exFAT 167/167, delete_guard 8/8 both filesystems, no wedge - the owed acceptance run delivered. FS_directory exFAT is WEDGE-BLOCKED, not port-blocked: three runs 02:09/02:41 wedged progressively EARLIER (tests #29 -> #20 -> #11), every write-side event opening with the new op=$2A hs=$D2 signature, run 3 firing on a mkdir 128KB cluster zero-fill burst; drive A ran 33/33 between them and the SSD had passed the whole gate at 01:07 - the device's susceptibility is worsening across wedge cycles, recorded on 1464ab6579 (which also gains: FIRST observed successful inline BOT recovery rst=1 clrO=1 clrI=1 - 6080's 'cannot clear' needs a severity qualifier). Both fuller runs agree the genuine divergence set = cd x2 + fs_move group x3 = 25/30 true; the gate row is PARKED in run_fs_gate.sh with uncomment instructions pending a wedge-free measurement, which likely needs an SSD reseat. README + regression report updated. Six suites remain unported: date-API, lifecycle, and the white-box-by-design group. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QV5uNCPuTaXxxymx8mvCo3 check-in: a4cf3f55ae user: refaqtor tags: trunk | |
| 05:28 |
6110: FS_error_handling joins the facade ladder (19/19 FAT32, 11/18 exFAT) - ten suites, FAT32 190/190, exFAT 167/189, every FAT32 column matching its RT_ original. Its FAT32 control caught a REAL FACADE DEFECT on first run: fs_eof's error leg bypassed xlat() and leaked FAT32's raw -93 through the published contract (same class as the fs_dir_open leak) - fixed in usb_app, control re-verified 19/19. The RT_ original could never see it (asserted only nonzero).
Hybrid pattern established for suites with a white-box MINORITY: the one raw-sector MBR-preservation test stays driver-bound under #IFDEF USB_FAT32 (safe: the driver is a DAT singleton, same fact every RT_ suite relies on); FAT32 runs all 19 for control parity, exFAT runs the 18 that belong to the application surface. Gate FAT32 column gains -D SD_INCLUDE_RAW (only FS_error_handling EXPORTDEFs it; inert elsewhere). NEW MEASURED PARITY GAP (08de1c7aaa): exFAT has no fs_error() write-on-exit contract - a successful operation never stores 0, so a stale error survives success. Plus 4 more dir-handle-misuse code instances counted; full 22-shortfall ledger on the ticket. delete_guard exFAT corrected UPWARD to 8/8: the 01:07 wedge-free gate pass scored the true count and the gate failed the run on the improvement, exactly as designed; expected updated in this commit. 6691a78673 fully disposed - the suite is green on both filesystems. Wedge claim on 1464ab6579 corrected: NOT deterministic - three consecutive wedges 23:43-00:15 then the identical sequence clean at 01:07; wedges cluster in time, and the gate header now says how to read a wedge-window deviation. Ten-suite single gate pass still owed (started twice, externally stopped twice; build matrix green at 58_588/71_757/81_748). README + regression report updated to the ten-suite table and 22-item shortfall ledger. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QV5uNCPuTaXxxymx8mvCo3 check-in: 95fc3702bb user: refaqtor tags: trunk | |
| 04:41 |
6110: FS_dirhandle (25/25 FAT32, 19/25 exFAT) and FS_subdir_ops (18/18, 15/18) join the facade ladder - 171/153 across nine suites, every FAT32 column matching its RT_ original. Facade gains fs_rename (FAT32 forwards; exFAT returns NEW FS_E_UNSUPPORTED -120 honestly) and fs_dir_open now xlats FAT32 codes per the published contract - it was the one path-taking method leaking raw -40/-43. Two new error-code parity gaps filed on 08de1c7aaa (fs_dir_open on a file: -85 vs -111; file ops on a dir handle: -87 vs -114).
6230 (6691a78673) RESOLVED: the exFAT populated-directory-delete 'defect' was FS_delete_guard's own fixture - fs_chdir no-ops on exFAT, so the child landed in ROOT and the directory really was empty; the driver's SUCCESS was correct. Proven by ZZ_dgprobe (absolute paths, child ENUMERATED INSIDE, delete -> -96 on BOTH filesystems). FS_delete_guard rebuilt with absolute paths + a child-enumerates-inside setup check; FAT32 control still 8/8. Port discipline now written down: fixtures use absolute paths, only cwd-subject tests keep chdir, subdir counts exclude dot entries. The corrected delete_guard surfaced a DETERMINISTIC 6080-signature BOT wedge (op=$28 during the 384KB verify read, 3 of 3 runs at test #6) - recorded on 1464ab6579 as the first reliable reproducer; read_write exFAT also wedged once mid-gate then passed 49/49 standalone, so run pacing/wedge proximity is a variable. delete_guard exFAT stays 5/8 as WEDGE-TRUNCATED, stated in the gate header (same count as the old artefact for a different reason - the same-number-different-set hazard, written down). run_fs_gate.sh: rows for both new suites; 8-suite table ran end to end (one explained wedge deviation, standalone-reverified); the 9-suite single pass was externally stopped pre-hardware and is owed next session. README + docs/regression-report.md updated to the 171/153 table, the -110..-120 facade error contract, fs_rename, and current build-matrix sizes (58,588 / 71,753 / 81,744). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QV5uNCPuTaXxxymx8mvCo3 check-in: c1c490efb1 user: refaqtor tags: trunk | |
| 02:17 |
6130 CLOSED FIXED: the geometry matrix is now STANDING - run_geometry_matrix.sh, verified working. 6220 CLOSED FIXED. (tickets 74f78296de, 1107a9f1cc)
TWO TICKETS MEET THEIR STATED ACCEPTANCE AND ARE CLOSED. The rest stay open, and the survey below says why for each, so the ticket list reflects reality rather than optimism. 6130 asked for TWO things and both are done. 1. FINISH DRIVE C AND DRIVE B - drive C had never had a single suite run against it before this work: drive geometry 16 non-defrag suites RT_defrag_tests A 32 KB clus, align 0 332 pass 0 fail 0 recov 14 pass B 4 KB clus, align 0 332 pass 0 fail 0 recov WEDGES THE DRIVE C 4 KB clus, ALIGN 5 332 pass 0 fail 0 recov WEDGES THE DRIVE All three geometries produce THE IDENTICAL 332. Commit a359d54149's entryOffsetInCluster sign fix is now exercised on live media - drive C's non-zero alignment is the only geometry on the bench that reaches it. 2. MAKE THE MATRIX STANDING - "a documented way to run the ladder across all three thumbs and record a per-drive score table, so a geometry regression is visible at a glance." That is tests/regression/run_geometry_matrix.sh, and it is VERIFIED WORKING rather than merely written: run against drive A it produced the expected row exactly - 332 passed, 0 failed, 0 recovery events, defrag pass=14 fail=0 recoveries=0. It runs RT_defrag_tests LAST AND SEPARATELY, in its own column, and says why at length: defrag wedges 4 KB media, the gate runs alphabetically which puts defrag SECOND, and on drive C that wedged the drive and cost the remaining fifteen suites - which reported "pass=0 fail=1" apiece. Those were NOT test failures but the fixture reporting "no FAT32 volume matched" against a drive that had left the roster, and that run's "16 passed 21 failed" total was pure artefact. Reporting defrag separately is what stops one destructive suite poisoning the other sixteen. The script closes by stating what a regression LOOKS like - a row that stops matching the others - so the table interprets itself. 6220 closed on its acceptance being met in full: a fail-first test on the synthetic-volume fixture with the assertion OBSERVED going red (4 of 6 before, 7 of 7 after), the out-of-range access measured at LBA 4_194_316 exactly as predicted from reading the code beforehand, and the gate re-verified at 346/346. STILL OPEN, AND CORRECTLY SO:
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BsYQdKe1BGsfm1wqLw4KUo check-in: a2501a1fe2 user: refaqtor tags: trunk | |
| 01:48 |
FS GATE GREEN END TO END, FIRST FULL PASS: build matrix 3/3, FAT32 128/128, exFAT 119/119 - every suite matched its expected count on both filesystems. README rewritten around the results (ticket 8bcfaaa605)
==== PRODUCT BUILD MATRIX ==== USB_MSC USB_EXFAT BUILDS 58_548 bytes USB_MSC USB_FAT32 BUILDS 71_701 bytes USB_MSC USB_EXFAT USB_FAT32 BUILDS 81_676 bytes ==== FAT32 (drive A, by serial) ==== ==== exFAT (SSD, by serial) ==== FS_seek_tests 38 = 38 OK FS_seek_tests 36 = 36 OK FS_cogcwd_tests 5 = 5 OK FS_cogcwd_tests 3 = 3 OK FS_stress_tests 4 = 4 OK FS_stress_tests 4 = 4 OK FS_fatchain_tests 2 = 2 OK FS_fatchain_tests 2 = 2 OK FS_delete_guard_tests 8 = 8 OK FS_delete_guard_tests 5 = 5 OK FS_multihandle_tests 22 = 22 OK FS_multihandle_tests 20 = 20 OK FS_read_write_tests 49 = 49 OK FS_read_write_tests 49 = 49 OK total 128 passed, 128 expected total 119 passed, 119 expected ==== FS GATE GREEN: every suite matched its expected count on both filesystems ==== This is the run that was owed. The expected counts had been MEASURED from individual suite runs during the session but the gate had never executed in one pass; now it has, and it agrees with every one of them. The exFAT driver is gated - it had no gate at all before today, not a failing one, NONE - and the three product configurations are proven to compile with three distinct footprints, which is the evidence that a disabled filesystem leaves no remnant. README REWRITTEN AROUND THE RESULTS rather than around the history of getting them. It now opens with the two gates and what each REFUSES to call green - run_gate.sh fails on any missing result and on ANY BOT recovery event, run_fs_gate.sh fails on any deviation from expected counts IN EITHER DIRECTION so a new failure cannot hide behind a fixed one - then the FAT32 geometry matrix, the facade table for both filesystems, the product build matrix, a short section on WHAT TESTING THROUGH THE FACADE FOUND (the missing open-path table, the colliding error codes, the two silent-success parity divergences, and the populated-directory delete that is still open), and the gaps. The gaps section names them plainly: RT_defrag has never passed on 4 KB media; ten suites still cannot run on exFAT and the wall is error constants plus absent facade methods (rename, move, dates) plus a group that is WHITE-BOX BY DESIGN and rightly stays FAT32-only; and multi-drive, hot-plug, malformed media and exFAT structural are not exercised at all. It closes on the sentence the estate has always deserved - one drive at a time, well-formed media, no removals: A FLOOR, NOT A FINISH LINE. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BsYQdKe1BGsfm1wqLw4KUo check-in: ea5c42b8ce user: refaqtor tags: trunk | |
| 01:20 |
THE exFAT DRIVER IS NOW GATED, AND PROVEN BUILDABLE FOR PRODUCT INTEGRATION: tests/regression/run_fs_gate.sh (ticket 8bcfaaa605)
exFAT HAD NO GATE AT ALL - not a failing one, NONE. run_gate.sh is the FAT32 DRIVER gate: seventeen RT_ suites bound to usb_fat32_fs, one filesystem, 346/346. It cannot reach exFAT, because the suites it runs declare the FAT32 driver object directly. A driver with no gate cannot be regression-tested. run_fs_gate.sh runs the FS_ suites, which talk to usb_app's fs_* facade, so the SAME SOURCE runs on both filesystems. That is what made the exFAT driver gateable at all. IT CHECKS EXPECTED COUNTS, NOT "ZERO FAILURES", and the reasoning is the part worth keeping. exFAT legitimately fails nine of these tests today and every one is filed on a ticket. A gate demanding zero would be red forever and ignored within a week. A gate printing only a total would let a NEW FAILURE HIDE BEHIND A FIXED ONE - the number stays 119 while the set underneath changes. So every suite carries its expected pass count PER FILESYSTEM and ANY DEVIATION IN EITHER DIRECTION FAILS THE RUN. A count going UP fails too, deliberately: it means either a defect was fixed - in which case the EXPECT table is updated IN THE SAME COMMIT as the fix - or a test stopped running. Both need a human to look. The table carries a one-line reason for every exFAT shortfall and its ticket, so the gate explains itself to whoever reads it next. suite FAT32 exFAT why exFAT is lower FS_seek_tests 38 36 seek past EOF: FAT32 refuses, exFAT silently clamps (08de1c7aaa) FS_cogcwd_tests 5 3 fs_chdir: no cwd on exFAT, returns success anyway (08de1c7aaa) FS_stress_tests 4 4 FS_fatchain_tests 2 2 FS_delete_guard_tests 8 5 populated-directory delete; the guard exists and its scan misses the child (6691a78673) FS_multihandle_tests 22 20 MAX_OPEN 4 vs 6, a capability difference (08de1c7aaa) TOTAL 128 119 THE PRODUCT BUILD MATRIX - the "buildable for integration into products" half - RUN AND GREEN: USB_MSC USB_EXFAT BUILDS 58_548 bytes USB_MSC USB_FAT32 BUILDS 71_701 bytes USB_MSC USB_EXFAT USB_FAT32 BUILDS 81_676 bytes THE THREE DIFFERENT FOOTPRINTS ARE THE EVIDENCE THAT THE GATING WORKS: each build carries only what it asked for, which is Phase 0's requirement that a disabled class leave no remnant - checked now rather than assumed. The exFAT-ONLY build is the one nothing used to verify: usb_app declares xfs : "usb_exfat_fs" inside #IFDEF USB_EXFAT, so every FAT32-only build skips the exFAT driver entirely and nothing proved the exFAT-only configuration still compiled. Now something does. --build-only runs the matrix with NO HARDWARE AT ALL, which is what makes it usable in a build pipeline rather than only on the bench. STATUS, PRECISELY, because half of this is verified and half is not: THE BUILD MATRIX IS RUN AND GREEN - all three configurations, sizes above. THE HARDWARE HALF IS NOT YET RUN END TO END. The estate collapsed to 1 device / 0 mounted after the previous gate attempt wedged drive A, so there was nothing to run it against. The expected counts in the table are the MEASURED results of this session's individual suite runs, not guesses - but the gate itself has not yet executed in one pass. That needs a reseat and is the next thing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BsYQdKe1BGsfm1wqLw4KUo check-in: c1ac4dc81e user: refaqtor tags: trunk | |
| 00:58 |
exFAT NOW HAS THE OPEN-PATH TABLE IT WAS MISSING - two of three defects closed by ONE mechanism, and it cleared NINE failures in a different suite (tickets 6691a78673, 8bcfaaa605, 08de1c7aaa)
suite FAT32 exFAT before exFAT after FS_seek_tests 38/38 36/38 36/38 FS_cogcwd_tests 5/5 3/5 3/5 FS_stress_tests 4/4 4/4 4/4 FS_fatchain_tests 2/2 2/2 2/2 FS_delete_guard_tests 8/8 2/8 5/8 FS_multihandle_tests 22/22 18/22 20/22 FS_read_write_tests 49/49 40/49 49/49 TOTAL 128/128 105/128 119/128 THE MECHANISM, AND IT NEEDED NO NEW PER-HANDLE STORAGE. A file is identified by WHERE ITS DIRECTORY ENTRY SET LIVES: the parent directory's first cluster plus the entry-set base index. exFAT already recorded both per handle - h_pdir_f and h_dent, set by init_file_handle - because the write path needs them to flush the entry set. The only missing piece was a predicate that asks the question: PRI path_is_open(pdir_f, dsi, write_only) : yn scan the handle table for h_used == 1 and matching (h_pdir_f, h_dent); write_only nonzero -> only handles opened for writing count. That is the same identity FAT32 keys on - (dir_sector, dir_offset) in isFileOpen / isFileOpenForWrite - so the two drivers now answer the question the same way rather than one of them not answering it at all. WHY NOT A PATH STRING: paths are not unique (trailing slashes, case, "."), and storing them would cost MAX_OPEN x path bytes for an identity the driver already had. WHY NOT FIRST CLUSTER: a zero-length file has none, so two opens of an empty file would not collide - exactly what a create-then-open sequence produces. THE TWO GUARDS: openFileWrite - single-writer policy matching FAT32, checked BEFORE alloc_handle so a refused open consumes no handle. deleteFile - refuses while any handle holds the file, checked FIRST because the directory-emptiness branch below re-points the scan cursor and the identity is gone after it. New constants E_FILE_OPEN = -112 and E_FILE_ALREADY_OPEN = -113 EQUAL THE FACADE VALUES deliberately: usb_app publishes those numbers and passes exFAT statuses through untranslated, because exFAT's numbering IS the facade contract. NINE FAILURES IN FS_read_write WERE DOWNSTREAM OF THE SAME BUG, and that is the part worth remembering. They were the large-file and cluster-boundary tests (#40 read 128KB, #41 create 256KB, #44 delete reclaims space, #45/#46 all-$00 and all-$FF round-trips, #48 write spanning a cluster boundary, #49 seek across one), logged as an open question with only 5 BOT lines in the run so not wedge fallout. They were state corruption left by an EARLIER TEST IN THE SAME SUITE that deleted or re-opened a file while a handle was live. ONE MISSING GUARD, NINE DOWNSTREAM FAILURES, IN A SUITE THAT LOOKED LIKE IT HAD ITS OWN DEFECT. THE NINE REMAINING exFAT FAILURES ARE NOW FULLY ACCOUNTED FOR - none is unexplained:
GATE STATUS, STATED PRECISELY. The change is inside usb_exfat_fs.spin2, and usb_app declares `xfs : "usb_exfat_fs"` inside #IFDEF USB_EXFAT. The RT suites build -D USB_MSC -D USB_FAT32 with no USB_EXFAT, so THE exFAT DRIVER IS NOT COMPILED INTO ANY GATE BUILD and this change cannot reach 346/346. The immediately preceding gate run was 346 passed / 0 failed / 0 recoveries and contained every other change in this series. A confirming run was still attempted, because compile-gating is an argument and a green gate is evidence. It produced NO VERDICT: drive A wedged inside RT_delete_guard (1 pass, 7 fail, 15 recoveries) and every later suite reported the fixture's "no volume matched" setup failure. That is the wedge, not the code - and it is not claimed either way. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BsYQdKe1BGsfm1wqLw4KUo check-in: 5e3df3183c user: refaqtor tags: trunk | |
| 00:12 |
SEVEN SUITES NOW RUN ON BOTH FILESYSTEMS - 128 tests on exFAT where there were zero this morning - and the port found a THIRD exFAT defect with the same root (tickets 8bcfaaa605, 6691a78673). Gate re-established at 346/346.
suite FAT32 exFAT FS_seek_tests 38/38 36/38 FS_cogcwd_tests 5/5 3/5 FS_stress_tests 4/4 4/4 FS_fatchain_tests 2/2 2/2 FS_delete_guard_tests 8/8 2/8 FS_multihandle_tests 22/22 18/22 FS_read_write_tests 49/49 40/49 TOTAL 128/128 105/128 GATE FIRST, because it was owed: 346 passed, 0 failed, 0 recovery events, 17 suites, with the error-contract change in. The debt recorded two commits ago is cleared - translating FAT32's driver codes into the published facade contract costs the FAT32 baseline nothing. And the estate was brought back WITHOUT a reseat by using 6200's own finding: after the SSD reseat the probe showed 4 devices but only ONE mounted, thumbs failing st=-7 - 6200's signature exactly. The probe in use was the default build that calls begin2() on the EMPTY second header. Rebuilt -D NO_BUS1, the same rig gave 4 of 4 immediately. The separator recorded on 293576b2fc told us which build to run instead of which hardware to blame. THIRD exFAT DEFECT, SAME ROOT: A SECOND WRITE-OPEN OF THE SAME FILE SUCCEEDS, returning a valid second handle where FAT32 returns already-open. TWO CONCURRENT WRITE HANDLES ON ONE FILE - each with its own position and its own idea of the chain, whichever flushes second overwriting the other's metadata. Together with "delete an open file succeeds" and "delete a populated directory succeeds", all three trace to ONE MISSING MECHANISM: exFAT keeps handles (h_used, h_pos, h_size) but nothing that maps a PATH to them, which is what every one of those guards needs in order to ask its question. FAT32 has exactly that table. That is a much more useful statement than three separate bugs. THE FAT32 CONTROL IS NOT CEREMONIAL - IT CAUGHT A DEFECT IN MY OWN TRANSLATION. FS_multihandle first came back 20 of 22 on FAT32, and both failures were fs_error() returning FAT32's RAW -91 where the operation that set it returned the facade's -87. I had translated returned statuses but NOT the STORED last error, so fs_error() disagreed with the very call that set it. Fixed, re-verified at 22 of 22. Without the control that would have surfaced during the exFAT run and looked exactly like an exFAT defect. WHAT THE PORT NEEDED THIS ROUND: fs_sync_handle() - flush ONE handle and leave it open. Both drivers have carried syncHandle() all along; the facade offered only volume-wide fs_sync(), so an application wanting to make one file durable had to stop the world, and two suites could not port at all. FS_read_write on exFAT is 40 of 49, failures clustered in the large-file and cluster-boundary tests (#40 read 128KB, #41 create 256KB, #44 delete reclaims space, #45/#46 all-$00 and all-$FF round-trips, #48 write spanning a cluster boundary, #49 seek across one). Only 5 BOT lines in the whole log, so NOT wedge fallout. Recorded as observation and NOT chased - per operator direction the estate gets run and the data collected before anything is hunted. REMAINING BLOCKERS for the ten unported suites are now narrow and named: rename/moveFile (absent from the facade), the date API, driver lifecycle (start/stop/COGLESS_RET), and raw-sector plus debugGet* access. That last group is WHITE-BOX BY DESIGN and should stay driver-bound and FAT32-only. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BsYQdKe1BGsfm1wqLw4KUo check-in: 74b1c1a4a5 user: refaqtor tags: trunk | |
|
2026-08-26
| ||
| 23:05 |
Fixture: ONE SICK DEVICE NO LONGER DENIES SERVICE TO EVERY SUITE - the bystander problem at the harness level (ticket 8bcfaaa605). Gate re-verification of the error-contract change is PENDING and NOT claimed.
THE FAILURE. After a session of exFAT testing the SSD went unhealthy - TEST UNIT READY returning not-ready, then a STALL on READ CAPACITY. EVERY SUITE THEN FAILED SETUP with "no MSC drive ready within 15_000 ms", including suites pinned BY SERIAL to a completely healthy thumb. The probe seconds later showed all three thumbs mounting fine. THE CAUSE. rt_usb_fixture.setup_pins() opened with a bare `ifnot usb.await_drive(...): return false`. await_drive returns as soon as the FIRST MSC device is ready, and first-found on this rig is the SSD. A sick first-found device therefore failed setup for the whole estate - the selection loop that walks the roster by serial never even ran. That is the bystander problem one layer up: a bad device denying service to suites that never asked for it. THE FIX. That wait is a SETTLING HINT, NOT A GATE, and it is no longer fatal. The selection loop already does its own per-device await_drive and accepts a device only when it actually MOUNTS, so continuing costs nothing: a run with genuinely no usable device still fails, just at the honest place with "no volume matched" instead of blaming a readiness timeout on a drive it never tried. VERIFIED ON HARDWARE: with the SSD still sick, the fixture now logs "first-found MSC drive not ready in 15_000 ms - CONTINUING; the selection loop waits per device" and then mounts drive A by serial - "[fixture] mounted FAT32 on addr 3 port 2 ... label='NO NAME'". Before the fix that same configuration failed setup outright. WHAT IS NOT DONE, AND MUST NOT BE READ AS DONE: THE 346/346 GATE HAS NOT BEEN RE-ESTABLISHED SINCE THE ERROR-CONTRACT CHANGE. usb_app now translates FAT32 driver codes into the published facade contract, which is a behaviour change on the FAT32 path, and the rule is that the gate runs for every driver change. Two attempts were made: - the first died on the fixture bug above; - the second got past setup and then ground through 70 BOT errors with tests failing, because the SSD is still unhealthy and the bus with it. Neither is a verdict on the change. A clean gate needs the SSD recovered first, and until that run exists the error-contract change is UNVERIFIED AGAINST THE BASELINE. It is committed, it is documented, and it is not claimed green. The SSD needs a reseat; the three thumbs are healthy. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BsYQdKe1BGsfm1wqLw4KUo check-in: 5e4ae39283 user: refaqtor tags: trunk | |
| 22:13 |
The facade's error codes now MATCH across filesystems - and the first suite that could then be ported found TWO CRITICAL exFAT DELETE DEFECTS (new ticket 6691a78673, plus 08de1c7aaa/8bcfaaa605)
"Why can't you make the E_* codes match?" - they can, and the state they were in was worse than "different". THEY COLLIDED: value FAT32 means exFAT means -90 E_TOO_MANY_FILES E_READ_ONLY -91 E_INVALID_HANDLE E_TOO_BIG -92 E_FILE_ALREADY_OPEN E_EXISTS -93 E_NOT_A_DIR_HANDLE E_DISK_FULL -94 E_INVALID_PARAM E_DIR_FULL -96 E_NO_ASYNC_OP E_DIR_NOT_EMPTY and the same NAME carried different numbers - E_DISK_FULL -60 vs -93, E_DIR_NOT_EMPTY -44 vs -96, E_NOT_MOUNTED -20 vs -80. An application testing `== -93` got "not a dir handle" on one filesystem and DISK FULL on the other. THIS WAS NOT A NEW API TO INVENT. The facade has PUBLISHED an error table since exFAT landed (README, "Error codes", -80..-96). FAT32's codes were never translated into it, so the facade was violating a contract it already documented. usb_app now translates them, and adds codes at -110.. for the concepts FAT32 can report that the exFAT-derived table had no word for (not-a-file, file-open, bad-dir-handle, invalid-param, end-of-file, no-contiguous-space, wrong-filesystem). BEHAVIOUR CHANGE, STATED PLAINLY: fs_* now returns the documented facade code for FAT32 volumes instead of the driver's raw number. That is the fix, but anything matching raw FAT32 numbers THROUGH THE FACADE needs updating. Code testing `st < 0` is unaffected. Direct callers of usb_fat32_fs are untouched. THE TRANSLATION IS VERIFIED, not assumed: FS_delete_guard_tests asserts FACADE codes (-96, -112) and passes 8 of 8 on FAT32, whose driver natively returns -44 and -47. Wrong translation would fail those assertions on FAT32. AND THEN THE SAME UNCHANGED SUITE, POINTED AT exFAT, SCORED 2 OF 8 AND FOUND TWO CRITICAL DEFECTS: 1. DELETING A POPULATED DIRECTORY SUCCEEDS, orphaning its contents. The guard EXISTS - usb_exfat_fs.spin2:603-613 selects the directory, scans it and returns E_DIR_NOT_EMPTY - and DOES NOT FIRE. The setup test confirming the directory was populated passed immediately before. The entry set is cleared and the parent's clusters freed while the child's clusters stay marked in use with nothing referencing them. Silent data loss on the project's stated PRIMARY DEFAULT filesystem. 2. DELETING AN OPEN FILE SUCCEEDS. exFAT's deleteFile has NO open-handle check at all - zero hits for h_used / h_open / E_FILE_OPEN across the method. The file is genuinely deleted, not merely reported deleted. FAT32 REFUSES, and that guard is upstream's A1 fix whose rationale is that the open handle's pending flush otherwise lands in a cluster ALREADY REALLOCATED TO ANOTHER FILE - the exact corruption shape this mission exists to stop. NO NEW TEST WAS WRITTEN FOR EITHER. RT_delete_guard_tests has passed for as long as the estate has existed, but it binds usb_fat32_fs directly, so it had only ever asked FAT32. Ported to the facade it asked exFAT and both defects appeared on the first run. That is the whole argument for the port, demonstrated rather than asserted. Incidental correction recorded on the ticket: UPSTREAM-FAT32-BACKPORT.md lists A1 as "ABSENT" from our fork. That is STALE - FAT32 refuses the delete and returns the code, verified by this suite passing 8 of 8 on drive A. suite FAT32 exFAT FS_seek_tests 38/38 36/38 FS_cogcwd_tests 5/5 3/5 FS_stress_tests 4/4 4/4 FS_fatchain_tests 2/2 2/2 FS_delete_guard_tests 8/8 2/8 TOTAL 57/57 47/57 Report and README updated with the error-contract change, the two defects, and the note that a green exFAT column can mean a suite asserted nothing there. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BsYQdKe1BGsfm1wqLw4KUo check-in: 612011f73a user: refaqtor tags: trunk | |
| 21:31 |
6110: FOUR SUITES NOW RUN THROUGH THE FACADE ON BOTH FILESYSTEMS - 49/49 FAT32, 45/49 exFAT - and the four failures are two real parity divergences (tickets 8bcfaaa605, 08de1c7aaa)
suite FAT32 (drive A) exFAT (SSD) note FS_seek_tests 38 of 38 36 of 38 seek-past-EOF divergence FS_cogcwd_tests 5 of 5 3 of 5 cwd divergence FS_stress_tests 4 of 4 4 of 4 clean FS_fatchain_tests 2 of 2 2 of 2 clean, with a caveat TOTAL 49 of 49 45 of 49 exFAT had ZERO suite coverage this morning. EVERY FAT32 COLUMN MATCHES ITS RT_ ORIGINAL EXACTLY (38, 5, 4, 2) - that is the control set, and it is what makes the exFAT column a result rather than a port artefact. TWO PARITY DIVERGENCES, BOTH THE SAME SHAPE: SILENT SUCCESS HIDING A SEMANTIC DIFFERENCE, which is the most dangerous API failure mode because the caller cannot detect it. fs_seek past EOF FAT32 refuses with an error | exFAT SILENTLY CLAMPS to EOF and returns success fs_chdir FAT32 changes directory | exFAT returns SUCCESS HAVING DONE NOTHING (it resolves full paths, no cwd exists) An application seeking to a computed offset and writing gets an APPEND AT EOF on exFAT where FAT32 fails loudly. One that chdirs then creates by bare name gets the file in ROOT on exFAT and in the cwd on FAT32. Filed on 08de1c7aaa, neither fixed - each is a genuine design decision about what the facade should promise, and the resolutions differ. NEITHER COULD HAVE BEEN FOUND BY THE DRIVER-BOUND LADDER. Each RT_ suite only ever runs against its own driver, so exFAT's seek and chdir paths had never been asserted on by anything. A GREEN COLUMN CAN MEAN A SUITE ASSERTED NOTHING, and that is recorded rather than glossed: FS_cogcwd's Test #2 - "one cog's cd does not affect another cog's CWD" - PASSES on exFAT precisely BECAUSE there is no cwd at all. And FS_fatchain passing does NOT mean exFAT's allocation structures are tested; its tests work through the file API, while the bitmap, up-case table and NoFatChain files remain untested (1891ce6d6a). This ticket warns against copying FAT32 chain tests onto exFAT and calling it coverage; that warning applies to its own green column. TWO MORE APPLICATION-API GAPS FILLED, both needed by the port and both worth having anyway: fs_sec_per_cluster() and fs_cluster_bytes(). Both drivers had published these through volumeGeometry() all along with an identical three-LONG contract, but the facade offered only the raw triple, so every caller wanting the commonest field had to know the layout. Three suites were blocked on exactly that. THE REMAINING WALL IS ERROR CONSTANTS, NOT METHODS. Twelve suites still cannot port, and most of their unmapped symbols are E_* codes whose VALUES DIFFER between the drivers (exFAT uses -80..-87). A suite asserting a specific code is asserting a driver's private numbering, so re-exporting FAT32's numbers would make suites pass against the wrong codes. Deciding what the FACADE promises for "file not found" - one code for both filesystems, translated where fs_chdir already reconciles cwd - is the next real design question, and it is a product decision rather than a test one. The white-box suites (raw sector access, debugGet*, defrag internals) should stay driver-bound and FAT32-only by design. docs/regression-report.md and the README both carry the facade table, the two divergences, and the caveat that a green exFAT column is not automatically coverage. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BsYQdKe1BGsfm1wqLw4KUo check-in: 450213b14a user: refaqtor tags: trunk | |
| 20:20 |
6110: FIRST SUITE PORTED TO THE fs_* FACADE AND RUN ON exFAT - 38/38 FAT32, 36/38 exFAT, and the two failures are a real driver divergence (tickets 8bcfaaa605, 08de1c7aaa)
FS_seek_tests on FAT32 (drive A) 38 of 38 PASS <- the control FS_seek_tests on exFAT (the SSD) 36 of 38 <- first ladder suite ever on exFAT THE FAT32 RUN IS WHAT MAKES THE exFAT NUMBER MEAN ANYTHING. FS_seek_tests is a behavioural port of RT_seek_tests - same tests, same order, same expected count - so a port that changed behaviour would show a different number and one that quietly dropped assertions would show a smaller one. 38 matching 38 says the port is faithful; only then is 36 of 38 a result rather than an artefact. THE TWO FAILURES ARE A GENUINE FACADE PARITY DEFECT, filed on 08de1c7aaa: fs_seek(h, past_EOF) FAT32 -> REFUSES, per its explicit contract "position must be <= file size (cannot seek past EOF)" (usb_fat32_fs.spin2:1119) exFAT -> SILENTLY CLAMPS to h_size and returns SUCCESS (usb_exfat_fs.spin2:380-383) The exFAT side is the dangerous one: a silent clamp does not merely differ, it MISLEADS. An application seeking to a computed offset and writing gets an APPEND AT EOF instead of an error, where the same caller bug on FAT32 surfaces immediately. Silent relocation of a write position is exactly the defect shape this project exists to catch. NOT FIXED - the choice is real (make exFAT refuse, make FAT32 clamp, or define and enforce the contract in usb_app the way fs_chdir already reconciles cwd semantics), and fs_seek's documentation currently says nothing about past-EOF at all. THIS ONE RESULT IS THE ARGUMENT FOR THE WHOLE PORT. Neither driver-bound ladder could ever have found it: RT_seek_tests only ever runs against usb_fat32_fs, so exFAT's seek path had never been asserted on by anything. One suite, pointed at the interface applications actually use, ran on both filesystems and produced the divergence on the first attempt. THE RECIPE, now mechanical rather than a design question: - every sd.<driver method> becomes fx.fs_<facade method>. RT_seek used twelve distinct symbols and all twelve map. - sd.SUCCESS becomes the literal 0 - the facade's contract is 0 for success, and importing the driver's constant would mean importing the driver. - the driver OBJ line goes. The suite stops referencing usb_fat32_fs at all, which is precisely why the same source runs on exFAT. THE FIXTURE FORWARDS, AND IT HAS TO. usb_app keeps state in VAR, so a suite declaring its own usb_app gets a SECOND INDEPENDENT DRIVER INSTANCE - mounting nothing while the fixture's instance holds the real session. rt_usb_fixture therefore gained plain fs_* passthroughs: each a single forward to the identically-named facade method, no branching, no error translation, no defaults. NOT the rt_fs shim that was rejected, which would have been a second COMMON INTERFACE re-implementing usb_app's semantic reconciliation somewhere free to drift. NAMED FS_ RATHER THAN RT_ DELIBERATELY: run_gate.sh globs RT_*_tests.spin2, so the 346/346 baseline is untouched by this file existing. The facade ladder grows beside the driver ladder until it is complete enough to replace it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BsYQdKe1BGsfm1wqLw4KUo check-in: 587ac0ca4d user: refaqtor tags: trunk | |
| 19:38 |
6130: THE GEOMETRY MATRIX IS COMPLETE for the 16 non-defrag suites - drive C's alignment-5 media, never run before today, is 332/332 clean. Plus a comprehensive regression report and a README summary (ticket 74f78296de)
drive cluster size alignment 16 non-defrag suites RT_defrag_tests A 32 KB 0 332 pass 0 fail 0 recov 14 pass (in the 346 baseline) B 4 KB 0 332 pass 0 fail 0 recov WEDGES THE DRIVE C 4 KB 5 332 pass 0 fail 0 recov WEDGES THE DRIVE ALL THREE GEOMETRIES PRODUCE THE IDENTICAL 332, zero failures and zero recovery events on each. Drive C had never had a single suite run against it before today, and its non-zero cluster alignment is the only geometry on the bench that reaches commit a359d54149's entryOffsetInCluster sign fix - so that fix is now EXERCISED ON LIVE MEDIA AND PASSING rather than argued from code reading. That is the reason this ticket existed. AND THE MATRIX SETTLES SOMETHING BY ELIMINATION: ALIGNMENT IS EXONERATED in the defrag wedge. Drives B (align 0) and C (align 5) behave identically in both directions - both pass all sixteen suites cleanly, both wedge on RT_defrag_tests. The variable they share is the 4 KB CLUSTER SIZE, not the alignment, and drive A's 32 KB clusters pass the same suite 14/14. Recorded as a correlation between suite, cluster size and wedge. NO MECHANISM PROPOSED - per operator direction the chase waits until the estate has been run. A PROCEDURAL FINDING WORTH AS MUCH AS THE DATA. run_gate.sh runs alphabetically, which puts RT_defrag_tests SECOND. On drive C that wedged the drive and cost the remaining fifteen suites, which then reported "pass=0 fail=1" apiece - AND THOSE WERE NOT TEST FAILURES. Every one was the fixture reporting "no FAT32 volume matched the selection among 3 device(s)": the drive was gone from the roster and the suite never started. That run's own total, "16 passed, 21 failed", is misleading and must not be read as an alignment-5 result. Re-running with an explicit sixteen-suite list, defrag excluded, produced the clean 332. On 4 KB media, run the explicit list and leave RT_defrag out or last - run_gate.sh already accepts a suite list, so this is discipline, not code. NEW: docs/regression-report.md. A full correctness-coverage report, companion to characterization-report.md which covers throughput. It carries the geometry matrix, the exFAT position (working by-serial fixture, freeSpace, tell/eof, and the structural reason the ladder cannot run there yet), the measured exFAT scan-storm limit, every defect found and fixed today, EVERY CLAIM WITHDRAWN, and a section titled "What is still untested" - because a coverage report that only lists successes is a marketing document. README gains a "Filesystem regression coverage" section with the matrix, what the 346 gate actually enforces (including that it fails on any recovery event), and the known gaps stated plainly: RT_defrag has never passed on 4 KB media, exFAT has no suite ladder, and multi-drive, hot-plug, malformed-media and exFAT-structural are not exercised at all. It closes with the same sentence the estate has always deserved - one drive at a time, one filesystem, well-formed media, no removals: a floor, not a finish line. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BsYQdKe1BGsfm1wqLw4KUo check-in: 2ea8927a2d user: refaqtor tags: trunk | |
| 16:49 |
6150: the settling run staged correctly - 16, 32 and 40 KB in ONE binary. Plus a reproducible number: 32 KB single-cluster files fail at op 20-21 (ticket 96a1cf0413)
THE ONE INTERNALLY-CONSISTENT RESULT SO FAR, and it argues against cluster count: 32 KB files - EXACTLY ONE CLUSTER on drive A's 32 KB clusters - failed at OP 20 (log headless_260826-122003, commit 42904687cd) and at OP 21 (log headless_260826-124228, commit 923d80e6ef). Two runs, same workload code, one op apart. Under the cluster-count reading a single-cluster file has no FAT chain and should not fail at all. It still cannot be compared against the old 16 KB / 120-op clean number, which came from the pre-two-pass harness. That is the whole point of the staged run. STAGED AND VERIFIED PRESENT THIS TIME: -D SOAK_CLUS_EDGE runs 16, 32 and 40 KB in ONE binary on one drive. 16 and 32 KB are both single-cluster; 40 KB is two clusters. 16 clean, 32 clean, 40 fails -> cluster count, within one drive AND one harness 16 clean, 32 fails -> byte size per file; cluster count is dead all three fail similarly -> neither; something else scales with file size A RESEAT WAS SPENT ON A FAILED EDIT, and that is worth writing down because the check I used was wrong rather than absent. The 16 KB phase had not actually been inserted - an earlier combined command was killed before its python ran - and I "verified" it with a grep for `run_phase_n(AX_SIZE, BASE_CHUNK, 16` that MATCHED A DIFFERENT BLOCK ENTIRELY (the SOAK_SIZE_SMALL arm, which legitimately contains that exact call). The grep returned 1, I read it as confirmation, and the run that followed tested the old binary and wedged a drive for nothing. The header line in the log said so plainly - "32 KB (1 clus) then 40 KB", with no 16 - and the first phase was 32 KB. THE LOG DISAGREED WITH WHAT I BELIEVED I HAD BUILT, and that disagreement was visible in the first four lines of output. Verification now greps WITHIN the block being edited (sed -n '/IFDEF SOAK_CLUS_EDGE/,/ELSEIF/p'), and the run's own header is read before trusting that a build is the build intended - the same discipline that already applies to comparing commit ids across logs. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BsYQdKe1BGsfm1wqLw4KUo check-in: 1dda9881d7 user: refaqtor tags: trunk | |
| 16:34 |
6150: WITHDRAWING the cluster-count conclusion - the comparison spanned a harness change. The build stamp made it checkable; I did not check (tickets 96a1cf0413, 1464ab6579)
THE CONFOUND: drive A, 16 KB, 120 ops CLEAN -> commit b76957865c drive B, 16 KB, failed at op 34 -> commit 42904687cd the two-pass rewrite landed at commit 1ddfa594d2, BETWEEN THEM Before that change an op wrote each chunk ONCE; after it, one_op writes the file sequentially AND rewrites every chunk, so an op moves 50% more data plus an extra seek per chunk. Drive B's ops were doing strictly more work than drive A's. The sentence that carried the conclusion - "same bytes, same op count, same chunk size, same code" - IS FALSE across those two runs. The two-pass change was made for good reasons (it makes access pattern a single axis and removed a seek-past-EOF contract violation) but it changed the WORKLOAD, and every number from before it is incomparable with every number after. WHAT IS UNDONE AND WHAT IS NOT: - Cluster count is UNSUPPORTED, not disproven. It may still be right; the evidence offered does not establish it. - The exclusions all stand, because each was measured WITHIN one harness: transfer size (4096 clean as phase 1, 512 failing as phase 4, same run), BOT transaction count (2x difference, identical op count at failure), cumulative ops and cumulative bytes (120 ops / 3.84 MB clean in the very run that then failed at 256 KB). AND ONE NEW INTERNALLY-CONSISTENT MEASUREMENT that argues against cluster count: 32 KB files - EXACTLY ONE CLUSTER on drive A's 32 KB clusters - FAILED AT OP 20 under the current harness. A single-cluster file should not fail at all under that reading. It stands alone until 16 KB is re-measured with the same binary. STAGED, NOT RUN: -D SOAK_CLUS_EDGE now runs 16, 32 and 40 KB IN ONE BINARY on one drive - 16 and 32 both single-cluster, 40 two clusters - so the only numbers compared are produced by identical code in one session. The estate is down to the SSD alone and it needs a reseat before that can run. THE PROCESS CHANGE IS THE MORE VALUABLE OUTCOME. A COMPARISON ACROSS TWO LOGS IS ONLY VALID IF THEIR COMMIT IDS MATCH. The comparability header exists exactly so this is checkable, and it worked - the ids differ and the difference was sitting in both logs the whole time. What was missing was the discipline of LOOKING before comparing. From here, every multi-run claim on 6150 names the commit id of each run it rests on, and any claim spanning two ids is unsupported until re-measured in one binary. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BsYQdKe1BGsfm1wqLw4KUo check-in: 923d80e6ef user: refaqtor tags: trunk | |
| 15:53 |
6150: scatter fix VERIFIED, and a constraint that cost a reseat - killing pnut-term-ts does NOT stop the P2 (ticket 96a1cf0413)
VERIFIED, not assumed (log harness/logs/headless_260826-114103.log, commit 1ddfa594d2 clean): the SCATTERED phase now runs 8 ops with 0 failures, 0 verify failures and 0 recoveries, with the sequential control clean in the same log. So the 8-of-8 CONTENT MISMATCHES reported earlier are confirmed as MY harness bugs and not media or driver faults, and the two-pass rewrite with checked seeks is correct. A fix to a verifier is now seen passing rather than believed. THE CONSTRAINT, worth more than the run it cost: KILLING pnut-term-ts DOES NOT STOP THE P2. The terminal only reads the debug stream; the chip keeps executing whatever was downloaded into it. The scatter-verification run was killed at the terminal DELIBERATELY and for a good reason - to protect the thumbs for the drive-B cluster-count experiment that needed them healthy. The soak carried on headless through its remaining phases, wedged every thumb on the hub, and the only thing the "protective" action achieved was to hide the damage while it happened. The next command found "target serial not found among 1 device(s)". TO ACTUALLY STOP A RUN: download something harmless (top_rig_probe.bin). The download resets the P2, and the reset is what ends the program. Recorded in the soak header where someone about to do this will read it, and in PROMPT-NEXT-SESSION.md's running-things constraints beside the "never run two pnut-term-ts at once" rule it sits next to. CONSEQUENCE: the drive-B experiment is built and staged but did not run. It is the discriminator between CLUSTER COUNT and BYTE SIZE PER FILE - 16 KB files on drive B's 4 KB clusters are FOUR clusters each, where the same 16 KB on drive A's 32 KB clusters is a single cluster with no FAT chain, and that single-cluster case is the only file size that has never failed. Next thing to run. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BsYQdKe1BGsfm1wqLw4KUo check-in: 42904687cd user: refaqtor tags: trunk | |
| 15:32 |
6150 arm 2: BOTH cumulative hypotheses are DEAD - and two harness bugs of mine were raising false CONTENT MISMATCHES (ticket 96a1cf0413)
PHASE 1 16 KB files, 120 ops -> 120 ok, 0 fail, 0 verify_fail, 0 RECOVERIES PHASE 3 128 KB files, 8 ops -> clean PHASE 4 256 KB files -> FAILED AT OP 1 120 OPS CLEAN KILLS THE ~25-OP ACCOUNT. 3.84 MB CLEAN KILLS THE ~3 MB ACCOUNT. Arm 1's first failure at op 7 with 256 KB files matched the byte prediction and looked like an answer; arm 2 existed because arm 1 alone could not separate "cumulative bytes" from "large files are simply harder". It is not bytes: 3.84 MB of 16 KB files passed without one failure or recovery, then 256 KB files failed on their FIRST op. file size ops before first failure bytes clean 16 KB 128+ (never failed) 4.1 MB 128 KB 8+ (never failed) 2 MB 256 KB 1 (after 136 clean ops) / 7 (from fresh) 64 KB ~25 (three earlier runs) No constant op count and no constant byte count fits any two rows. THE HYPOTHESIS THIS SUGGESTS, AS A HYPOTHESIS: CLUSTERS PER FILE. Drive A has 32 KB clusters, so 16 KB is a SINGLE-CLUSTER file with no FAT chain - the one size that never failed. 64 KB is 2 clusters, 128 KB 4, 256 KB 8, and failure arrives sooner as the chain lengthens. It would also explain why this project's signature corruption has always been multi-cluster: 6100's ~25 ops and the /TOOLS blob damage are both multi-cluster stories. THE DISCRIMINATOR IS CHEAP AND THE MEDIA ARE ALREADY ON THE RIG. Drives B and C have 4 KB clusters, so 16 KB there is FOUR clusters instead of one. Arm 2 unchanged on drive B: fails early => CLUSTER COUNT is the variable; runs 120 ops clean => it is byte size per file after all. One run, existing hardware. BYSTANDER WITNESS, FIRST READING: SSD at addr 2 before and after, and ALL FOUR DEVICES still present and mounted at the end - despite 33 recoveries. This run produced NO bystander loss, so whatever took the thumbs out in three previous runs is not an unavoidable consequence of recoveries. Ganged power is weakened, not dead: no witness re-enumeration was seen because no bystander was lost. TWO HARNESS BUGS, BOTH MINE, BOTH RAISING THE MOST SERIOUS SIGNAL THIS PROJECT HAS. Phase 5 reported CONTENT MISMATCH on 8 of 8 ops - "the right number of the WRONG BYTES". IT WAS NOT REAL. BUG 1: the scattered writer stamped chunk i then seeked to scatter_ix(i), so position p held the stamp of whichever i mapped there while the verifier expected stamp p. scatter_ix(64)=0, so chunk 0 held stamp 64 - precisely the "MISMATCH at chunk 0" reported eight times. BUG 2, found when fixing bug 1 moved the symptom to chunk 1 rather than curing it: the writer SEEKED PAST EOF in a freshly created empty file, against seekHandle's explicit contract "position must be <= file size (cannot seek past EOF)" (usb_fat32_fs.spin2:1119) - AND NEVER CHECKED fs_seek's RETURN, so the documented refusal was discarded and the write landed at the wrong offset. THE DRIVER WAS RIGHT BOTH TIMES. Once I had the arithmetic wrong; once I violated a documented precondition and then ignored the error that said so. A false content mismatch is worse here than a missed one - it is the signal everything else defers to. FIXED AND EXPLICITLY NOT VERIFIED: one_op now writes sequentially first (identical in both arms, which also makes access pattern a genuinely single axis instead of "one write pass versus two"), then rewrites each chunk at the same position - in order or scattered - with EVERY SEEK CHECKED. -D SOAK_SCATTER_ONLY exists to prove the scattered phase verifies clean against a sequential control in one log. It has not run: the drives wedged first. A fix to a VERIFIER is not to be believed until it is seen passing, so nothing about it is claimed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BsYQdKe1BGsfm1wqLw4KUo check-in: 1ddfa594d2 user: refaqtor tags: trunk | |
| 15:12 |
6150: a BYSTANDER WITNESS in the soak - the free discriminator between ganged hub power and cross-device contamination (tickets 96a1cf0413, fea65c4b2d)
THREE CONSECUTIVE RUNS have ended with every untouched thumb gone from the roster while the soak bound exactly one drive. That is 6140's bystander invariant with three independent observations behind it, and two candidate mechanisms have stood unseparated the whole time: cross-device contamination, or GANGED HUB POWER (drive_recover() cycles the bound device's port, and the driver's own escalation message names a ganged hub as a possibility). THE DISCRIMINATOR COSTS NOTHING AND IS NOW WIRED IN. A device only re-enumerates after losing power or being reset, so: - a ganged power cycle re-enumerates EVERY device, and a witness that never faulted comes back AT A NEW BUS ADDRESS; - cross-device damage leaves the witness's address alone. The soak now records the address of a device it will never touch - the exFAT SSD when soaking a thumb, drive A when soaking the SSD - at the start of the run and again in the summary, and states which reading the result supports: gone from the roster -> the bystander invariant again, mechanism unseparated address CHANGED -> evidence for ganged power, not cross-device damage address unchanged -> whatever happened to the other drives was NOT a bus-wide power cycle It costs one poll() at each end, which is traffic the idle loop does anyway. This is deliberately read out of runs that are HAPPENING REGARDLESS rather than given runs of its own, because every wedging experiment costs a physical reseat of three thumbs and that reseat is the real budget constraint on this line of work. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BsYQdKe1BGsfm1wqLw4KUo check-in: b76957865c user: refaqtor tags: trunk | |
| 14:17 |
6150: the OPS-vs-BYTES experiment - two arms predicting opposite directions from the same hypothesis (ticket 96a1cf0413)
The reverse-order run established that the wedge tracks cumulative WORK, and ruled out both transfer size and BOT transaction count. What it could NOT separate is OPS from BYTES: every phase compared used a 64 KB file, so bytes were strictly proportional to ops and the two moved together in lockstep. FILE SIZE SEPARATES THEM, and the predictions are an order of magnitude apart - which is what makes this decisive rather than suggestive. At 64 KB the wedge arrived at ~25 ops = ~3 MB moved (each op writes the file and reads it back). arm 1 -D SOAK_SIZE_BIG 256 KB files = 512 KB/op, 40 ops BYTE-limited -> first failure ~OP 6 OP-limited -> ~OP 25 arm 2 -D SOAK_SIZE_SMALL 16 KB files = 32 KB/op, 120 ops BYTE-limited -> first failure ~OP 96 OP-limited -> ~OP 25 TWO ARMS BECAUSE ONE WOULD NOT SETTLE IT. Arm 1 alone, failing early, is equally well explained by "large files are simply harder" - more clusters, longer chains, bigger allocations. Arm 2 is what rules that out: if the wedge is byte-limited it must arrive LATE with small files, not early. A hypothesis that predicts opposite directions in the two arms can be killed by either one. MEASUREMENT PLUMBING ADDED FOR IT: - run_phase_n() takes an explicit op count, so an experiment can run one long phase instead of being quantised to the 8-op default. run_phase() keeps its old signature and forwards, so every existing phase is untouched. - FIRST FAILING OP is recorded globally, once, at the first failure - a later cascade cannot overwrite it - and printed in the summary. That index IS the measurement here; everything else in the run is context. Both arms hold every other axis at baseline: chunk 512 B, gap 0, sequential, same drive, same content verification. NOT YET RUN. This commit is the experiment and its plumbing, compiled clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BsYQdKe1BGsfm1wqLw4KUo check-in: a61759d6af user: refaqtor tags: trunk | |
| 13:46 |
6150: -D SOAK_CHUNK_REVERSE - the experiment that separates SIZE from CUMULATIVE LOAD in the 4096-byte correlation (ticket 96a1cf0413)
Two runs found chunk=4096 B failing 2 of 2 while 512/1024/2048 were clean 6 of 6. That is a real correlation and it is NOT yet an attribution, because in both runs the axis was walked 512 -> 4096 and 4096 was ALWAYS THE FOURTH PHASE. Three explanations survive equally: (a) the SIZE 4096 is what breaks it (b) the FOURTH PHASE breaks it, whatever size it carries (c) CUMULATIVE transactions since power-on break it, and phase 4 is merely where the count arrives THE FIRST PHASE OF THIS BUILD IS THE WHOLE EXPERIMENT: 4096 B runs FIRST, on a freshly bound device, with nothing before it. fails at phase 1 -> SIZE. (b) and (c) both die - no prior phases, minimal prior transactions. clean at phase 1 -> not size alone, and then the repeat decides: PHASE 5 runs 4096 AGAIN after the three smaller phases. Failing there, having passed at phase 1, means CUMULATIVE LOAD or phase ordinal - the size is exonerated. The smaller sizes still run in between, so this build re-tests them too and the two orderings stay comparable phase-for-phase on everything except order. The header now prints which ordering is in force, and what each outcome would mean, because two logs differing only in a compiler flag are indistinguishable afterwards - and this pair differs ONLY in a compiler flag. THE BUILD STAMP DID ITS JOB ON THE WAY IN: the first build of this change was made from a dirty tree and the generator stamped it "d333673d64+ *** UNCOMMITTED CHANGES - dumps from this build are not reproducible from the repo ***". That is precisely the case it exists to catch, so this landed before the run rather than after. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BsYQdKe1BGsfm1wqLw4KUo check-in: 1f90083ee5 user: refaqtor tags: trunk | |
| 13:41 |
6150 soak run 2: the 4096-byte transfer-size correlation REPRODUCED (2 of 2), and the bystander question raised again by three untouched drives going dark (ticket 96a1cf0413)
THE RESULT. Driver commit ae549801a6 (clean), drive A, one axis moved per phase: PHASE 1 baseline chunk=512 B 8 ops 0 fail 0 verify_fail 0 RECOVERIES PHASE 2 TRANSFER SIZE chunk=1024 B 8 ops 0 fail 0 verify_fail 0 RECOVERIES PHASE 3 TRANSFER SIZE chunk=2048 B 8 ops 0 fail 0 verify_fail 0 RECOVERIES PHASE 4 TRANSFER SIZE chunk=4096 B 5 fail, 1 CONTENT MISMATCH PHASE 5 FILE SIZE chunk=512 B wedged; re-bind FAILED; RUN ABANDONED ACROSS BOTH RUNS: 512/1024/2048 clean in 6 of 6 phase-runs; 4096 FAILED IN 2 OF 2, with a content mismatch each time. The first axis correlation this project has been able to state, and precisely what the 6150 apparatus was built to produce. WHAT IS NOT ESTABLISHED AND MUST NOT BE ROUNDED UP: the chunk axis is always walked in the same ORDER and always as phases 2-4, so "4096" is not yet separated from "the fourth phase" or "cumulative transactions since power-on". Next run walks it in REVERSE (4096 first) and repeats it mid-run. If 4096 fails first-thing on a fresh device, SIZE is the variable; if it only fails after the smaller phases, CUMULATIVE LOAD is. THE FAILURE SHAPE DIFFERED BETWEEN RUNS, which is itself data: run 1's phase 4 took 2_915 ms with 16 recoveries; run 2's took 1_451_415 ms - 24 MINUTES - with ZERO recoveries, then a short read at chunk 59 and every later op failing at create. Same axis, same failure count, completely different profile. THE 6220 E_DISK_FULL MASKING SHOWED UP IN LIVE EVIDENCE. Ops 4-8 report "fs_create FAILED st=-60" - DISK FULL - on a 7.5 GB thumb holding one 64 KB file, because do_create() collapses any negative from allocateCluster() into E_DISK_FULL (usb_fat32_fs.spin2:3878). Filed from a synthetic test yesterday; here it is actively corrupting the reading of a real soak. It should be fixed. BOTH HARNESS FIXES FROM LAST NIGHT EARNED THEIR PLACE: - The abandon logic is why this run is readable at all. Phase 5 wedged, one re-bind by serial was tried, it failed, and the run STOPPED - saying "any axis beyond the last valid phase is UNTESTED, not proven clean". Run 1 without it produced eight further phases of artefact that read like four axes correlating. - The sliced gap wait means the multi-minute phases would actually have waited, had the run reached them. It did not, and the summary says so rather than leaving them looking untried-but-fine. COLLATERAL, AND IT RAISES 6140's BYSTANDER INVARIANT DIRECTLY. After the run the roster holds ONE device: the exFAT SSD. All three FAT32 thumbs are gone while ALL FOUR PORTS REPORT $0103 - connected AND enabled - which by the discriminator is the device-wedged class needing a physical reseat. THE SOAK ONLY EVER BOUND DRIVE A. B and C were enumerated at settle, never selected, never addressed, never written - and they went dark anyway. Two candidate mechanisms, NOT chosen between: cross-device contamination (6140/6200, and 22 recoveries ran this session), or GANGED HUB POWER, since drive_recover() cycles the bound port and the driver's own escalation message names a ganged hub as a possibility. Against the ganged reading: the SSD on port 1 survived and stayed mounted, though it may simply have re-enumerated where the flakier thumbs did not. The cheap discriminator is to watch the SSD's BUS ADDRESS across a soak that takes recoveries - a ganged cycle re-enumerates it at a NEW address even though it never faulted. ALSO FIXED HERE, found by re-reading my own output: the summary line labelled evt_count() as "held" and printed "7_648 held" against a 256-SLOT ring. count() returns records WRITTEN since arm, including overwritten ones. Now printed as written / overwritten / (256 held). A wrong label survives because nobody re-reads their own summary. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BsYQdKe1BGsfm1wqLw4KUo check-in: d333673d64 user: refaqtor tags: trunk | |
| 05:38 |
6150 soak, first run: THE WEDGE CORRELATES WITH TRANSFER SIZE - clean at 512/1024/2048 B, wedged at 4096 B. Plus two flaws in my own harness that the run exposed (ticket 96a1cf0413)
THE FINDING, AND IT IS EXACTLY WHAT THIS TICKET WAS BUILT TO PRODUCE. Log harness/logs/headless_260826-013051.log, drive A, every axis but one held at baseline: PHASE 1 baseline chunk=512 B 8 ops 0 fail 0 verify_fail 0 RECOVERIES PHASE 2 TRANSFER SIZE chunk=1024 B 8 ops 0 fail 0 verify_fail 0 RECOVERIES PHASE 3 TRANSFER SIZE chunk=2048 B 8 ops 0 fail 0 verify_fail 0 RECOVERIES PHASE 4 TRANSFER SIZE chunk=4096 B FAILED - 5 fail, 1 CONTENT MISMATCH, device WEDGED Three phases clean, the fourth wedges the drive, and the ONLY thing that moved is the maximum BOT data transfer size. "If the wedge correlates with an axis, THAT IS THE FINDING" - and this is the first time the estate has been able to say so, because it is the first workload that varied one axis at a time with the axis recorded into the evidence ring. IT IS ONE SAMPLE. n=1 on the 4096 phase, and the run that produced it was contaminated afterwards (below). It is a strong lead, NOT an established correlation, and it must be repeated - ideally with the chunk axis walked twice and the order reversed, so "4096" is separated from "the fourth phase" and from "cumulative reads". THE NEW FAILURE-CAUSE INSTRUMENTATION EARNED ITS KEEP IMMEDIATELY. The phase-4 failure reads: BOT data fail op=$28 hs=STALE(no handshake this xfer) phy_st=-2 sop=0 rxn=0 Same total-silence signature as 6200: PHY timeout, NO start-of-packet at all, and the handshake byte correctly labelled STALE instead of being reported as a NAK it never was. Then recovery gave up (rst=0 clrO=0 clrI=0), escalation to drive_recover failed repeatedly, and control transfers came back "CFG hdr fail hs=$100" - $100 being "no peer SOP ever seen". The device went completely silent to EVERYTHING, not just to bulk. TWO FLAWS IN MY OWN HARNESS, BOTH FOUND BY RUNNING IT, BOTH FIXED HERE: 1. THE LONG-GAP AXIS NEVER WAITED. waitms() computes WAITX((clkfreq/1000) * ms); at 200 MHz that is 200_000 * ms, which OVERFLOWS 32 BITS above ~21_474 ms. So waitms(60_000) does not wait a minute. MEASURED: phases 11 and 12, nominally 60 s and 300 s gaps over 8 ops each, both "completed" in 1_715 ms. THIS ONE MATTERED MORE THAN AN ORDINARY BUG. The multi-minute gap is the ONE axis 6200's idle hypothesis needs, and a silent zero-length wait would have produced a confident "long gaps make no difference" from a test that never waited - the worst kind of green. p2kb's "1-4,294,967 range" for waitms is the PARAMETER range, not the achievable delay, which is clock-dependent; the real ceiling here is the same ~21 s boundary the evidence ring already documents for getct at this clock. Now sliced into 10 s pieces. 2. THE RUN KEPT "MEASURING" AFTER THE DEVICE DIED. Phase 4 wedged the drive and the soak then ran EIGHT MORE PHASES against it, reporting per-axis failure counts that were purely an artefact of the wedge. A reader skimming that summary would see failures spread across four axes and conclude all four correlate, when only the first one did. That is exactly the "reproduction is not verification" trap in a new costume: the numbers looked like data. Now: if the device is still wedged at the end of a phase, ONE re-bind is attempted (by SERIAL - it returns at a new address), and if that fails the run is ABANDONED with an explicit note naming the last VALID phase and stating that later axes are UNTESTED, not proven clean. A short honest run beats a long contaminated one. The recovery counter added with the harness worked exactly as intended and is what made the contamination visible: phases 5-12 reported RECOVERIES=25..32 apiece, which is not a workload result, it is a dying drive. The drive recovered fully on its own afterwards - probe 4 of 4 devices, all mounted, no operator intervention. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BsYQdKe1BGsfm1wqLw4KUo check-in: ae549801a6 user: refaqtor tags: trunk | |
| 05:29 |
6150: the instrumented soak - comparability header, one-axis-at-a-time workload variation, unattended auto-recover. Gate green 346/346 (ticket 96a1cf0413)
THE RECORDER WAS LANDED AND PROVEN FIRST; THIS IS THE THING THAT DRIVES IT. All three pieces the ticket said were still owed, and explicitly NOT claimed until now: 1. THE COMPARABILITY HEADER. Every run prints driver commit id, stamp time, clock, media name and SERIAL, filesystem kind and label, geometry (sec/clus, first data sector, cluster alignment), capacity, and the workload baseline. A pattern is a statement about MANY runs; without this each dump is an anecdote and two dumps cannot be compared at all. THE COMMIT ID NEEDED A GENERATED FILE because pnut-ts has NO -D NAME=VALUE - its -D takes bare symbols only, so a hash cannot be passed on the command line. tools/gen_build_stamp.sh writes src/build_stamp.spin2 before the build, and DIRTY TREES ARE MARKED, NOT HIDDEN: a build from a modified checkout gets a '+' and an explicit "UNCOMMITTED CHANGES - not reproducible from the repo" line in the header. A dump claiming a clean commit id while the source had uncommitted edits is worse than one with no id, because it invites a reader to diff against a revision that was never built. The generated file is in .fossil-settings/ignore-glob rather than committed - committing it would make the tree dirty on every build and the id circular. 2. ONE AXIS AT A TIME, EACH CHANGE RECORDED INTO THE RING. Phases: baseline (nothing varied, and it runs FIRST because every later phase is read against it), then TRANSFER SIZE (512 -> 1024 -> 2048 -> 4096 B via the new disk_set_max_chunk), FILE SIZE (16/64/128/256 KB - it drives cluster COUNT, which is what makes a chain long enough to break in the middle), ACCESS PATTERN (sequential vs scattered-by-seek: same bytes, same file, ONLY the order differs), and INTER-OP GAP. Every phase stamps MK_PHASE/MK_AXIS into the ring before anything moves, so a dump says which axis was in effect without needing the debug log beside it - and a ring wrap cannot desynchronise them. If the wedge correlates with an axis, THAT IS THE FINDING - but only if exactly one thing moved. A soak that varies size and gap together produces failures nothing can be attributed to. THE GAP AXIS REACHES MINUTES, behind -D SOAK_LONG_GAPS (60 s and 300 s phases). 6200's evidence is that the fault appears on the first activity after the rig has been idle; a gap axis topping out at a second could never see it. It is a build gate because those two phases alone take longer than the whole rest of the run, and the default build says so explicitly rather than silently omitting them. 3. UNATTENDED, WITH disk_auto_recover(1, 3000) ON - the one place it belongs. A soak that stops at the first wedge collects ONE sample per session; one that recovers and continues collects many, and each recovery is itself a datum. WHICH REQUIRED COUNTING THEM. With auto-recover on, escalation happens INSIDE disk_read/disk_write and was invisible to the caller. usb_app now counts at the single site that can escalate (auto_recover_retry) and exposes disk_recoveries() / disk_recoveries_clear(), so the soak attributes recoveries to the PHASE they happened in. Every phase line ends with RECOVERIES=n, and a nonzero count prints "NOT the same result as a clean phase". A run that finished after three recoveries is not a clean run and must never be reported as one. CONTENT VERIFICATION, NEVER SIZE. Every 512-byte chunk is stamped with a marker, its OWN index, the phase and the file size, then read back and compared LONG BY LONG. The signature failure of this project is a perfect directory entry over a broken chain - THE RIGHT NUMBER OF THE WRONG BYTES - and a size comparison passes straight through it. Mismatches are counted separately from I/O failures and called out as the headline in the summary, because they mean something completely different. THE SCATTER IS DETERMINISTIC (odd chunks then even), not random. A random order would make two runs incomparable, which is the one thing this soak must not do. FACADE ADDITIONS, all needed by the above and all useful beyond it: disk_set_max_chunk() (the transfer-size axis was reachable only on the MSC object - an axis the harness cannot move is an axis that never gets tested), disk_recoveries(), disk_recoveries_clear(). GATE: 346 passed, 0 failed, 0 recovery events, 17 suites, RT_DRIVE=A, with all of this in. The baseline is intact. NOT YET RUN AS A SOAK. This commit is the harness and its plumbing, compiled and gate-clean. The overnight run and whatever it finds are the next entry. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BsYQdKe1BGsfm1wqLw4KUo check-in: d885de7742 user: refaqtor tags: trunk | |
| 04:40 |
fs_tell() and fs_eof() added to the application facade, proven identical on BOTH filesystems - and fs_eof took two wrong versions to get right (ticket 8bcfaaa605)
WHY THESE TWO. Operator's question - "are you building this shim just so the TEST HARNESS can call a common interface? don't we actually need that common interface for applications to work with?" - killed the rt_fs shim idea and reframed the whole exFAT effort. The common interface ALREADY EXISTS: usb_app's fs_* facade. Building a second one for the test harness would have duplicated a reconciliation already done at the right layer, had the regression estate validate an interface no application uses, and hidden the real finding - that the application API is INCOMPLETE. fs_tell and fs_eof were the two most serious holes in it. Both drivers have carried tellHandle()/eofHandle() all along; only the facade lacked them. An application streaming a file through fs_read() could not ask where it was or whether it had finished. The gap survived this long precisely because all seventeen regression suites talk to the DRIVER OBJECT directly and never exercise this layer at all - which is the argument for porting them to the facade, not for wrapping them in a shim. fs_eof IS NORMALISED TO 1 / 0 / negative, AND THAT NORMALISATION IS THE WHOLE REASON IT BELONGS AT THE FACADE. Underneath, both drivers return Spin2's native TRUE, which is -1. Their error codes are also negative. So at the driver level "at end of file" and "that handle does not exist" are BOTH negative, and an application writing the obvious `if eof(h)` treats an error as end-of-file: it stops reading and reports success on a file it never read. TWO WRONG VERSIONS PRECEDED THE RIGHT ONE, and both are recorded in the method because the trap is not obvious: 1. Forward the raw value, with a comment telling callers to test `> 0`. WRONG - -1 is not > 0, so the advice failed on the exact case it was written for. 2. Normalise with `(yn < 0) ? yn : ...` - which classifies TRUE (-1) as an error and passes it through unchanged, REPRODUCING THE EXACT AMBIGUITY the normalisation existed to remove. The discriminator cannot be the SIGN, because TRUE and every error code are negative. It has to be the VALUE: the drivers return exactly -1 for true, and no error constant in either driver is -1 - checked, not assumed (exFAT's E_BAD_HANDLE is -87; FAT32's are -7 and -22..-65). BOTH WRONG VERSIONS COMPILED, AND THE SECOND PASSED EVERY "eof is FALSE" ASSERTION. Only asserting the TRUE case caught them, and only a diagnostic printing the RAW returned value showed why - the failure looked identical to a driver bug until the -1 was on screen. That is the argument for asserting both polarities of a predicate rather than the interesting one. THE TEST RUNS AGAINST THE FACADE ON BOTH FILESYSTEMS, which is the only way to test a common interface. harness/top_6110_tell_eof.spin2, built -D T_EXFAT for the SSD and -D T_FAT32 for the drive-A thumb, both selected BY SERIAL: exFAT (P2SSDxf) 15 of 15 passed, 0 FAILED FAT32 (drive A) 15 of 15 passed, 0 FAILED identical answers on every assertion Asserted: tell == 0 on a fresh handle; tell tracks writes EXACTLY (not "advances" - a short write cannot hide); tell == 0 on reopen; fs_file_size agrees; seek to a NON-sector-aligned position (700) and tell it back; eof FALSE mid-file; a read advances tell by exactly the bytes read; tell == size after seek(size); eof TRUE there. THE DISCRIMINATING ASSERTION IS eof-FALSE-AFTER-SEEKING-BACK. It proves eof is COMPUTED from the position rather than latched by having once hit the end. A latched flag passes every other assertion in the file and fails only that one. STILL MISSING FROM THE APPLICATION API, recorded on the ticket and not fixed here: fs_rename, fs_move, fs_set_vol_label, the contiguous/defrag family, and sectorsPerCluster/clusterBytes (derivable from fs_geometry but not offered). Each is a product gap, and each should be decided once at the facade the way fs_chdir already reconciles cwd semantics between the two filesystems. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BsYQdKe1BGsfm1wqLw4KUo check-in: 877c4d99bf user: refaqtor tags: trunk | |
| 03:38 |
6110: exFAT freeSpace() - the single method blocking the most suites - implemented, proven fail-first, and the SSD's scan-storm limit measured (ticket 8bcfaaa605)
WHY THIS METHOD FIRST. The per-suite blocking table recorded on the ticket says freeSpace() alone blocks RT_seek (38 tests) and RT_multihandle (22), and together with two accessors that volumeGeometry() ALREADY answers on both filesystems it also blocks RT_read_write (49), RT_stress (4) and RT_fatchain (2). It is the highest tests-unblocked-per-unit-work item in the whole exFAT estate. IMPLEMENTED: usb_exfat_fs.freeSpace(), returning SECTORS to match usb_fat32_fs's units so one FS-agnostic caller can ask either filesystem the same question. exFAT's on-media PercentInUse is a rounded percentage and useless for a test that must see a one-cluster change, so the count is a population count over the allocation bitmap. Also added usb_app.fs_free_space() - the fs_* facade had NO free-space call at all - dispatching on fs_fmt like fs_geometry does. RETURNS -1, NOT 0, ON AN UNREADABLE BITMAP, and the divergence from FAT32 is deliberate: zero is a LEGITIMATE answer meaning the volume is full, so reporting an I/O failure as 0 makes a broken device indistinguishable from a full one. 6210 was exactly that class of silent degradation and it cost a cross-device investigation a wrong turn; not repeating the pattern in a method written the same day. That decision earned itself back within the hour - see below. PROVEN FAIL-FIRST, AND THE PROOF FOUND A HOLE IN MY OWN TEST FIRST. The assertions are deltas across a known change, because "freeSpace() returned a number" proves nothing - a stub returning the volume size returns a plausible number too. But writing them was not enough: THE PADDING-BIT BUG WOULD HAVE PASSED EVERY ONE OF THEM. The bits above clus_count are undefined on media and counting them as free is the easy mistake, and it leaves BOTH deltas exactly right. So a bound was added that catches it: free space cannot exceed the CLUSTER HEAP, derived from geometry as (block device - heap start) / sectors-per-cluster, which on this volume is exactly the 7_813_842 clusters the driver reports at mount. Then the mask was DELIBERATELY REMOVED and the suite re-run on hardware: free = 2_000_344_576 vs 2_000_340_992 correct = +3_584 sectors = EXACTLY 14 clusters - the 14 padding bits predicted from clus_count mod 32 = 18 "does not exceed the block device" PASSED <- too loose, as predicted "allocate then free returns exactly" PASSED <- deltas unaffected, as predicted "does not exceed the CLUSTER HEAP" FAILED <- the only one that discriminates That is the whole point of fail-first: it proved the new assertion works AND proved the others cannot catch this bug, which is why the assertion had to exist. Mask restored, both arms 5 of 5. THE SSD'S SCAN-STORM LIMIT, MEASURED THE HARD WAY. The first version of the harness took THREE full bitmap scans in one session. Each is ~1_900 sector reads on this 953 GB volume, and THE THIRD ONE WEDGED THE SSD - twice out of two, at the same point, the second ending in "BOT recover GIVING UP after 3 ... ST_BOT_WEDGED" (logs headless_260825-232250 and -232334). That is ticket 6030's scan-storm hazard, until now seen only on the FAT32 thumbs, REPRODUCED ON exFAT AND ON A DIFFERENT VENDOR'S CONTROLLER. Two things follow, and both are recorded rather than worked around: - The harness now splits its two assertions across TWO BUILDS of two scans each (default = round trip, -D FS6110_DELTA = one-cluster delta). Two scans per run is a MEASURED HARD LIMIT on this media, not a style choice, and the header says so with the log names. - freeSpace() on exFAT has freeSpaceRescan()'s COST under freeSpace()'s NAME, because there is no tracked count to serve. Documented in the method itself as an audit call and not a poll. THE TRACKED COUNT IS THE REAL FIX and is deliberately NOT built here: a counter maintained by alloc_cluster/free_chain would make this O(1), and it needs its own test because a counter that drifts from the bitmap is worse than a slow exact answer. And the -1-on-error decision paid for itself here: when the third scan hit the wedge, the method returned -1 and the test FAILED LOUDLY. Had it returned 0 it would have reported a full 953 GB volume as a plausible-looking number. The SSD self-healed under ordinary poll() afterwards - probe 4 of 4 mounted, wear hint present, highest in-use clus=11 confirming the test file was cleanly deleted and no residue was left on the operator's volume. WHAT THIS UNBLOCKS AND WHAT IT DOES NOT. freeSpace() exists now, and sectorsPerCluster()/clusterBytes() are derivable from volumeGeometry() today. The remaining barrier to running the suites on exFAT is structural and unchanged: all seventeen bind `sd : "usb_fat32_fs"` directly. The rt_fs shim that would fix that with a one-line change per suite is still NOT built - it alters what the 346/346 baseline exercises and remains flagged for the operator. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BsYQdKe1BGsfm1wqLw4KUo check-in: 612aad450d user: refaqtor tags: trunk | |
| 03:13 |
6110: the exFAT fixture works and bound the SSD for the first time - and the port is not the shape the ticket assumed (8bcfaaa605). Plus a correction that downgrades 6210 (e480049b6d) from Critical to Low.
FIRST: A CORRECTION TO MY OWN TICKET FROM EARLIER TODAY. 6210 claimed exFAT's fs_mount() returning success over a failed allocation-bitmap scan was a CORRUPTION risk - "a next-fit hint derived from half a bitmap could hand out a cluster that is in use". THAT IS FALSE, on two counts, and both were findable by reading two more methods before filing: - The hint is NOT half-derived. bmp_hint_init()'s read-error path is a bare return (usb_exfat_fs.spin2:1254-1256) leaving bmp_hint at its mount value of 2. An aborted scan contributes NOTHING rather than something wrong. - Allocation FAILS SAFE BY DESIGN. bmp_test() returns 1 (in use) on I/O error - "so allocation never grabs unknown state" (:1218-1223) - and bmp_mark() returns 0 on the same failure. It cannot hand out an occupied cluster this way. The tolerance is deliberate and was ALREADY DOCUMENTED at :1246-1247: "On any bitmap read error the hint stays 2 - allocation remains correct (bmp_test also fails safe), only the wear spread is lost." What survives is small but real and worth keeping: fs_mount() returns st=0 with NO programmatic signal that the scan aborted - the only evidence is the ABSENCE of a debug line and a mount that returns in 1.7 s instead of 4.8 s. That is a diagnosability gap, and its one demonstrated cost is that it made the SSD look healthy in 6200's analysis. Retitled, downgraded to Low/Minor, and the "WRITE NOTHING while this is open" instruction WITHDRAWN - it was based on the false claim and would have blocked this ticket's estate for no reason. Method note recorded on the ticket: the LBA prediction on 6220 was right because it was traced end to end; this one was wrong because it was extrapolated two methods past what I had read. THE FIXTURE, which was 6110's blocking sub-task, IS DONE AND PROVEN: - rt_drive_ssd = "2L192LQ5ANU9", selected by -D RT_DRIVE_SSD, never first-found - auto-binding on this rig finds the SSD FIRST, which is precisely the trap. - RT_FS_KIND (2 under -D RT_FS_EXFAT, else 1) so all seventeen existing suites build and mean exactly what they meant before. -D RT_DRIVE_SSD IMPLIES -D RT_FS_EXFAT, as an implication rather than an equivalence. - fs_kind_expected() / fs_kind_name() so a suite can assert the kind itself. - THE WRONG-FILESYSTEM CASE FAILS LOUDLY WHEN THE DEVICE WAS PINNED. Scanning still skips quietly - that is how the FAT32 suites step over the SSD and it must not change - but a PINNED device that mounts as the other kind now says so and returns false, instead of reporting "no volume matched", which reads as missing media when the drive is present and healthy. PROVEN: "[fixture] mounted exFAT on addr 2 port 1 VID=$174C serial='2L192LQ5ANU9' label='P2SSDxf'". First time any regression fixture has bound the exFAT volume. REGRESSION-CHECKED: RT_read_write_tests on drive A, FAT32, 49 Tests - Pass: 49, Fail: 0 - its exact historical count, so the fixture change costs the existing estate nothing. AND THEN THE SUITE FAILED EVERY SINGLE TEST, WHICH IS THE FINDING WORTH HAVING. All seventeen suites bind the FAT32 DRIVER OBJECT DIRECTLY - `sd : "usb_fat32_fs"` - and query it, not the usb_app fs_* facade the fixture mounts through. RT_read_write_tests alone makes 258 `sd.` calls. So it mounts exFAT correctly and then asks a FAT32 object with no volume, getting freeSpace=0, secPerClus=0, clusterBytes=0, and failing through. NO BUILD GATE CAN RETARGET THESE SUITES. This ticket proposed "same names, _xf suffix or an FS-agnostic build gate - decide once and apply uniformly"; the prior question is what the suites talk TO, and neither option answers it. Recorded so the next context does not spend the same run discovering it. THE GAP, MEASURED RATHER THAN ESTIMATED. FAT32 exposes 133 PUBs, exFAT 31, 28 shared; the suites use 85 distinct `sd.` symbols. 24 already exist in exFAT and are the whole core file/directory API. 29 are CONSTANTS - cheap to re-export, EXCEPT THAT THE VALUES DIFFER (exFAT's error codes are -80..-82), so re-exporting FAT32's numbers would make suites pass against the wrong codes. 32 are methods exFAT genuinely lacks. TWO OF THE THREE COMMONEST BLOCKERS ARE ALREADY SOLVED AND NOBODY NOTICED: volumeGeometry() exists in BOTH drivers with the identical three-LONG contract (usb_exfat_fs.spin2:264, usb_fat32_fs.spin2:1247), so sectorsPerCluster() is geo[0] and clusterBytes() is geo[0]*512 on either filesystem, today, with no driver change. That leaves freeSpace() as THE highest-value single gap - it alone blocks RT_seek (38 tests) and RT_multihandle (22), and with the two derivable accessors it also unblocks RT_read_write (49), RT_stress (4) and RT_fatchain (2). exFAT has no free-space method because the format stores only PercentInUse; an exact count needs a zero-bit population count over the allocation bitmap. The per-suite blocking table is on the ticket. Genuine FEATURE gaps needing the exFAT driver to grow, portable at no price today: changeDirectory, rename, moveFile, setVolumeLabel, getDate/setDate, and the contiguous/defrag family. NOT DONE, AND DELIBERATELY NOT DONE UNILATERALLY: the rt_fs shim that would let suites change one OBJ line and keep all 258 call sites. It is a structural change to all seventeen suites and therefore to what the 346/346 baseline exercises. Flagged for the operator rather than landed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BsYQdKe1BGsfm1wqLw4KUo check-in: cd1b5d4478 user: refaqtor tags: trunk | |
| 02:32 |
6150: the recorder was manufacturing evidence - fix the cause field, and the fault turns out to be TOTAL DEVICE SILENCE (tickets 96a1cf0413, 293576b2fc)
THE MEASUREMENT, which is the point of all of it. 11 failures across 4 four-device runs, every one identical: BOT data fail op=$28 hs=STALE(no handshake this xfer) phy_st=-2 sop=0 rxn=0 BOT data fail op=$28 hs=STALE(no handshake this xfer) phy_st=-2 sop=0 rxn=448 phy_st -2 (UH_ST_TIMEOUT) in 11 of 11 - nothing came back in the response window sop 0 in 11 of 11 - NO START-OF-PACKET EVER ARRIVED rxn 0, 320, 384, 448 - VARIES sop=0 KILLS EVERY CANDIDATE THAT REQUIRES THE DEVICE TO BE TRANSMITTING - CRC or bit corruption, a garbled or truncated packet, a late-but-present response. The device put NOTHING on the wire. What remains is the class where it is not transmitting at all. BOTH HALVES OF 6200's HEADLINE OBSERVATION ARE NOW WITHDRAWN. It read "three physically distinct devices stopping at the SAME BYTE COUNT (rxn=448) with a NAK (hs=$5A), twice, is not three device faults." - NOT A NAK: bulk_in assigns last_hs only on the SUCCESS path, so the byte printed on the failure exit is a leftover from an earlier packet. Confirmed on hardware now, not just by reading - the driver itself reports STALE. - NOT A FIXED BYTE COUNT: today's runs gave 0, 320, 384 and 448. The 448 was one session's coincidence. The finding that survives is real and unchanged: a failing begin2 plus a four-device estate produce transfers where the device stops answering. WHY THIS IS 6150 WORK. That ticket exists to make every run leave comparable evidence behind. The ring was recording a CAUSE FIELD THAT IS STALE ON THE ONLY EXIT THAT MATTERS - it manufactured evidence rather than merely lacking it, and that is what gave 6200 a NAK it never had. Fixed at the source rather than papered over in a harness. usb_client.spin2 - last_hs_valid(), last_phy_status(), last_saw_sop(). The bulk_in phy_buf failure exit and the bulk_out retry branch capture UH_L_STATUS and UH_L_RES bit 0 while the mailbox still holds them. p_saw has been written by the PHY since 0130 (usb_host.spin2:893-897) and setup_stage has surfaced it as $100/$101 since 1190 - the BULK path simply threw it away. Costs NOTHING on the wire: the PHY has already computed it. last_hs is deliberately NOT cleared. usb_class_msc.csw_patient() branches on `last_handshake() <> PID_NAK` to decide whether to keep retrying, so zeroing it would have silently changed a retry path while calling itself an evidence fix. The new flag reports the truth without moving behaviour. usb_evt_ring.spin2 - FL_HS_STALE / FL_PHY_TIMEOUT / FL_PHY_ERR / FL_SAW_SOP, packed into the existing flags word rather than widening put()'s signature, which would have broken top_6150_ring's proof harness for nothing. The dump prints "STALEhs=" instead of "hs=" when the byte is a leftover and decodes the cause in words: TIMEOUT(no SOP at all - DEVICE SILENT) vs TIMEOUT(SOP seen, packet never completed - GARBLED/LATE) vs PHY ERR. VERIFIED: top_6150_ring 17 of 17 (the recorder's own proof harness, unchanged). Rig probe on the no-begin2 build: 4 of 4 devices mounted, 0 failed transactions in 1_973 - the instrumentation costs nothing on a green run. A REAL SCARE, RECORDED BECAUSE THE LESSON IS REUSABLE. The first version of this change BROKE ENUMERATION COMPLETELY - both headers dead, "no device answered (attach timeout)", on a rig that had run all 17 gate suites clean twelve minutes earlier. It looked exactly like the hub or the board had died. IT WAS MINE. A substring replace inserted `last_hs_ok := 1` at FOUR spaces inside a SIX-space block. In Spin2 indentation IS block structure, so that CLOSED THE ENCLOSING BLOCK EARLY and silently rewrote control flow in status_in_zlp(). It compiled without a word. Proven by control rather than assumed: with the change stashed, the committed source enumerated 4 of 4 and mounted 4 of 4 on the same rig in the same minute. That is the standing "don't blame the hardware" rule earning its place - the instinct was that the rig had died, and the rig was fine. CONSTRAINT FOR THE NEXT SESSION: anchor Spin2 edits to WHOLE LINES and preserve the captured indent; a substring replace on an indented statement can silently re-parent everything after it. A mechanical check - every inserted line's indent must equal its anchor's - caught the rest. The corrected pass found NINE assignment sites where the broken one found eight. NEXT, and it follows directly from sop=0: the device is not transmitting. Either it is suspended - which needs >3 ms of SOF-less idle, and usb_host.spin2:1088-1094 documents that exact hazard - or it never saw the token. The SOFEN-left-enabled candidate on 6200 would produce the former and now has a matching observable. The counter-evidence any theory must still explain: recover_bot() returned rst=1 clrO=1 clrI=1 in one run, so the device answered THREE control transfers on ep0 immediately after its bulk IN went silent. NOT RESOLVED. STILL OWED ON 6150 AND NOT CLAIMED: the comparability header (media serial, geometry, clock, driver commit id, workload params - needs a generated build_stamp.spin2, since pnut-ts has no -D NAME=VALUE), deliberate one-axis workload variation via evt_mark(), and long unattended runs with disk_auto_recover on. This commit improves what a record CONTAINS. The soak that drives it is still not built. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BsYQdKe1BGsfm1wqLw4KUo check-in: c150d96f11 user: refaqtor tags: trunk | |
| 02:10 |
6220: the untrusted FSInfo next-free hint escapes its bounds check - fixed, proven fail-first, gate green 346/346 (ticket 1107a9f1cc). Plus two withdrawals on 6200 (293576b2fc).
WHY THIS EXISTS AT ALL: the operator asked whether our wear-levelling is at least as good as the FAT32 driver that inspired this work. Compared against the pinned upstream head (v1.8.0-dev, ca1f519, the clone UPSTREAM-FAT32-BACKPORT.md pins). The POLICY is equivalent and on one axis BETTER - but the comparison turned up a robustness REGRESSION in the very hint that implements it, and that is what this commit fixes. axis upstream ours verdict next-fit, hint never rolled back on delete yes yes equivalent persisted at unmount/idle-flush/sync yes (3 sites) yes (same 3) equivalent hint when FSInfo missing/invalid restarts at deriveNextFreeHint() OURS BETTER cluster 2 backward FAT scan validation of the UNTRUSTED media hint unsigned, bounded signed, unbounded OURS WORSE exFAT is unaffected and untouched: its hint never comes from media and is already clamped (usb_exfat_fs.spin2:1267). THE DEFECT. allocateCluster() validated the media-supplied hint with a SIGNED compare and no upper bound. A hint in [$2000_0000, $7FFF_FFFF] is signed-positive and >= 2, so it passes; `<< 2` then sets bit 31 and the signed `>= fat_limit` guard cannot catch a negative; Spin2's `>>` is LOGICAL (checked against p2kb, not assumed) so the negative becomes a large positive and the allocator PRE-LOADS A SECTOR FAR OUTSIDE THE FAT. If that read errors, noise. If it lands on an LBA that exists, the loop scans arbitrary data AS FAT ENTRIES and can return a cluster number derived from a wild index - a structurally perfect file over the wrong bytes, which is the corruption class this whole mission exists to stop. Upstream fixed this and carries a comment naming the trap; we were the pre-fix version, and it was NOT on the backport worklist - missed, not deferred. THE TEST WAS WRITTEN AND OBSERVED RED BEFORE THE FIX EXISTED. harness/top_6220_fsinfo.spin2 - NO HARDWARE, NO USB TRAFFIC: a complete FAT32 volume built in hub RAM and served through set_block_device() method pointers, so the driver runs unmodified against media whose FSInfo carries a hostile hint. 6190's synthetic-volume vehicle, applied to one defect. case before after A sane hint: no read outside volume PASS PASS A next-fit honoured, allocates AT the hint PASS (20) PASS (20) B trap hint: NO READ OUTSIDE THE VOLUME FAIL PASS B allocated cluster inside the volume FAIL (-1) PASS (11) B rejected hint DERIVES, not cluster 2 (new) PASS (11) C hint absent: no read outside the volume PASS PASS C derived hint resumes past highest in-use PASS (11) PASS (11) 4 of 6 7 of 7 THE RED RUN CONFIRMED THE MECHANISM, NOT JUST THE SYMPTOM: it reported the out-of-range access at LBA 4_194_316 on a 160-sector volume - exactly the address predicted from reading the code BEFORE the test was run ($2000_0000 << 2 = $8000_0000, logical >> 9 = 4_194_304, + fat_sec 12). Predicted and measured agree to the sector. TWO THINGS THE RUN CORRECTED IN MY OWN WRITE-UP, both found by running and neither guessed: 1. The error is NOT E_IO_ERROR as the ticket claimed. do_create() collapses ANY negative from allocateCluster() into E_DISK_FULL (:3878), discarding E_IO_ERROR (-7) - the red run reported DISK FULL on a volume with 120+ free clusters. That masking is a separate diagnosability defect and IS NOT FIXED HERE. 2. Allocation is not lazy on this path: do_create() allocates the first cluster itself (:3875), so createFileNew() trips the defect, before any write. Correction 1 is also WHY the assertions are on the ACCESS rather than the returned status - a status-based test would have been testing the media's response to an out-of-range read, and would still pass if the masking changed. THE FIX IS TWO PARTS AND THE SECOND IS NOT UPSTREAM'S: - allocateCluster(): upstream's guards - `+<` bounding the RAW hint against sec_per_fat << 7, and `fat_idx +>= fat_limit` unsigned. `+<` rejects $FFFF_FFFF for free, so the explicit <> $FFFF_FFFF test is GONE, not merely redundant. - do_mount(): deriveNextFreeHint() now fires for ANY hint allocateCluster() will not use, widened from `== $FFFF_FFFF` which caught "no hint" but not "garbage hint" - so a hostile FSInfo skipped the derive AND was then used unchecked. Written as the EXACT INVERSE of the acceptance test and commented as such: if the two drift, a hint accepted in one place and rejected in the other silently restarts allocation at cluster 2, which is the low-cluster hammering the whole mechanism exists to prevent. Without this half the bounds fix would COST the one axis on which this fork beats upstream. Case B's third assertion is what separates the two: upstream's fix alone allocates cluster 3 there, ours allocates 11. Cluster 3 is a PASS for a plain backport and a FAIL for this fix. Cases A and C pass in BOTH arms by construction - controls on the SHAPE of the fix, worthless as evidence of it - which is exactly why B had to be seen red. GATE: 346 passed, 0 failed, 0 recovery events, 17 suites, RT_DRIVE=A, with the fix in the driver. The baseline is intact. ALSO IN THIS COMMIT - TWO WITHDRAWALS ON 6200, because leaving wrong evidence standing is worse than having none: - "109_591 us and 109_584 us on two devices is a host-side timer" is TRUE BUT EMPTY and stops being a clue: it is BOT_BACKOFF_MS = 100 (usb_class_msc.spin2:50) spent before recover_bot() (:498-501), whose failure returns -6 (:503). Two devices agree to 7 us because waitms(100) is exact. - "hs=$5A - three distinct devices stopping at the same byte count with a NAK" is MISREAD: bulk_in() sets last_hs only on the SUCCESS path (usb_client.spin2:412), and the phy_buf failure exit (:402-407) quits without touching it, so the printed handshake is stale. A real NAK exit is arithmetically impossible there - budget 10000 with waitms(1) per retry needs ~10 s, not 11.251 ms. The real exit was a PHY-level error or silence. This ticket's headline signature has been mis-stated since it was opened. - NEW CANDIDATE, verified by reading: a FAILED begin2 leaves bus-1 SOF generation enabled permanently - bus_reset_now() issues SOFEN(1) on the calling cog's bus (usb_host.spin2:291,447), the failure return never clears it (usb_app.spin2:558), and the only sof_enable(0) for bus 1 is gated `if bus2_up` (:1559) which a failed begin2 never sets. Suggestive - B2X_NO_RESETS is precisely the arm that removes the call that sets it - but NOT SUFFICIENT: additive arms 3 and 4 call the same bus_reset_now(), also leave SOF-B on, and were green. - AND A SUPPORTING ARGUMENT FOR IT, REFUTED BEFORE IT WAS BELIEVED. It was proposed that the ring's frame columns prove both grids were live (18 frames in 11.251 ms being impossible at 1 grid = 1/ms). Measured against a control instead: the FAILING run runs at 2.22 frames/ms and a GREEN -D NO_BUS1 run - where begin2 never runs and bus 1 is never SOF-enabled - at 1.99. The counter does not tick at 1/ms in this driver, so frame arithmetic discriminates nothing. The candidate stands on the code reading alone. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BsYQdKe1BGsfm1wqLw4KUo check-in: 8cade0ac91 user: refaqtor tags: trunk | |
| 01:26 |
6200 round 2 (subtractive): the ESTATE SIZE is a second necessary condition - it invalidated half the round, and it corrects a count in my last commit (ticket 293576b2fc)
THE FINDING THAT DWARFS THE ARMS. THE FAULT DOES NOT REPRODUCE ON A 3-DEVICE ESTATE AT ALL. When the port-4 thumb wedges out of the roster the UNMODIFIED begin2 baseline goes green: 4 runs, 4 of 4 mounted, zero BOT faults, the exFAT wear hint present every time. Seven consecutive 3-device runs across two builds, seven greens. estate arm runs runs with >=1 BOT fault 4 dev begin2 (any variant) 9 8 4 dev no-begin2 / additive arms 14 0 3 dev begin2 baseline 4 0 3 dev B2X_NO_PROBE 3 0 The fault needs BOTH a failing begin2 AND the full four-device estate. Neither alone does it. HOW IT BIT ME, WRITTEN DOWN SO IT IS NOT REPEATED. Round 2's arms ran in sequence. B2X_NO_VBUS cascaded (3 and 4 faults, 1 of 4 mounted, both runs) and WEDGED THE PORT-4 THUMB OUT OF THE ROSTER partway through. Every arm after that ran against 3 devices - a condition under which nothing faults - so their greens measured the estate, not the arm. I read three B2X_NO_PROBE greens as an arm result. They are not one. AND THE RUN THAT SETTLED IT WAS ONE I NEARLY THREW AWAY. Log -211559 is a B2X_NO_PROBE run on a still-intact 4-device estate, killed mid-run by a harness timeout, so it has no summary line and no verdict. It contains the full cascade: op=$28 hs=$5A rxn=0 on the SSD, then rxn=448 on the next thumb. B2X_NO_PROBE IS RED ON FOUR DEVICES; removing the address-0 probing does not stop it. A truncated log is still evidence. ROUND 2, SCORED ON MATCHED 4-DEVICE ESTATES ONLY: arm (element removed from the reproducing ladder) runs red BASELINE nothing removed 2 1 B2X_NO_VBUS VBUS pin never driven, recycle time spent 2 2 full cascade both B2X_NO_INIT host.start_bus() PHY init skipped 2 2 fault, no cascade B2X_NO_PROBE probe_addr0() never called 1 1 the -211559 run B2X_NO_RESETS no bus_reset_now() in the bus-1 ladder 2 0 B2X_NO_RESETS is the ONLY arm green on four devices - and that is a candidate, not a result. Two runs against a baseline that is itself only ~50% red per run at this estate size is not evidence; it must be re-run many more times with the BASELINE INTERLEAVED rather than run in a block, so estate drift cannot masquerade as an arm effect a second time. TIMING HELD CONSTANT IN EVERY ARM, because round 1 already produced one wrong conclusion from an under-timed control: B2X_NO_VBUS still spends VBUS_RECYCLE_MS, B2X_NO_RESETS still spends RESET_RECOVERY_MS after each gated-out reset, B2X_NO_PROBE still paces at PROBE_INTERVAL_MS. Every gate touches begin2_pins() ALONE - a gate that quietly changed bus 0's ladder would produce a green that means nothing - and every build announces which arm it is in the log, because two logs differing only in a compiler flag are indistinguishable afterwards. CORRECTION TO MY LAST COMMIT. It claims "Zero failed transactions across five consecutive no-begin2 runs, ~9_800 recorded transactions". FOUR OF THOSE FIVE WERE 3-DEVICE RUNS (-193138, -193153, -193314, -193337), where nothing faults regardless; only -192444 was a 4-device no-begin2 run. The begin2 separator SURVIVES and is better supported than that sentence claimed, but by different runs: after the reseat restored the estate, FOURTEEN 4-device runs without a full begin2 were all green against 8 of 9 red for begin2 on the same estate. The claim was right; the evidence I cited for it was not the evidence that supports it. WHAT "FOUR DEVICES" IS NOT. It is not a mechanism, it is a clue about one, and it is NOT guessed here. It could be the count of bound devices, the specific port-4 device, cumulative enumeration traffic, the number of hub ports in use, or an address-range effect (the fourth leaf takes addr 5). The cheap discriminator after a reseat is to run the baseline with that thumb on a DIFFERENT hub port, and separately with a different fourth device - which separates "that device" from "four devices". A NEW STANDING PRECONDITION FOR ALL 6200 WORK: CHECK THE ROSTER IS 4 BEFORE SCORING ANY ARM. With 3 devices every arm is green and the round measures nothing. RIG: the port-4 thumb (...809328, the alignment-5 drive 6130 needs) is wedged again - port 4 reads $0103, connected AND enabled, so device-wedged class, physical reseat required. The bisection cannot continue until it returns. NOTHING WAS WRITTEN TO ANY DEVICE in any arm; the probe's read-only contract holds in all five builds. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BsYQdKe1BGsfm1wqLw4KUo check-in: 6bd349dd1e user: refaqtor tags: trunk | |
| 00:56 |
6200 carrier bisection round 1: five candidates dead, none of begin2's side effects reproduces it alone - and a correction to my own control (ticket 293576b2fc)
A NEGATIVE RESULT, COMMITTED AS ONE. Twelve arm runs, zero BOT faults; seven real-begin2 runs, a fault in every one. The carrier is NOT any single side effect of begin2 in isolation and NOT elapsed time. Five dead candidates is what stops the next context re-deriving them, which is the whole reason this lands rather than being left in a transcript. FIRST, THE CORRECTION TO MY OWN CONTROL - it is the most important thing in this commit. The previous commit claimed "elapsed time is EXONERATED by the third arm: idling the same 8 s reproduces nothing". THE 8 s WAS WRONG. A failing begin2 on this rig costs 13.6 s: banner 20:40:43.042 -> begin2 result 20:40:57.589 = 14.547 s (log -204041), minus the 0.949 s begin(BASE0) itself takes (log -203422). The idle control was 5.6 SECONDS SHORT of the thing it was built to match - it would have let elapsed time look exonerated on an arm that never spent the elapsed time. BUS1_COST_MS corrected to the measured 13_600 and the arm RE-RUN three times: 4 of 4 devices mounted every run, zero failed transactions in 5_919 recorded. The conclusion survives - elapsed time IS exonerated - but it now rests on an arm that actually spends the time. The constant carries its measurement AND the size of the original error in a comment, so the mistake cannot silently come back. THE ARMS, BUILT UP FROM GREEN RATHER THAN DOWN FROM RED. Subtracting from begin2 changes the meaning of what is left - a bus reset posted to a pair that was never INITed is not the same event as one posted to a pair that was - so a subtractive arm cannot attribute cleanly. Adding one effect at a time to a run known green can. arm runs BOT faults
Arm 4 exists because arms 1-3 being green ALONE does not license assuming their combination is green. Arm 5 exists because arms 1-4 only configure pins and post resets, while the one part of begin2 that puts TRAFFIC on the wire is its address-0 probing loop - and it logs "x_bus2_probe answered=0 of 80", confirming the probing really ran and the empty header really answered nothing, rather than leaving a silent arm to be mistaken for a tested one. A SIXTH CANDIDATE DIED BY READING, NOT BY RUNNING. begin2_pins() leaves dm2_pin and vbus2_pin set after it fails, which looked like it might hand the later storage workload bus-1 activity through the shared PHY. It cannot: bus2_up stays 0 on the failure path and every later bus-1 scan gates on bus2_up (usb_app 628, 914, 1350, 1369, 2690). Cost nothing to kill; recorded so nobody pays for it twice. THE EXPERIMENT HOOKS ARE NOT PRODUCTION API AND SAY SO. x_bus2_init / x_bus2_reset / x_bus2_probe / x_bus_selected sit behind -D USB_BUS2_EXPERIMENT and do not compile without it, so the shipped object surface is unchanged. They exist only to let a harness perform ONE of begin2_pins()'s side effects in isolation. x_bus_selected() is there to CHECK begin2's single restoration claim rather than trust it - every arm reads it back and every arm reports 0, as claimed. THE HONEST LIMIT OF THIS ROUND. The endpoint scored is ">=1 BOT data fail in the run", because that is what begin2 produces RELIABLY (7 of 7). The full cascade - SSD faults, then thumbs fail to mount - is intermittent (4 of 7) and is a weaker endpoint, so the arms were not scored on it. Arms 1-3 have ONE run each; against a 7-of-7 red arm a single green run has real but limited power and they must be repeated if anything comes to hang on them individually. Arms 4 and 5 have three each. NEXT ROUND IS SUBTRACTIVE, and the switch is justified only because additive has now been exhausted without reproducing anything - there is nothing to attribute. Gating out one element at a time inside begin2_pins() starting from the sequence that DOES reproduce names the carrier IN CONTEXT, which is weaker than an additive result but is what remains. RIG: the port-4 thumb (...809328, the alignment-5 drive 6130 needs) was reseated by the operator and came back at a new address, matched by serial. Fresh green baseline before the bisection, log -203422: 4 of 4 mounted, 0 failed in 1_973 transactions, exFAT wear hint present with the full 4.8 s bitmap scan completing. Every arm in this commit ran against 4 devices. NOTHING WAS WRITTEN TO ANY DEVICE in any arm; the probe's read-only contract holds in all eight builds. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BsYQdKe1BGsfm1wqLw4KUo check-in: f51e385cb8 user: refaqtor tags: trunk | |
|
2026-08-25
| ||
| 23:37 |
6200: the separator is a failing begin2() on the EMPTY second header, not recover_bot() - isolated with a three-arm control, and the ring pointed at the fault for the first time (new ticket e480049b6d / 6210)
THE SESSION GATE FAILED AGAIN, IDENTICALLY, AND THAT IS WHY IT IS A GATE. top_rig_probe at 19:12 and again at 19:13: SSD mounts, all three FAT32 thumbs fail with BOT data fail op=$28 hs=$5A rxn=448 -> fs_mount st=-7. Per the procedure written into PROMPT-NEXT-SESSION.md, that is NOT evidence of bad media: re-run, then run top_6150_order before concluding anything about the hardware. Both builds of the isolator: 12 of 12 thumb mounts, correct $55AA, real labels. THIRTEEN clean isolator runs against SIX failing probe runs on the same drives minutes apart. THE IDLE HYPOTHESIS ON 6200 IS REFUTED. It predicted the failure was a wake-from-idle behaviour - first activity after the rig sat idle. Today's first run followed ~20 h idle and failed; the second ran 50 seconds later and failed identically; so did a third. Recorded on the ticket as refuted rather than quietly dropped. WHAT SEPARATES THE PROBE FROM THE ISOLATOR. Exactly one structural thing: the probe calls begin2(BASE1) on the EMPTY second header before binding anything. It fails there after ~8 s, and begin2_pins()'s failure path restores host.set_bus(0) but does NOT undo the rest - bus 1's pins stay configured as a USB pair, VBUS on pin 21 stays driven, and several bus_reset_now() plus a VBUS recycle have been posted, on a PHY cog whose own header says BOTH CHANNELS SHARE IT. THREE ARMS, BECAUSE TWO CANNOT ATTRIBUTE IT. begin2 costs its side effects AND ~8 s of elapsed time at once, and 6200 already carried an idle-time hypothesis, so a two-arm test proves nothing about which. Added as compile gates to top_rig_probe: -D NO_BUS1 skips begin2; -D NO_BUS1_WAIT skips it and idles the same measured 8 s. arm runs BOT transaction failures begin2 (as-is) 6 >=1 in EVERY run -D NO_BUS1 3 0 -D NO_BUS1_WAIT 2 0 Zero failed transactions across five consecutive no-begin2 runs, ~9_800 recorded transactions. The third arm EXONERATES elapsed time: idling the same 8 s reproduces nothing. AND THE GREEN ARM DID MORE WORK, NOT LESS. This is the control that makes the result mean something: a no-begin2 run records MORE transactions (1_959-1_973) than a failing one, because the exFAT mount COMPLETES its bitmap scan instead of aborting partway. A green arm that got there by doing less would be worthless. THE RING, POINTED AT THIS FOR THE FIRST TIME (harness/logs/headless_260825-192134.log). top_rig_probe now arms the 6150 ring across the WHOLE per-device loop and dumps once at the first failure. Armed across the loop, NOT per device, because arming clears the ring and the entire finding is that fault and damage sit on different devices - a dump starting at the failing device has already discarded the transaction under suspicion. Every evt_* call is an empty method without -D USB_EVTRING, so the gate build is unchanged and the probe stays read-only in both. A green run dumps at the end too: "what normal looked like" is the other half of the evidence. THREE THINGS IT SHOWED, NONE OF THEM GUESSED: 1. THE NEXT DEVICE IS NOT POISONED AT BIND. After the SSD's fault, the thumb's INQUIRY, TEST UNIT READY, READ CAPACITY and its FIRST READ(10) at LBA 0 all SUCCEED; it fails on the SECOND read (lba=32). This is why top_6150_order could never reproduce it - its arm2 tests exactly READ(10) LBA 0 plus fs_mount, which is the one transaction that still works. The isolator was measuring the wrong side of the boundary, and that is now written on the ticket so the next context does not trust it as a negative. 2. 109_591 us AND 109_584 us, ON TWO DIFFERENT DEVICES. The give-up retry takes the same 109.6 ms on the SSD and on the thumb, matching to 7 us out of 110 ms. Two devices do not agree to that precision; that is a host-side timer. 3. fs_mount RETURNS st=0 OVER A FAILED BITMAP SCAN -> NEW TICKET e480049b6d (6210). The ring stamps a successful mount in the same millisecond as two failed reads. A completed scan emits "exfat: wear hint - highest in-use clus" and takes ~4.8 s; an aborted one emits nothing and returns in ~1.7 s, and both return st=0. Present in every clean run, absent from every failing one. This is what made the SSD look healthy in 6200's earlier analysis, which recorded "the SSD recovers fine" - it did not, it reported success on a scan it never finished. Filed separately because fixing it would not fix 6200, and because a next-fit hint derived from half a bitmap can hand out a cluster that is in use: the same shape as the corruption this whole mission exists to stop, a perfect-looking structure over wrong data. WHAT THIS DOES NOT SETTLE, AND MUST NOT BE GUESSED. The CARRIER is undetermined. begin2's side effects are the SEPARATOR, established by controls; WHICH of them - pins left configured, VBUS left driven, the posted resets, shared-PHY servicing - is unknown. Four obvious mechanisms have already been wrong on this project and a fifth guess is not wanted. The next arm is one variable at a time inside begin2's failure path. It also WEAKENS WITHOUT KILLING the recover_bot()-as-carrier reading: in one run today the SSD's recovery SUCCEEDED (rst=1 clrO=1 clrI=1) and the thumbs still failed; in another it FAILED (rst=0 clrO=0 clrI=0) and they still failed. Both shapes sit downstream of the same begin2. RIG DAMAGE, RECORDED RATHER THAN GLOSSED. The port-4 thumb (serial ...809328) - the ALIGNMENT-5 drive 6130 needs and which has never been run - stopped enumerating after the 19:29 begin2 run and has not returned across four subsequent runs. Hub port 4 still reports $0103, CONNECTED AND ENABLED, so by the discriminator this is the device-wedged class and needs a physical hub reseat, not software. The last two arms therefore ran against 3 devices; their zero-failure result stands on its own transactions, but the estate is down one drive. The arm that reproduces the fault is also the arm that destroys media availability, and that cost belongs in the record. RING WRAP IS A LIVE LIMIT: 256 slots against ~1_950 transactions discards ~1_700 records. The dump announces it rather than hiding it, and the failure window is captured, but the start of the bitmap scan is gone. If the carrier hunt needs the whole scan, the ring has to grow. NOTHING WAS WRITTEN TO ANY DEVICE. Every mount released with fs_abandon(); the probe's read-only contract is intact in all three builds. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BsYQdKe1BGsfm1wqLw4KUo check-in: 2baf1b9376 user: refaqtor tags: trunk | |
| 23:06 |
Restart prompt rewritten around the filesystem-test mission, and the regression gate committed as a script
TWO THINGS, both aimed at the next context starting productively instead of re-deriving what this one learned. 1. tests/regression/run_gate.sh - THE GATE, AS A COMMITTED SCRIPT. It existed only as throwaway shell in a session scratchpad, which meant the next session would rebuild it and probably rebuild it differently. Now: - it parses each suite's OWN 'Build: pnut-ts' header line rather than assuming a shared one. The suites genuinely differ (SD_INCLUDE_RAW/_DEBUG/ _DEFRAG), and a shared line builds the wrong thing while still producing a green-looking number - the worst possible failure mode for a gate; - it COUNTS RECOVERY EVENTS and fails the run if any occurred, because a suite that passed after a BOT recovery is not the same result as one that never needed it, and that difference is the signal 6100/6200 are chasing; - a suite that produces no result line is reported as INCOMPLETE rather than silently contributing 0/0, so the total can never look green by omission. Verified on the error path: an unknown suite name yields "the number above is INCOMPLETE", not a clean zero. - RT_DRIVE=A|B|C pins every suite to one thumb by serial, which is what 6130's geometry matrix needs. Exit 0 only on 346 passed / 0 failed / 0 recoveries. 2. PROMPT-NEXT-SESSION.md REWRITTEN around the mission rather than around yesterday's incident list. Structure is now the order a fresh context should actually work in: read prompt.md, load p2kb, confirm the rig, understand what 346/346 does and does not cover, see WHAT TOOLS ALREADY EXIST (a new table - the ring, its proof harness, the order isolator, wedgewatch, the 0186 synthetic rig, the fixture's per-drive pinning, the gate script, the recovery guide), then the work. Substantive changes beyond reorganisation: - The rig-confirmation step now carries the 6200 warning IN LINE: a mount failure in top_rig_probe is NOT evidence of bad media, re-run it, and run top_6150_order before blaming hardware. This session opened with exactly that failure and it read precisely like three dead drives. - "Start here" is now 6150's SOAK, with the three owed pieces named concretely - the comparability header (including the note that the driver commit id needs a generated build_stamp.spin2, since pnut-ts has no -D NAME=VALUE), the workload axes to vary one at a time via evt_mark(), and long unattended runs with disk_auto_recover on. The recorder is described as landed and proven; nothing about the soak is claimed. - 6200 gets a full section with its exoneration set and its two DEAD mechanism candidates spelled out, so the next context cannot spend time re-deriving them. - 6100's "next step" corrected to say explicitly that 6150's ring records TRANSACTIONS and cannot see inside one - the PHY-level per-packet ring is the layer below it and is still unbuilt. Also updates the count: FOUR obvious mechanisms have now been wrong, not three. - Standing rules section now cites top_6150_ring as the worked example of fail-first (15 of 17 assertions OBSERVED going red), so the rule has a concrete model in-tree rather than only a cautionary tale. - Constraints grown and regrouped into binding/media, pnut-ts/Spin2, and running-things: the `field` reserved word, parameter-vs-VAR name collisions (both give the same m220 with no hint which), `...` not `_` for line continuation, uhex_*_() emitting their own `$`, and pnut-ts -o disliking an absolute path. - Baseline section states 346/346 was RE-ESTABLISHED with the ring wrapper in the BOT path and points at run_gate.sh to reproduce it. Claim discipline: 6000's children are ten (6100-6190); 6200 and 6080 are open defects listed alongside them, not children, and the text now says so rather than rounding up to "eleven children". Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DJDNhzBk5mbbUEjiC13MXq check-in: 3b82ec395f user: refaqtor tags: trunk | |
| 22:58 |
Make PROMPT-NEXT-SESSION.md agree with fossil after the 6150/6200 session
Per prompt.md: this file is a MAP subordinate to the tickets and commit messages, and every claim of done/closed/verified in it must be traceable to one. Bringing it back into agreement. - "Start here" no longer says to land 6150 first: its RECORDER is landed (src/usb_evt_ring.spin2, proven fail-first, 346/346 re-run clean). What is still owed on that ticket is named explicitly and NOT claimed done - the comparability header, workload-axis variation, and long unattended runs. The ordering rule now reads "build the soak next, then run every ladder through it", which is the same rule with the first half discharged. - 6200 (293576b2fc) added to the ticket table and given its own section: the controlled isolation, the exoneration set (media, bind ordering, exFAT auto-detect, SSD root enumeration), the one separator (a successful recover_bot() on the SSD, present in both failing runs and neither of the nine passing ones), and the two mechanism candidates that DIED in the checking so nobody re-derives them. The idle hypothesis is recorded as a hypothesis with its test attached, not as a finding. - New section: the rig probe can now fail spuriously, and a thumb mount failure is NOT by itself evidence of bad media. Re-run it, then run top_6150_order before concluding anything about the drives. Written because the session gate failing was this session's first event and it read exactly like dead media. - Constraints grown with what cost build cycles today: `field` is a Spin2 RESERVED WORD (FIELD pointers); a parameter may not share a name with an object-level VAR (evt_mark(tag,...) vs usb_class_msc's `long tag`) - both give the same m220 message with no hint which cause it is; and Spin2's line continuation is `...`, not `_` (checked against p2kb rather than guessed). - Baseline section notes 346/346 was RE-ESTABLISHED with the ring wrapper in the BOT path, rather than leaving the reader to assume the old run still covers a changed driver. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DJDNhzBk5mbbUEjiC13MXq check-in: 5494447430 user: refaqtor tags: trunk | |