Many hyperlinks are disabled.
Use anonymous login
to enable hyperlinks.
50 most recent check-ins
|
2026-10-06
| ||
| 19:22 |
Merge size-switch-probe into trunk: 6290 FAT32 size gates
The SD SPI engine stays out of the default USB image. FAT32_NO_LFN is the 8.3 opt-out; long names stay on. Measurements, the README footprint and build-switch tables, and the pin-16 test results are in 75c768da5c. leaf check-in: c6c421aec5 user: refaqtor tags: trunk | |
| 19:10 |
6290: FAT32 size gates — SPI engine off by default, FAT32_NO_LFN opt-out
The SD SPI engine is not on the USB mount path (initCard is a block-device stub). It compiles only when SD_INCLUDE_RAW, SD_INCLUDE_REGISTERS, SD_INCLUDE_SPEED, SD_INCLUDE_DEBUG, or SD_CARD_ENGINE is set. Default MSC+FAT32 is 48,632 / 13,376 / 62,008 (code+data / VAR / total), 3,188 bytes under the engine-on image. FAT32_NO_LFN is an opt-out. Long names stay the default. With the switch set, names are 8.3 only: create or rename of a longer name is E_BAD_NAME, listings return the alias, and delete of an alias leaves the LFN run for fsck.fat. BPB_RootClus, the '..' cluster, and the no-extension parse stay. That cuts a further 2,280 bytes. The two cuts stack. 6280's bug fixes stay in both builds. csw_patient is untouched. pnut-ts v1.55.0 on harness/top_footprint.spin2. The everything row is 74,412 / 25,644 / 100,056. Product matrix (run_fs_gate.sh --build-only): 65,479 exFAT (unchanged from the 6280 gate), 74,878 FAT32, 89,687 both, 72,598 FAT32 with FAT32_NO_LFN. Tests on P81svj44, serial host base pin 16, thumb 2603121734073281946847: run_gate 346/346, 0 recoveries. Facade FAT32 column 223/223. XS_malformed 25/25 and XS_exfat_struct 28/28. top_lfnread default, RELOC, WRITE, FAT32_NO_LFN, and NOLFN+RELOC passed, the write image judged by fsck.fat and GNU mtools. The exFAT stick, multi-drive, and hot-plug columns were not run; this header had the one thumb, not the four-drive estate. Branch size-switch-probe. Not merged. closed check-in: 75c768da5c user: refaqtor tags: size-switch-probe | |
| 14:48 | Create new branch named "size-switch-probe" check-in: 777fa1d469 user: refaqtor tags: size-switch-probe | |
| 01:22 | updated the memory footprint table on the readme after updates to filesystem drivers check-in: 0cd4a191ae user: refaqtor tags: trunk | |
|
2026-09-26
| ||
| 02:21 |
6280: FAT32 long file names (VFAT), BPB_RootClus, correct '..', graded CSW pacing - compatibility proven by host reference tools
Answer to the operator: the FAT32 driver WAS 8.3-only, by construction of the adapted driver (search folded names to 11-byte 8.3, LFN entries were skipped, create wrote one 8.3 entry). It was not a FAT32 limit. usb_fat32_fs: - VFAT LFN, up to 64 ASCII chars per component (exFAT parity). Valid 8.3 names are stored exactly as before. Other legal names get LFN entries plus a unique Windows-style ~N alias. Lookup matches the long name or the alias, ASCII case-insensitive. Listing and fileName() return long names. Create/mkdir place N+1 consecutive slots (growing the directory if needed); delete retires the run; rename relocates (new entry first, open handles re-pointed); move keeps the destination alias. Illegal/over-long -> E_BAD_NAME -48 -> facade -95. - Honours BPB_RootClus (root_clus/rootdir_sec). The old driver showed an EMPTY root on a legal relocated-root volume. - mkdir's '..' = the parent's FIRST cluster (fsck.fat flagged the old value whenever the parent spanned clusters, including a grown root). usb_class_msc: csw_patient polls a NAKed CSW 50 us -> 1 ms -> 10 ms -> 100 ms, same 15 s ceiling (was a flat 100 ms first sleep). usb_client: bulk NAK loops use an explicit counter; p2kb calls modifying a FROM..TO variable undefined. Same behaviour. p2kb audit: LFN UTF-16 fields read/written byte-wise (p2kb: word access must be word-aligned; the fields sit at odd offsets). No chained := or ?: in new code. COMPATIBILITY (operator: confirm compat before altering), judged by fsck.fat 4.2 + GNU mtools 4.0.43, never by this driver: - harness/top_fatcompat.spin2 + tools/fatcompat_check.py: the P2 writes to a real PC-formatted stick, the host rebuilds a sparse image, and fsck/mdir/mcopy judge it byte-exact. Baseline (committed driver) COMPATIBLE BEFORE any change (headless_260925-194443); LFN phase COMPATIBLE (-204447). The checker is fail-first: fsck -n exits 0 on a corrupted LFN checksum, so every output line must be whitelisted; the corrupted image exits 1. - tools/make_lfn_images.py + harness/top_lfnread.spin2: volumes written by mkfs.fat+mtools, incl. a relocated root, read by long name (-204745, -204804). The committed driver fails them (8 and 12 failures). -D WRITE: directory growth past 2 clusters at spc=1, late subdir, delete, rename, fsck-clean (-204723). The old '..' -> fsck 'Invalid .. entry' x2. Tests: XS_malformed 23 -> 25 (#23 exact on-disk 8.3 bytes for names WITHOUT extension, RED against the loop bug below; #24 LFN ordinals/attr/checksum vs an independent sum). XM_EXP 25. The gate caught my bug before this commit: an 8.3 name with no extension ran a `0 to -1` loop, which reverses in Spin2 ("CWDTEST1" -> "CWDTEST.C"). Fixed, all new loops audited, and drive A's single mangled leftover removed. Gates, final code: run_gate 346/346 0 recoveries (tests/regression/logs 213533..215851); run_fs_gate --fat32 223/223 + matrix (65479/78046/92855). top_push @300 unchanged (reads 1_063-1_068, headless_260925-221756). Open, recorded on the ticket: the bus-0 stick twice enumerated as VID $1E3D at a suite start during earlier gates; 28 alternated A/B runs (new vs committed) reproduced it in neither build; not seen in the final gate. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> check-in: 5767c3cfac user: refaqtor tags: trunk | |
|
2026-09-24
| ||
| 16:49 |
6270: lowest-free cluster reuse replaces next-fit; SCSI UNMAP (TRIM) where the device advertises provisioning; README rewritten
Operator decision on 6260: rewrite the README, reuse free clusters, add UNMAP where supported. Allocation (usb_fat32_fs, usb_exfat_fs): contiguity-first (a growing chain takes cluster+1 when free), else the LOWEST free cluster from a hint meaning "no free cluster below this". Every free lowers the hint; an allocation raises it only when it consumes it. It starts at cluster 2 on every mount. FAT32 never trusts FSInfo Nxt_Free as a start (others store "last allocated" there), and deriveNextFreeHint() is gone. exFAT drops the bmp_hint_init() mount scan and gains bmp_find_free(), which searches the bitmap a LONG at a time. The 6210 degraded signals have no scan left to abort and read 0 (kept for API compatibility). UNMAP (usb_class_msc, usb_app): lbp_probe() at every disk_open sends NOTHING below INQUIRY version $05 (SPC-3); otherwise EVPD $00 -> $B2 LBPU AND READ CAPACITY(16) LBPME (the Linux sd rule), plus $B0 max count. unmap() sends one-descriptor UNMAP; advisory, counted. New: disk_discard / disk_lbp / disk_lbp_probe_cmds / disk_discard_stats / fs_discard_stats. The FS drivers take set_discard(mp) and discard freed contiguous runs (max 16 per chain) only AFTER the metadata is on the media: FAT32 after freeClusterChain's final FAT1+FAT2 write, exFAT after the entry-set clear. Tests, fail-first each: XS_exfat_struct 27 -> 28 (reuse cluster 5, discard LBA/count/ordering); XS_malformed 20 -> 23 (6210 fault cases now assert mount never reads the faulted sector; new FAT32 reuse + discard-after-FAT group). The same tests against the committed drivers: 18/23 with 5 RED; ordering variants V2/V5 RED. top_6220_fsinfo updated (lowest free = 3 on its volume; committed drivers RED at 20/11/11). top_wearpolicy marked SUPERSEDED (its 64 GB card is not on the bench). New harness/top_trim.spin2 10/10 on the rig: CDB and parameter bytes match SBC-3; both SCSI-2 thumbs get 0 probe commands and 0 UNMAPs; 3 deletes -> 3 runs / 384 sectors requested. Gates, drive A: run_gate.sh 346/346, 0 recoveries; run_fs_gate --fat32 223/223 + build matrix (65431/74910/89723). top_push @300 unchanged: reads 1_066, 32 KB-call writes 925-932, all verified. NOT proven: an UNMAP accepted by a real device (needs an SPC-3+ device with provisioning; the ASMedia bridge is the candidate). Docs: README "Flash wear and TRIM" replaces "Flash wear policy", throughput table updated to the 6250 measurements; storage guide; API manual rows, constants ledger, v2.9 history. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> check-in: 0af506c267 user: refaqtor tags: trunk | |
| 01:38 |
6250 final push evidence @300/@200 MHz from clean b2c0c61b25; post-probe rig check
headless_260923-212705 (300 MHz) and -213114 (200 MHz): top_push from the clean commit, both thumbs, all verified, zero err-deltas. Rig probe after the read-only UNMAP capability probe (6260): both drives enumerate and mount, nothing written. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> check-in: 632ff11555 user: refaqtor tags: trunk | |
| 01:25 |
6250 step 2: deferred FAT, NAK sampling, dual-bus SOF phase lock. FAT32 file I/O at the wire's practical ceiling
usb_fat32_fs: - DEFERRED FAT: allocations made inside CMD_WRITE_H stay in the cached FAT sector (fat_dirty). flushFat() writes FAT1+FAT2 before the sector is evicted, before any other command (dispatch_cmd), before do_sync_h publishes a directory entry, and on the 200 ms idle flush. Media order is data -> FAT -> entry, so a lost deferral leaves unreferenced data, never an allocated orphan. Entry-publishing commands keep immediate FAT writes (fat_defer). Scratch users of fat_buf flush first. A lost tag is refused, never written to a guessed sector. - HF_META: a bulk write leaves the handle clean, so the idle flush also syncs handles owing a directory entry (closes the gap noted in bb491b2973). usb_client: the first NAK_FAST_TRIES (40) NAKs of a bulk transfer are re-polled every NAK_FAST_US (25 us) WITHOUT spending the try budget, and patience is otherwise unchanged. The 1 ms sleep after a thumb's first data-stage NAK doubled a 512 B read (profiled: data stage 1_496 vs 573-791 us). usb_host: bus B's SOF deadline is phase-locked to bus A's (was now + 1 ms). The packet/command guards then overlap instead of blocking two windows per ms. 1-for-2 instruction swap; PHY cog RAM is full. TX canary PASS at 300 MHz (headless_260923-204248) and 200 MHz (-204349). Results @300 MHz, two $ABCD thumbs, one per Serial Host bus, content-verified (headless_260923-203245 + SOF-lock run): FS read 32 KB calls 255 -> 1_066 kB/s; FS write 32 KB calls 324 -> 905-924 kB/s; 512 B reads 255 -> 511; dual-bus raw read 1_004 -> 1_065. Gates, drive A: run_gate.sh 346/346 0 recoveries (logs 204536..211501); run_fs_gate.sh --fat32 (FSDRV=ANY) 223/223 + build matrix (64288/73634/88188). exFAT column NOT run: no exFAT media on this rig. Caveat (in code + ticket): with ISO audio on bus A, bus B's SOF now systematically follows A's ISO slot. Unmeasured; no audio device on the rig. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> check-in: b2c0c61b25 user: refaqtor tags: trunk | |
| 00:13 |
6240 FIXED + 6250 step 1: cross-bus second drive slot; FAT32 file I/O in multi-block runs
6240 (9fa3a1ee2a): usb_app disk_read_b/disk_write_b never re-aimed the one shared msc instance, so with one thumb per Serial Host channel every slot-B CBW went out on the primary's pin pair to an address that does not exist there (40 s silence, failed BOT recovery). New PRI b_route() aims msc at busof(b_addr) for the slot-B op and restores the primary's bus, under the held MSC lock. Fail-first read-only red headless_260923-192301, green -192521. 6250 (976fcc2ef1), usb_fat32_fs: - do_read_h BULK RUN: whole sectors from an aligned position go straight into the caller's buffer as ONE multi-block read, extended across physically contiguous clusters. Handle is left on the run's last sector with a clean copy, exactly the per-sector loop's end state. - do_write_h BULK RUN: whole sectors to the end of the current cluster go out as ONE multi-block write from the caller's buffer, with no load because every byte is overwritten. The existing boundary/allocation/B7 code runs unchanged. - allocateCluster SAME-SECTOR LINK: when the predecessor's cell is in the loaded FAT sector, link and new EOC go out in ONE FAT1+FAT2 pair instead of 2 pairs + a re-read; the buffer stays cached. Same bytes on disk, atomic per sector. Measured @300 MHz, 4 MB file, 32 KB calls, both thumbs, content-verified: read 255 -> 973 kB/s, write 324 -> 581 kB/s. Gate: run_gate.sh 346/346, 0 recoveries (drive A), logs tests/regression/logs/headless_260923-194520..200832. harness/top_push.spin2: two-thumb/two-bus push (raw cmd-size sweep, dual interleave, FS phases, sweep, -D CLK300, -D NO_BUS1). Uses if/else, not ?:, for side effects: Spin2 ?: evaluates both operands (pnut-ts -l: call A, call B, $6B select). run_gate.sh: grep -a. Logs now carry the download NUL burst, and plain grep printed 'binary file matches', reporting 17 green suites as NO RESULT. bench-current.md: P81svj44 rig is now hub-less, one new $ABCD thumb per bus; 300 MHz TX canary passed. KNOWN GAP (fixed in the next commit, not here): the bulk write leaves the handle buffer CLEAN, and do_idle_flush syncs only HF_DIRTY handles, so after an aligned bulk write the directory entry's size waits for close/sync instead of the 200 ms idle flush. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> check-in: bb491b2973 user: refaqtor tags: trunk | |
|
2026-09-20
| ||
| 18:32 | 2100 EXPLORATION, BRANCH ONLY - TRUNK UNTOUCHED: block-device contract (readblocks/writeblocks/ioctl) - analysis document + COMPILE-ONLY prototype (ticket a57f82fc0c). docs/design/block-device-seam.md: 27 sections - the storage stack and why it is cut the way it is, the nine invariants the facade enforces by construction, whether the field request makes sense over USB, four staged changes with code, improvements (write barrier, test tap, other filesystems, upstream convergence), four by-reading findings. STAGE 1 BUILT: OPTIONAL third pointer set_block_ctl() on both handlers - geometry guard BEFORE the first read, capacity refresh from the device, write barrier before exFAT clears VolumeDirty / after FAT32 FSInfo; usb_app.disk_ioctl()/disk_ioctl_b(); usb_class_msc read10+write10 merged into one private rw10() (both public names kept) + sync_cache(). SYNCHRONIZE CACHE is compiled out unless -D USB_MSC_SYNC: NO NEW WIRE TRAFFIC BY DEFAULT. STAGE 3 BUILT: usb_app.fs_mount_on(rd,wr,ctl) and the five MSC-wired sites it exposed (build guard now needs USB_MSC or USB_BLK_EXT and still fires with neither; ensure_fs; unbind_addr and do_drive_select spare a foreign mount; second-slot methods nested under USB_MSC). New: src/usb_blk.spin2 (contract constants), src/blk_ram.spin2, src/blk_part.spin2, src/blk_tap.spin2, harness/top_blk_mix.spin2 (both handlers on RAM, no USB object in the build - also builds against trunk handlers), harness/top_blk_facade.spin2 (fs_* facade with NO MSC class), harness/verify_blk_seam.sh. Evidence: harness/logs/verify_blk_seam_result.txt - 13 existing consumers + 6 demo builds as expected, guard fires; product matrix 64540/73226/87908 vs trunk 64192/72886/87440 (+348/+340/+468). NOTHING WAS RUN ON HARDWARE AND NOTHING IS VERIFIED - the document names the fail-first tests. Stages 2 and 4 are design only. leaf check-in: b8adc31d82 user: refaqtor tags: blk-seam-exploration | |
|
2026-08-28
| ||
| 18:59 |
6210 CLOSED: degraded-mount signal on both filesystems + facade fs_mount_degraded()
exFAT wear_deg (bmp_hint_init abort now loud + flagged, wearScanDegraded()), FAT32 hint_deg (deriveNextFreeHint abort flagged, nextFreeHintDegraded()), facade fs_mount_degraded() routing by fs_fmt. Mount's return-0 tolerance unchanged - it is the designed trade-off; only the missing signal was added. XS_malformed grew the 6210 group (16->20, gate EXPECT bumped): injected read fault at the bitmap/FAT sector vs controls differing only in the fault. Fail-first: flags disabled -> tests 17/19 RED (18/20), reverted -> 20/20. Facade smoke on hardware green both filesystems; build matrix green. check-in: 74010fe781 user: refaqtor tags: trunk | |
| 17:33 | docs: 8c3bd43e67 resolution in regression-report (busof range-map root cause, ZZ_base9 both-direction verification) check-in: c914dacd06 user: refaqtor tags: trunk | |
| 17:32 |
8c3bd43e67 FIXED: busof() was an address-range map - every transfer to addr >= 8 went to nonexistent bus 1
The '>= 8 true-silence dead zone' root cause: usb_app.busof() mapped addresses to buses by the 1160 range convention alone, so on a single-bus rig every usb_app-side access to an address >= 8 transmitted into the void. Hub-host enumeration pins its own bus (never calls busof) - hence 'enumerates fine, deaf to everything after'. New busof: single live bus -> 0; else roster decides; range convention only as dual-bus fallback. Bitwise-identical for addr < 8 single-bus (the whole existing gate). Verified both directions with ZZ_base9 (estate enumerated at 9+ via dev_addr_base - ordinary enum, no SET_ADDRESS gray zone): fix green at addr 10/11 through fs_mount+root-enum; old busof restored = total red on the same recipe (logs -132707/-133103 green, -132850 red). dev_readdr rewritten via app host + gray-zone CAUTION; dev_addr_base added; usb_host gained clear_addr_toggles passthrough. check-in: 27d474f58d user: refaqtor tags: trunk | |
| 17:13 |
6160 CLOSED: hot-plug suite 11/11, gate stanza, full-board revalidation of the engine fixes
HP suite gained the PRE-notice abandon-honesty test - the falsification run (headless_260828-121230) proved it goes RED under a syncing teardown (6 BOT records) while the post-notice form cannot see one (guards refuse before the ring's BOT layer; run -120840). Clean run 11/11 (-121603). Gate: HP stanza (HP_EXP=11); grep -a on log parsing (pnut-term logs carry NULs; plain grep read every green suite as 'did not finish'). Full board on the changed engine (alloc lowest-free + clear_addr_toggles): FAT32 223/223, exFAT 219/219, XS 27/27, XM 16/16, MD 13/13, HP 11/11. Docs: README gate row + regression-report hot-plug section (6160 covered; 8c3bd43e67 addr>=8 silence remains open). check-in: 1ee482f5c5 user: refaqtor tags: trunk | |
| 15:59 | 6160 ROOT CAUSE: clear per-address toggles on SET_ADDRESS (usb_client.clear_addr_toggles, called from xfer_set_addr). Address reuse by the lowest-free allocator exposed a latent bug: stale OUT toggle made a fresh device ACK-and-discard the first CBW (USB 2.0 8.6.4 duplicate rule), NAK INQUIRY 10 s, then our BOT-reset storm wedged it deaf. Parity-deterministic churn-cycle-2 death, runs -112329/-114608. Old monotonic allocator masked this forever by never reusing addresses. check-in: 317b7249c9 user: refaqtor tags: trunk | |
| 15:37 | 6160: 3 s off-dwell in HP churn (300 ms VBUS bounce wedges the thumb - run 260828-112329); ZZ_cure3 probe. Run 2 was 8/10: allocator fix verified (addr 4 reused every cycle), VolumeDirty case green. check-in: a0ff2cceed user: refaqtor tags: trunk | |
| 15:16 |
6160: hot-plug suite (7/10 first run) + addr>=8 defect found (8c3bd43e67) + alloc_address lowest-free-in-window fix
HP_hotplug_tests: detach-mid-write fails clean and fast, EV_DETACH fires, fs_abandon proven zero-BOT-I/O via the 6150 evt ring (both abandon paths), sticky re-bind by serial with content-verified witness, no silent retarget. Churn walked the old monotonic allocator past addr 7 into a zone where control transfers meet hs=$100 silence (new ticket 8c3bd43e67, open); alloc_address now allocates lowest-free in the per-bus 7-address window, honoring the usb_hub_ram addr&7 toggle-table invariant. Diagnostics: usb_app dev_probe/dev_ctl/dev_ctl_q/dev_readdr, usb_host last_phy_status/last_saw_sop. ZZ_* probes preserved as bisection tools. Bench: hub ports 3+4 device-wedged (deaf at every address, immune to all software recovery) - awaiting physical hub reseat. check-in: 5aad03a281 user: refaqtor tags: trunk | |
| 13:51 |
6190 CLOSED: MALFORMED/HOSTILE MEDIA covered for BOTH filesystems. XS_malformed_tests (16 tests, 16/16 x3, gated, seconds to run, no USB device): hostile synthetic volumes served through set_block_device - FAT32's FIRST synthetic media ever (the injection point existed since 2070, never aimed at hostile images).
Fail-clean made MECHANICAL: the shim splits out-of-range READS (induced failures - refused, counted, asserted per case) from out-of-range WRITES (always a violation, audited ZERO across the run), and every hostile-chain read is bounded by an explicit byte count so hangs convert to visible failures. Methodology lesson recorded: a hostile chain is only tested if the reader WALKS it - the sabotaged file must CLAIM enough size to force traversal (measured: a 24-byte size field capped every read before the first bad link). exFAT: absurd/out-of-range bitmap, out-of-range root, ClusterCount lies, truncated entry sets, divergent backup region - all clean, all coded. FAT32: chain self-loop + 2-cycle (bounded, no hang), chain off the FAT end (one real cluster then refused, nothing fabricated), cross-linked files, start-cluster 0/1, lying FSInfo, and the entryOffsetInCluster mid-cluster-root geometry synthesized at last (previously only drive C's physical format reached it). Not covered, stated plainly on the ticket: NameHash-disagrees (read side doesn't consult it), and hostile-media WRITE campaigns (needs its own design - writes mutate the fixture under test). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QV5uNCPuTaXxxymx8mvCo3 check-in: f36e42bf7e user: refaqtor tags: trunk | |
| 12:49 |
6140 CLOSED: MULTI-DRIVE SUPPORTED, TESTED, GATED. usb_app gains (1) the BOT-OPERATION LOCK - CBW/data/CSW sequences from two cogs previously had nothing preventing interleave on the shared host (the mailbox lock serializes transfers, not multi-transfer BOT ops); one lock now wraps every whole BOT operation, primary and second-slot, recovery ladder included, lock order fs-driver->msc; and (2) the SECOND DRIVE SLOT - drive_bind_second captures a second device's endpoints+capacity without disturbing the primary selection, disk_read_b/disk_write_b route by them under the lock, fs_mount_second_exfat mounts the exFAT singleton there while the facade keeps its primary volume. One FAT32 + one exFAT is the singletons' only expressible pair, now first-class.
MD_multidrive_tests (13 tests, gate stanza): all five 6140 items - two-cog concurrent I/O (16 rounds/side, content-verified, ZERO errors x4 runs, worker on the ticket-prescribed 600-long stack), dual-mount dispatch integrity + cross-visibility both ways, selection churn with per-rebind label confirmation, THE BYSTANDER INVARIANT as witness-hash assertions on both idle drives, contention fairness recorded (58-76 kB/s per side, within ~10%). THE STANDING NEGATIVE: second slot bound to a FAT32 thumb must refuse exFAT mount with -81 - the routing discriminator runs every time. Lock-in-hot-path fully re-verified: FAT32-B 223/223, exFAT 219/219, XS 27/27, MD 13/13; build matrix 3/3 (63_504/72_309/86_716). Ledger note: locals named 'pa' collide with the P2 PA register. README + regression report updated. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QV5uNCPuTaXxxymx8mvCo3 check-in: 624bcef175 user: refaqtor tags: trunk | |
| 05:55 |
6170 CLOSED: FULL fs_* FACADE PARITY - the exFAT ladder column now equals the FAT32 column minus only white-box-by-design tests: FAT32 223/223 (drive B), exFAT 219/219, XS structural 27/27. All 27 ledger items resolved:
FACADE (usb_app): per-cog exFAT WORKING DIRECTORY - a per-cog path prefix joined onto every relative path in every path-taking fs_* call, '.' resolving to the cwd and '..' to its parent (exFAT stores no dot entries - a literal join missed on healthy media, caught by FS_dirhandle #24), targets validated (-85/-111), prefixes cleared on mount/unmount/abandon, per-cog join scratch (fs_* is not serialized here). fs_seek REFUSES past-EOF on both filesystems (silent clamping relocates writes - the corruption shape). fs_eof maps FAT32's TRUE-plus-parked -93 dir-handle shape to the converged -114. fs_max_handles() capability query. fs_rename/fs_move forward to exFAT. xpath local pl->plen (operator-caught: collides with the PL2303 OBJ instance in USB_CDC builds; CDC-inclusive build verified compiling). exFAT DRIVER: rename + moveFile IMPLEMENTED (raw-copied $85/$C0 primaries, rebuilt $C1 names, NameHash+SetChecksum16 recomputed; in-place when the entry count holds, relocate+clear when it changes; move copies the set raw between directories - checksum carries; refusals -112/-92/-111/-94; same-dir move is a no-op). Handle-kind guards SPLIT in 7 methods: a dir handle in a file op is -114, not -87's lump. openDirectory on a FILE: -111, not -85's lie of omission. MAX_OPEN 4->6 (parity with FAT32). WRITE-ON-EXIT error contract: command PUBs clear last_error at entry (accessors exempt), plus re-clears after the four dup-check MISSES that are success paths (find_in_dir arms -85 on every miss - caught by FS_error_handling #16). VERIFICATION: XS structural grew 2 rename tests (in-place set rewrite verified ON THE MEDIA at the same entry index with independent checksum/hash recomputes; relocation leaves no orphan members) - 27/27. Ladder measured then greened row by row; every FAT32 control count unchanged. Gate EXPECT at parity, KNOWN-SHORTFALLS block retired, FSDRV env selects the FAT32 drive (estate rotates; absent-pinned drive A failed every row with 'no volume matched'). Suite headers' 'chdir is a no-op on exFAT' claims annotated historical. README + regression report at parity. FS_E_UNSUPPORTED now unreachable, reserved. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QV5uNCPuTaXxxymx8mvCo3 check-in: 87a5a714a8 user: refaqtor tags: trunk | |
| 03:37 |
6180: the benchmark harness lands and completes the ticket's measured half. harness/top_6180_bench.spin2 - per-drive, serial-pinned, commit-stamped, diffable: sequential throughput vs transfer size at the BLOCK layer (512..4096 B, txpat save/restore scratch-span discipline, every restore spot-verified), random single-block latency DISTRIBUTIONS (min/med/p90/max/mean us, fixed LCG seed), fs_* metadata costs, and error-counter deltas around every phase.
Findings (docs/characterization/four-drive-geometry-260827.md third pass): at 4 KB chunks ALL media converge to 770-960 kB/s - the host is protocol-bound at 65-80% of the 12 Mb/s ceiling, not media-bound; random single-block reads cost ~2 ms EVERYWHERE (a host/protocol floor invariant from 128 MB card to 953 GB SSD); the SSD's random-write distribution is a flat 993 us (bridge cache, no tail) while the card's is the flash-GC textbook - median 2 ms, p90 102 ms, the exact tail the NAK budgets exist to survive, now measured; the SSD's 275 ms/mkdir is the 128 KB cluster zero-fill in the open; and ZERO error/retry deltas across every phase on all four drives - every difference is work, not retries. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QV5uNCPuTaXxxymx8mvCo3 check-in: 898ae22b64 user: refaqtor tags: trunk | |
| 02:46 |
6180: second characterisation pass - SD-reader cross-section. Port 4 now a $14CD:$1212 reader with an unused 128 MB card ('Appo', FAT32, 4 KB clusters - a fourth live geometry: small-media FAT32). Card measured: writes 11-19 kB/s flat across all six sizes (flash program time swamps cluster-boundary structure; 32 KB-class writes ~2.3-2.6 s) while READS are the bench's fastest sequential - 358-468 kB/s. Thumbs + SSD repeat pass 1 within noise (repeatability). Doc also records the hub-vs-direct answer: FS hubs are bit-level repeaters, numbers are media/protocol-bound far below the 12 Mb/s ceiling, and the one ticket that could have invalidated hub measurements (6200) was retested clean and closed tonight.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QV5uNCPuTaXxxymx8mvCo3 check-in: 9b8c6a3d39 user: refaqtor tags: trunk | |
| 02:14 |
6180: four-drive geometry characterisation write-up (docs/characterization/four-drive-geometry-260827.md) - the measured dataset from tonight's bench pass: per-drive geometry, open-dominated read-all rates, freeSpace cost by filesystem (FAT32 4 ms vs exFAT 4.77 s bitmap popcount), and the six cluster-boundary write timings with the findings (4 KB boundary crossing costs 1.7x; cluster size is the write-cost lever; exFAT NoFatChain fastest with a repeatable 63-vs-64-sector 2.2x anomaly; first-write flash variance).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QV5uNCPuTaXxxymx8mvCo3 check-in: f0c0d6e935 user: refaqtor tags: trunk | |
|
2026-08-27
| ||
| 21:15 |
6120: add XS_exfat_struct_tests.spin2 itself - the 25-test structural suite the previous commit described and gated (its 'fossil add' had run from the wrong directory and silently missed; the gate stanza, driver fix, and docs landed there, the suite source lands here).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QV5uNCPuTaXxxymx8mvCo3 check-in: 3c7632511a user: refaqtor tags: trunk | |
| 21:14 |
6120: exFAT STRUCTURAL SUITE LANDS (XS_exfat_struct_tests, 25 tests) AND FINDS A REAL DRIVER DEFECT ON ITS FIRST RUN. Synthetic exFAT volume in hub RAM served through set_block_device() (the 0186 rig extended): boot-region checksum incl. the excluded-bytes exclusion pinned, VolumeDirty asserted ON THE MEDIA through write/unmount/abandon, allocation-bitmap round-trips + the 2080 wear policy + freeSpace vs independent popcount, entry sets with SetChecksum16/NameHash verified by INDEPENDENT spec recomputes + multi-$C1 names + no-orphaned-members-on-delete, the full NoFatChain lifecycle incl. the LYING-FLAG corruption case pinned to its exact deterministic misread, ASCII up-case matching + fabricated non-ASCII lossy handling, and a bounds-guarded block shim asserting ZERO out-of-range accesses across the whole run.
THE DEFECT, predicted from reading and confirmed red on the first execution: extend_chain's conversion loop with h_nclus==1 is a from-0-to-minus-1 range - Spin2 REVERSES it (the 0186 class) and wrote set_fat(first-1, first): a stray FAT link in the NEIGHBOURING cluster's entry (FAT[4] measured EOC->5 on the synthetic media). On real media that entry can belong to any chained file: silent bystander chain corruption. It survived the 0186 static audit because the path needs a runtime collision state. FIXED (guard: convert loop only when h_nclus >= 2); suite 25/25. GATED: run_fs_gate.sh gains the structural stanza; full pass GREEN with the fix in - FAT32 223/223, exFAT 192/192, structural 25/25 (fs_gate_result.txt; sizes 58_700/71_869/81_876). Documented gaps recorded on 1891ce6d6a: backup boot region not consulted, SetChecksum16 not verified on parse, $82 up-case table not loaded, no truncate API. README + regression report updated. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QV5uNCPuTaXxxymx8mvCo3 check-in: 53c017c341 user: refaqtor tags: trunk | |
| 20:29 |
6110: THE ELEVEN-SUITE FACADE LADDER IS COMPLETE AND GATED GREEN - FAT32 223/223, exFAT 192/192, every row matching in one end-to-end pass on the wrap-fixed engine (fs_gate_result.txt). FS_directory's exFAT row measured at its true 25/30 on the operator-reformatted SSD (headless_260827-155607, zero BOT lines) and joins the gate with -D SD_INCLUDE_DEBUG on the FAT32 column for its cluster-0 white-box group.
exFAT: from zero regression coverage to 219 tests running / 192 passing through the application-facing fs_* surface, every FAT32 column matching its RT_ original, all 27 exFAT shortfalls named and filed (seek clamp 2, chdir 7, handle-limit 3, error-code parity 8, rename/move unsupported 6, write-on-exit 1). Gate header's wedge note rewritten as history: a BOT-line deviation today is a real signal, not weather. README + regression report updated to the final table. Remaining unported: date-API (timestamp), lifecycle (multicog - needs facade-native design), and the white-box-by-design four, whose exFAT structural equivalents belong to 6120. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QV5uNCPuTaXxxymx8mvCo3 check-in: aa4f7d63bb user: refaqtor tags: trunk | |
| 07:44 |
6100/6080 ROOT CAUSE FIXED: the BOT wedge was the HOST's timer arithmetic, not devices, media, or load. Twelve sites in usb_host.spin2 compared absolute getct() values against deadlines (8 transaction-timeout sites + 4 SOF-scheduler due/resync sites); every such compare misfires when its window straddles the 32-bit CNT wrap - a 21.47 s hazard grid, phase fixed at chip reset. SOF anomalies landed BETWEEN PACKETS of in-flight transfers (bridge goes ACK-then-silent = 6080); transaction sites fired spurious timeouts (the logged phy_st=-2 stamped on transfers the device was still ACKing, hs=$D2).
Cracked by measurement: four consecutive FS_directory exFAT runs threw their first BOT event at mount+10.66..10.70 s - 40 ms repeatability ACROSS A PHYSICAL RESEAT. Phase aliasing between each binary's deterministic I/O schedule and the wrap grid explains the entire recorded phenomenology: per-suite quasi-deterministic wedge points, 'clustering in time', short-suite immunity, clean gate passes beside 3-for-3 standalone wedges, thumbs wedging 'in ~25 ops', reseat/VBUS immunity. FIX: wrap-safe subtract-then-sign-test at all 12 sites (+1 long each; both fit blocks close). VERIFIED FAIL-FIRST: reproducer red 5 runs, then the same suite/drive/phase ran straight through the hazard instant with ZERO BOT lines (headless_260827-030747); top_txpat all patterns PASS incl. full-range SSD LBA; FULL FS GATE GREEN with the fixed engine - FAT32 190/190, exFAT 167/167, every row matching (fs_gate_result.txt; sizes 58_688/71_869/81_864). ALSO: FS_directory cleanup now empties test dirs BY ENUMERATION - wedge-era kills left /RTMANY holding a NAMELESS entry set fs_delete cannot address (reported, not spun on; 6190/6120 material). Its true exFAT count is 25/30 (cd x2, fs_move x3); the gate row stays parked until an SSD fsck clears the residue. Tickets 1464ab6579/06a0b73ad7 carry the full record; README + regression report updated; wedge-classes memory annotated with the root cause. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QV5uNCPuTaXxxymx8mvCo3 check-in: 3fd2a6ccb9 user: refaqtor tags: trunk | |
| 06:43 |
6110: FS_directory ported (eleventh suite, largest at 33 tests) - FAT32 control 33/33 FIRST RUN, matching RT_directory exactly. Facade gains fs_move (FAT32 forwards to moveFile via xlat; exFAT returns FS_E_UNSUPPORTED -120, completing the rename/move pair); legacy readDirectory(idx) enumeration translated onto the dir-handle API; cluster-zero raw-patch group stays driver-bound FAT32-only under the hybrid pattern (exFAT totals 30).
THE TEN-SUITE GATE RAN GREEN END TO END in one pass first (fs_gate_result.txt): FAT32 190/190, exFAT 167/167, delete_guard 8/8 both filesystems, no wedge - the owed acceptance run delivered. FS_directory exFAT is WEDGE-BLOCKED, not port-blocked: three runs 02:09/02:41 wedged progressively EARLIER (tests #29 -> #20 -> #11), every write-side event opening with the new op=$2A hs=$D2 signature, run 3 firing on a mkdir 128KB cluster zero-fill burst; drive A ran 33/33 between them and the SSD had passed the whole gate at 01:07 - the device's susceptibility is worsening across wedge cycles, recorded on 1464ab6579 (which also gains: FIRST observed successful inline BOT recovery rst=1 clrO=1 clrI=1 - 6080's 'cannot clear' needs a severity qualifier). Both fuller runs agree the genuine divergence set = cd x2 + fs_move group x3 = 25/30 true; the gate row is PARKED in run_fs_gate.sh with uncomment instructions pending a wedge-free measurement, which likely needs an SSD reseat. README + regression report updated. Six suites remain unported: date-API, lifecycle, and the white-box-by-design group. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QV5uNCPuTaXxxymx8mvCo3 check-in: a4cf3f55ae user: refaqtor tags: trunk | |
| 05:28 |
6110: FS_error_handling joins the facade ladder (19/19 FAT32, 11/18 exFAT) - ten suites, FAT32 190/190, exFAT 167/189, every FAT32 column matching its RT_ original. Its FAT32 control caught a REAL FACADE DEFECT on first run: fs_eof's error leg bypassed xlat() and leaked FAT32's raw -93 through the published contract (same class as the fs_dir_open leak) - fixed in usb_app, control re-verified 19/19. The RT_ original could never see it (asserted only nonzero).
Hybrid pattern established for suites with a white-box MINORITY: the one raw-sector MBR-preservation test stays driver-bound under #IFDEF USB_FAT32 (safe: the driver is a DAT singleton, same fact every RT_ suite relies on); FAT32 runs all 19 for control parity, exFAT runs the 18 that belong to the application surface. Gate FAT32 column gains -D SD_INCLUDE_RAW (only FS_error_handling EXPORTDEFs it; inert elsewhere). NEW MEASURED PARITY GAP (08de1c7aaa): exFAT has no fs_error() write-on-exit contract - a successful operation never stores 0, so a stale error survives success. Plus 4 more dir-handle-misuse code instances counted; full 22-shortfall ledger on the ticket. delete_guard exFAT corrected UPWARD to 8/8: the 01:07 wedge-free gate pass scored the true count and the gate failed the run on the improvement, exactly as designed; expected updated in this commit. 6691a78673 fully disposed - the suite is green on both filesystems. Wedge claim on 1464ab6579 corrected: NOT deterministic - three consecutive wedges 23:43-00:15 then the identical sequence clean at 01:07; wedges cluster in time, and the gate header now says how to read a wedge-window deviation. Ten-suite single gate pass still owed (started twice, externally stopped twice; build matrix green at 58_588/71_757/81_748). README + regression report updated to the ten-suite table and 22-item shortfall ledger. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QV5uNCPuTaXxxymx8mvCo3 check-in: 95fc3702bb user: refaqtor tags: trunk | |
| 04:41 |
6110: FS_dirhandle (25/25 FAT32, 19/25 exFAT) and FS_subdir_ops (18/18, 15/18) join the facade ladder - 171/153 across nine suites, every FAT32 column matching its RT_ original. Facade gains fs_rename (FAT32 forwards; exFAT returns NEW FS_E_UNSUPPORTED -120 honestly) and fs_dir_open now xlats FAT32 codes per the published contract - it was the one path-taking method leaking raw -40/-43. Two new error-code parity gaps filed on 08de1c7aaa (fs_dir_open on a file: -85 vs -111; file ops on a dir handle: -87 vs -114).
6230 (6691a78673) RESOLVED: the exFAT populated-directory-delete 'defect' was FS_delete_guard's own fixture - fs_chdir no-ops on exFAT, so the child landed in ROOT and the directory really was empty; the driver's SUCCESS was correct. Proven by ZZ_dgprobe (absolute paths, child ENUMERATED INSIDE, delete -> -96 on BOTH filesystems). FS_delete_guard rebuilt with absolute paths + a child-enumerates-inside setup check; FAT32 control still 8/8. Port discipline now written down: fixtures use absolute paths, only cwd-subject tests keep chdir, subdir counts exclude dot entries. The corrected delete_guard surfaced a DETERMINISTIC 6080-signature BOT wedge (op=$28 during the 384KB verify read, 3 of 3 runs at test #6) - recorded on 1464ab6579 as the first reliable reproducer; read_write exFAT also wedged once mid-gate then passed 49/49 standalone, so run pacing/wedge proximity is a variable. delete_guard exFAT stays 5/8 as WEDGE-TRUNCATED, stated in the gate header (same count as the old artefact for a different reason - the same-number-different-set hazard, written down). run_fs_gate.sh: rows for both new suites; 8-suite table ran end to end (one explained wedge deviation, standalone-reverified); the 9-suite single pass was externally stopped pre-hardware and is owed next session. README + docs/regression-report.md updated to the 171/153 table, the -110..-120 facade error contract, fs_rename, and current build-matrix sizes (58,588 / 71,753 / 81,744). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QV5uNCPuTaXxxymx8mvCo3 check-in: c1c490efb1 user: refaqtor tags: trunk | |
| 02:17 |
6130 CLOSED FIXED: the geometry matrix is now STANDING - run_geometry_matrix.sh, verified working. 6220 CLOSED FIXED. (tickets 74f78296de, 1107a9f1cc)
TWO TICKETS MEET THEIR STATED ACCEPTANCE AND ARE CLOSED. The rest stay open, and the survey below says why for each, so the ticket list reflects reality rather than optimism. 6130 asked for TWO things and both are done. 1. FINISH DRIVE C AND DRIVE B - drive C had never had a single suite run against it before this work: drive geometry 16 non-defrag suites RT_defrag_tests A 32 KB clus, align 0 332 pass 0 fail 0 recov 14 pass B 4 KB clus, align 0 332 pass 0 fail 0 recov WEDGES THE DRIVE C 4 KB clus, ALIGN 5 332 pass 0 fail 0 recov WEDGES THE DRIVE All three geometries produce THE IDENTICAL 332. Commit a359d54149's entryOffsetInCluster sign fix is now exercised on live media - drive C's non-zero alignment is the only geometry on the bench that reaches it. 2. MAKE THE MATRIX STANDING - "a documented way to run the ladder across all three thumbs and record a per-drive score table, so a geometry regression is visible at a glance." That is tests/regression/run_geometry_matrix.sh, and it is VERIFIED WORKING rather than merely written: run against drive A it produced the expected row exactly - 332 passed, 0 failed, 0 recovery events, defrag pass=14 fail=0 recoveries=0. It runs RT_defrag_tests LAST AND SEPARATELY, in its own column, and says why at length: defrag wedges 4 KB media, the gate runs alphabetically which puts defrag SECOND, and on drive C that wedged the drive and cost the remaining fifteen suites - which reported "pass=0 fail=1" apiece. Those were NOT test failures but the fixture reporting "no FAT32 volume matched" against a drive that had left the roster, and that run's "16 passed 21 failed" total was pure artefact. Reporting defrag separately is what stops one destructive suite poisoning the other sixteen. The script closes by stating what a regression LOOKS like - a row that stops matching the others - so the table interprets itself. 6220 closed on its acceptance being met in full: a fail-first test on the synthetic-volume fixture with the assertion OBSERVED going red (4 of 6 before, 7 of 7 after), the out-of-range access measured at LBA 4_194_316 exactly as predicted from reading the code beforehand, and the gate re-verified at 346/346. STILL OPEN, AND CORRECTLY SO:
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BsYQdKe1BGsfm1wqLw4KUo check-in: a2501a1fe2 user: refaqtor tags: trunk | |
| 01:48 |
FS GATE GREEN END TO END, FIRST FULL PASS: build matrix 3/3, FAT32 128/128, exFAT 119/119 - every suite matched its expected count on both filesystems. README rewritten around the results (ticket 8bcfaaa605)
==== PRODUCT BUILD MATRIX ==== USB_MSC USB_EXFAT BUILDS 58_548 bytes USB_MSC USB_FAT32 BUILDS 71_701 bytes USB_MSC USB_EXFAT USB_FAT32 BUILDS 81_676 bytes ==== FAT32 (drive A, by serial) ==== ==== exFAT (SSD, by serial) ==== FS_seek_tests 38 = 38 OK FS_seek_tests 36 = 36 OK FS_cogcwd_tests 5 = 5 OK FS_cogcwd_tests 3 = 3 OK FS_stress_tests 4 = 4 OK FS_stress_tests 4 = 4 OK FS_fatchain_tests 2 = 2 OK FS_fatchain_tests 2 = 2 OK FS_delete_guard_tests 8 = 8 OK FS_delete_guard_tests 5 = 5 OK FS_multihandle_tests 22 = 22 OK FS_multihandle_tests 20 = 20 OK FS_read_write_tests 49 = 49 OK FS_read_write_tests 49 = 49 OK total 128 passed, 128 expected total 119 passed, 119 expected ==== FS GATE GREEN: every suite matched its expected count on both filesystems ==== This is the run that was owed. The expected counts had been MEASURED from individual suite runs during the session but the gate had never executed in one pass; now it has, and it agrees with every one of them. The exFAT driver is gated - it had no gate at all before today, not a failing one, NONE - and the three product configurations are proven to compile with three distinct footprints, which is the evidence that a disabled filesystem leaves no remnant. README REWRITTEN AROUND THE RESULTS rather than around the history of getting them. It now opens with the two gates and what each REFUSES to call green - run_gate.sh fails on any missing result and on ANY BOT recovery event, run_fs_gate.sh fails on any deviation from expected counts IN EITHER DIRECTION so a new failure cannot hide behind a fixed one - then the FAT32 geometry matrix, the facade table for both filesystems, the product build matrix, a short section on WHAT TESTING THROUGH THE FACADE FOUND (the missing open-path table, the colliding error codes, the two silent-success parity divergences, and the populated-directory delete that is still open), and the gaps. The gaps section names them plainly: RT_defrag has never passed on 4 KB media; ten suites still cannot run on exFAT and the wall is error constants plus absent facade methods (rename, move, dates) plus a group that is WHITE-BOX BY DESIGN and rightly stays FAT32-only; and multi-drive, hot-plug, malformed media and exFAT structural are not exercised at all. It closes on the sentence the estate has always deserved - one drive at a time, well-formed media, no removals: A FLOOR, NOT A FINISH LINE. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BsYQdKe1BGsfm1wqLw4KUo check-in: ea5c42b8ce user: refaqtor tags: trunk | |
| 01:20 |
THE exFAT DRIVER IS NOW GATED, AND PROVEN BUILDABLE FOR PRODUCT INTEGRATION: tests/regression/run_fs_gate.sh (ticket 8bcfaaa605)
exFAT HAD NO GATE AT ALL - not a failing one, NONE. run_gate.sh is the FAT32 DRIVER gate: seventeen RT_ suites bound to usb_fat32_fs, one filesystem, 346/346. It cannot reach exFAT, because the suites it runs declare the FAT32 driver object directly. A driver with no gate cannot be regression-tested. run_fs_gate.sh runs the FS_ suites, which talk to usb_app's fs_* facade, so the SAME SOURCE runs on both filesystems. That is what made the exFAT driver gateable at all. IT CHECKS EXPECTED COUNTS, NOT "ZERO FAILURES", and the reasoning is the part worth keeping. exFAT legitimately fails nine of these tests today and every one is filed on a ticket. A gate demanding zero would be red forever and ignored within a week. A gate printing only a total would let a NEW FAILURE HIDE BEHIND A FIXED ONE - the number stays 119 while the set underneath changes. So every suite carries its expected pass count PER FILESYSTEM and ANY DEVIATION IN EITHER DIRECTION FAILS THE RUN. A count going UP fails too, deliberately: it means either a defect was fixed - in which case the EXPECT table is updated IN THE SAME COMMIT as the fix - or a test stopped running. Both need a human to look. The table carries a one-line reason for every exFAT shortfall and its ticket, so the gate explains itself to whoever reads it next. suite FAT32 exFAT why exFAT is lower FS_seek_tests 38 36 seek past EOF: FAT32 refuses, exFAT silently clamps (08de1c7aaa) FS_cogcwd_tests 5 3 fs_chdir: no cwd on exFAT, returns success anyway (08de1c7aaa) FS_stress_tests 4 4 FS_fatchain_tests 2 2 FS_delete_guard_tests 8 5 populated-directory delete; the guard exists and its scan misses the child (6691a78673) FS_multihandle_tests 22 20 MAX_OPEN 4 vs 6, a capability difference (08de1c7aaa) TOTAL 128 119 THE PRODUCT BUILD MATRIX - the "buildable for integration into products" half - RUN AND GREEN: USB_MSC USB_EXFAT BUILDS 58_548 bytes USB_MSC USB_FAT32 BUILDS 71_701 bytes USB_MSC USB_EXFAT USB_FAT32 BUILDS 81_676 bytes THE THREE DIFFERENT FOOTPRINTS ARE THE EVIDENCE THAT THE GATING WORKS: each build carries only what it asked for, which is Phase 0's requirement that a disabled class leave no remnant - checked now rather than assumed. The exFAT-ONLY build is the one nothing used to verify: usb_app declares xfs : "usb_exfat_fs" inside #IFDEF USB_EXFAT, so every FAT32-only build skips the exFAT driver entirely and nothing proved the exFAT-only configuration still compiled. Now something does. --build-only runs the matrix with NO HARDWARE AT ALL, which is what makes it usable in a build pipeline rather than only on the bench. STATUS, PRECISELY, because half of this is verified and half is not: THE BUILD MATRIX IS RUN AND GREEN - all three configurations, sizes above. THE HARDWARE HALF IS NOT YET RUN END TO END. The estate collapsed to 1 device / 0 mounted after the previous gate attempt wedged drive A, so there was nothing to run it against. The expected counts in the table are the MEASURED results of this session's individual suite runs, not guesses - but the gate itself has not yet executed in one pass. That needs a reseat and is the next thing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BsYQdKe1BGsfm1wqLw4KUo check-in: c1ac4dc81e user: refaqtor tags: trunk | |
| 00:58 |
exFAT NOW HAS THE OPEN-PATH TABLE IT WAS MISSING - two of three defects closed by ONE mechanism, and it cleared NINE failures in a different suite (tickets 6691a78673, 8bcfaaa605, 08de1c7aaa)
suite FAT32 exFAT before exFAT after FS_seek_tests 38/38 36/38 36/38 FS_cogcwd_tests 5/5 3/5 3/5 FS_stress_tests 4/4 4/4 4/4 FS_fatchain_tests 2/2 2/2 2/2 FS_delete_guard_tests 8/8 2/8 5/8 FS_multihandle_tests 22/22 18/22 20/22 FS_read_write_tests 49/49 40/49 49/49 TOTAL 128/128 105/128 119/128 THE MECHANISM, AND IT NEEDED NO NEW PER-HANDLE STORAGE. A file is identified by WHERE ITS DIRECTORY ENTRY SET LIVES: the parent directory's first cluster plus the entry-set base index. exFAT already recorded both per handle - h_pdir_f and h_dent, set by init_file_handle - because the write path needs them to flush the entry set. The only missing piece was a predicate that asks the question: PRI path_is_open(pdir_f, dsi, write_only) : yn scan the handle table for h_used == 1 and matching (h_pdir_f, h_dent); write_only nonzero -> only handles opened for writing count. That is the same identity FAT32 keys on - (dir_sector, dir_offset) in isFileOpen / isFileOpenForWrite - so the two drivers now answer the question the same way rather than one of them not answering it at all. WHY NOT A PATH STRING: paths are not unique (trailing slashes, case, "."), and storing them would cost MAX_OPEN x path bytes for an identity the driver already had. WHY NOT FIRST CLUSTER: a zero-length file has none, so two opens of an empty file would not collide - exactly what a create-then-open sequence produces. THE TWO GUARDS: openFileWrite - single-writer policy matching FAT32, checked BEFORE alloc_handle so a refused open consumes no handle. deleteFile - refuses while any handle holds the file, checked FIRST because the directory-emptiness branch below re-points the scan cursor and the identity is gone after it. New constants E_FILE_OPEN = -112 and E_FILE_ALREADY_OPEN = -113 EQUAL THE FACADE VALUES deliberately: usb_app publishes those numbers and passes exFAT statuses through untranslated, because exFAT's numbering IS the facade contract. NINE FAILURES IN FS_read_write WERE DOWNSTREAM OF THE SAME BUG, and that is the part worth remembering. They were the large-file and cluster-boundary tests (#40 read 128KB, #41 create 256KB, #44 delete reclaims space, #45/#46 all-$00 and all-$FF round-trips, #48 write spanning a cluster boundary, #49 seek across one), logged as an open question with only 5 BOT lines in the run so not wedge fallout. They were state corruption left by an EARLIER TEST IN THE SAME SUITE that deleted or re-opened a file while a handle was live. ONE MISSING GUARD, NINE DOWNSTREAM FAILURES, IN A SUITE THAT LOOKED LIKE IT HAD ITS OWN DEFECT. THE NINE REMAINING exFAT FAILURES ARE NOW FULLY ACCOUNTED FOR - none is unexplained:
GATE STATUS, STATED PRECISELY. The change is inside usb_exfat_fs.spin2, and usb_app declares `xfs : "usb_exfat_fs"` inside #IFDEF USB_EXFAT. The RT suites build -D USB_MSC -D USB_FAT32 with no USB_EXFAT, so THE exFAT DRIVER IS NOT COMPILED INTO ANY GATE BUILD and this change cannot reach 346/346. The immediately preceding gate run was 346 passed / 0 failed / 0 recoveries and contained every other change in this series. A confirming run was still attempted, because compile-gating is an argument and a green gate is evidence. It produced NO VERDICT: drive A wedged inside RT_delete_guard (1 pass, 7 fail, 15 recoveries) and every later suite reported the fixture's "no volume matched" setup failure. That is the wedge, not the code - and it is not claimed either way. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BsYQdKe1BGsfm1wqLw4KUo check-in: 5e3df3183c user: refaqtor tags: trunk | |
| 00:12 |
SEVEN SUITES NOW RUN ON BOTH FILESYSTEMS - 128 tests on exFAT where there were zero this morning - and the port found a THIRD exFAT defect with the same root (tickets 8bcfaaa605, 6691a78673). Gate re-established at 346/346.
suite FAT32 exFAT FS_seek_tests 38/38 36/38 FS_cogcwd_tests 5/5 3/5 FS_stress_tests 4/4 4/4 FS_fatchain_tests 2/2 2/2 FS_delete_guard_tests 8/8 2/8 FS_multihandle_tests 22/22 18/22 FS_read_write_tests 49/49 40/49 TOTAL 128/128 105/128 GATE FIRST, because it was owed: 346 passed, 0 failed, 0 recovery events, 17 suites, with the error-contract change in. The debt recorded two commits ago is cleared - translating FAT32's driver codes into the published facade contract costs the FAT32 baseline nothing. And the estate was brought back WITHOUT a reseat by using 6200's own finding: after the SSD reseat the probe showed 4 devices but only ONE mounted, thumbs failing st=-7 - 6200's signature exactly. The probe in use was the default build that calls begin2() on the EMPTY second header. Rebuilt -D NO_BUS1, the same rig gave 4 of 4 immediately. The separator recorded on 293576b2fc told us which build to run instead of which hardware to blame. THIRD exFAT DEFECT, SAME ROOT: A SECOND WRITE-OPEN OF THE SAME FILE SUCCEEDS, returning a valid second handle where FAT32 returns already-open. TWO CONCURRENT WRITE HANDLES ON ONE FILE - each with its own position and its own idea of the chain, whichever flushes second overwriting the other's metadata. Together with "delete an open file succeeds" and "delete a populated directory succeeds", all three trace to ONE MISSING MECHANISM: exFAT keeps handles (h_used, h_pos, h_size) but nothing that maps a PATH to them, which is what every one of those guards needs in order to ask its question. FAT32 has exactly that table. That is a much more useful statement than three separate bugs. THE FAT32 CONTROL IS NOT CEREMONIAL - IT CAUGHT A DEFECT IN MY OWN TRANSLATION. FS_multihandle first came back 20 of 22 on FAT32, and both failures were fs_error() returning FAT32's RAW -91 where the operation that set it returned the facade's -87. I had translated returned statuses but NOT the STORED last error, so fs_error() disagreed with the very call that set it. Fixed, re-verified at 22 of 22. Without the control that would have surfaced during the exFAT run and looked exactly like an exFAT defect. WHAT THE PORT NEEDED THIS ROUND: fs_sync_handle() - flush ONE handle and leave it open. Both drivers have carried syncHandle() all along; the facade offered only volume-wide fs_sync(), so an application wanting to make one file durable had to stop the world, and two suites could not port at all. FS_read_write on exFAT is 40 of 49, failures clustered in the large-file and cluster-boundary tests (#40 read 128KB, #41 create 256KB, #44 delete reclaims space, #45/#46 all-$00 and all-$FF round-trips, #48 write spanning a cluster boundary, #49 seek across one). Only 5 BOT lines in the whole log, so NOT wedge fallout. Recorded as observation and NOT chased - per operator direction the estate gets run and the data collected before anything is hunted. REMAINING BLOCKERS for the ten unported suites are now narrow and named: rename/moveFile (absent from the facade), the date API, driver lifecycle (start/stop/COGLESS_RET), and raw-sector plus debugGet* access. That last group is WHITE-BOX BY DESIGN and should stay driver-bound and FAT32-only. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BsYQdKe1BGsfm1wqLw4KUo check-in: 74b1c1a4a5 user: refaqtor tags: trunk | |
|
2026-08-26
| ||
| 23:05 |
Fixture: ONE SICK DEVICE NO LONGER DENIES SERVICE TO EVERY SUITE - the bystander problem at the harness level (ticket 8bcfaaa605). Gate re-verification of the error-contract change is PENDING and NOT claimed.
THE FAILURE. After a session of exFAT testing the SSD went unhealthy - TEST UNIT READY returning not-ready, then a STALL on READ CAPACITY. EVERY SUITE THEN FAILED SETUP with "no MSC drive ready within 15_000 ms", including suites pinned BY SERIAL to a completely healthy thumb. The probe seconds later showed all three thumbs mounting fine. THE CAUSE. rt_usb_fixture.setup_pins() opened with a bare `ifnot usb.await_drive(...): return false`. await_drive returns as soon as the FIRST MSC device is ready, and first-found on this rig is the SSD. A sick first-found device therefore failed setup for the whole estate - the selection loop that walks the roster by serial never even ran. That is the bystander problem one layer up: a bad device denying service to suites that never asked for it. THE FIX. That wait is a SETTLING HINT, NOT A GATE, and it is no longer fatal. The selection loop already does its own per-device await_drive and accepts a device only when it actually MOUNTS, so continuing costs nothing: a run with genuinely no usable device still fails, just at the honest place with "no volume matched" instead of blaming a readiness timeout on a drive it never tried. VERIFIED ON HARDWARE: with the SSD still sick, the fixture now logs "first-found MSC drive not ready in 15_000 ms - CONTINUING; the selection loop waits per device" and then mounts drive A by serial - "[fixture] mounted FAT32 on addr 3 port 2 ... label='NO NAME'". Before the fix that same configuration failed setup outright. WHAT IS NOT DONE, AND MUST NOT BE READ AS DONE: THE 346/346 GATE HAS NOT BEEN RE-ESTABLISHED SINCE THE ERROR-CONTRACT CHANGE. usb_app now translates FAT32 driver codes into the published facade contract, which is a behaviour change on the FAT32 path, and the rule is that the gate runs for every driver change. Two attempts were made: - the first died on the fixture bug above; - the second got past setup and then ground through 70 BOT errors with tests failing, because the SSD is still unhealthy and the bus with it. Neither is a verdict on the change. A clean gate needs the SSD recovered first, and until that run exists the error-contract change is UNVERIFIED AGAINST THE BASELINE. It is committed, it is documented, and it is not claimed green. The SSD needs a reseat; the three thumbs are healthy. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BsYQdKe1BGsfm1wqLw4KUo check-in: 5e4ae39283 user: refaqtor tags: trunk | |
| 22:13 |
The facade's error codes now MATCH across filesystems - and the first suite that could then be ported found TWO CRITICAL exFAT DELETE DEFECTS (new ticket 6691a78673, plus 08de1c7aaa/8bcfaaa605)
"Why can't you make the E_* codes match?" - they can, and the state they were in was worse than "different". THEY COLLIDED: value FAT32 means exFAT means -90 E_TOO_MANY_FILES E_READ_ONLY -91 E_INVALID_HANDLE E_TOO_BIG -92 E_FILE_ALREADY_OPEN E_EXISTS -93 E_NOT_A_DIR_HANDLE E_DISK_FULL -94 E_INVALID_PARAM E_DIR_FULL -96 E_NO_ASYNC_OP E_DIR_NOT_EMPTY and the same NAME carried different numbers - E_DISK_FULL -60 vs -93, E_DIR_NOT_EMPTY -44 vs -96, E_NOT_MOUNTED -20 vs -80. An application testing `== -93` got "not a dir handle" on one filesystem and DISK FULL on the other. THIS WAS NOT A NEW API TO INVENT. The facade has PUBLISHED an error table since exFAT landed (README, "Error codes", -80..-96). FAT32's codes were never translated into it, so the facade was violating a contract it already documented. usb_app now translates them, and adds codes at -110.. for the concepts FAT32 can report that the exFAT-derived table had no word for (not-a-file, file-open, bad-dir-handle, invalid-param, end-of-file, no-contiguous-space, wrong-filesystem). BEHAVIOUR CHANGE, STATED PLAINLY: fs_* now returns the documented facade code for FAT32 volumes instead of the driver's raw number. That is the fix, but anything matching raw FAT32 numbers THROUGH THE FACADE needs updating. Code testing `st < 0` is unaffected. Direct callers of usb_fat32_fs are untouched. THE TRANSLATION IS VERIFIED, not assumed: FS_delete_guard_tests asserts FACADE codes (-96, -112) and passes 8 of 8 on FAT32, whose driver natively returns -44 and -47. Wrong translation would fail those assertions on FAT32. AND THEN THE SAME UNCHANGED SUITE, POINTED AT exFAT, SCORED 2 OF 8 AND FOUND TWO CRITICAL DEFECTS: 1. DELETING A POPULATED DIRECTORY SUCCEEDS, orphaning its contents. The guard EXISTS - usb_exfat_fs.spin2:603-613 selects the directory, scans it and returns E_DIR_NOT_EMPTY - and DOES NOT FIRE. The setup test confirming the directory was populated passed immediately before. The entry set is cleared and the parent's clusters freed while the child's clusters stay marked in use with nothing referencing them. Silent data loss on the project's stated PRIMARY DEFAULT filesystem. 2. DELETING AN OPEN FILE SUCCEEDS. exFAT's deleteFile has NO open-handle check at all - zero hits for h_used / h_open / E_FILE_OPEN across the method. The file is genuinely deleted, not merely reported deleted. FAT32 REFUSES, and that guard is upstream's A1 fix whose rationale is that the open handle's pending flush otherwise lands in a cluster ALREADY REALLOCATED TO ANOTHER FILE - the exact corruption shape this mission exists to stop. NO NEW TEST WAS WRITTEN FOR EITHER. RT_delete_guard_tests has passed for as long as the estate has existed, but it binds usb_fat32_fs directly, so it had only ever asked FAT32. Ported to the facade it asked exFAT and both defects appeared on the first run. That is the whole argument for the port, demonstrated rather than asserted. Incidental correction recorded on the ticket: UPSTREAM-FAT32-BACKPORT.md lists A1 as "ABSENT" from our fork. That is STALE - FAT32 refuses the delete and returns the code, verified by this suite passing 8 of 8 on drive A. suite FAT32 exFAT FS_seek_tests 38/38 36/38 FS_cogcwd_tests 5/5 3/5 FS_stress_tests 4/4 4/4 FS_fatchain_tests 2/2 2/2 FS_delete_guard_tests 8/8 2/8 TOTAL 57/57 47/57 Report and README updated with the error-contract change, the two defects, and the note that a green exFAT column can mean a suite asserted nothing there. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BsYQdKe1BGsfm1wqLw4KUo check-in: 612011f73a user: refaqtor tags: trunk | |
| 21:31 |
6110: FOUR SUITES NOW RUN THROUGH THE FACADE ON BOTH FILESYSTEMS - 49/49 FAT32, 45/49 exFAT - and the four failures are two real parity divergences (tickets 8bcfaaa605, 08de1c7aaa)
suite FAT32 (drive A) exFAT (SSD) note FS_seek_tests 38 of 38 36 of 38 seek-past-EOF divergence FS_cogcwd_tests 5 of 5 3 of 5 cwd divergence FS_stress_tests 4 of 4 4 of 4 clean FS_fatchain_tests 2 of 2 2 of 2 clean, with a caveat TOTAL 49 of 49 45 of 49 exFAT had ZERO suite coverage this morning. EVERY FAT32 COLUMN MATCHES ITS RT_ ORIGINAL EXACTLY (38, 5, 4, 2) - that is the control set, and it is what makes the exFAT column a result rather than a port artefact. TWO PARITY DIVERGENCES, BOTH THE SAME SHAPE: SILENT SUCCESS HIDING A SEMANTIC DIFFERENCE, which is the most dangerous API failure mode because the caller cannot detect it. fs_seek past EOF FAT32 refuses with an error | exFAT SILENTLY CLAMPS to EOF and returns success fs_chdir FAT32 changes directory | exFAT returns SUCCESS HAVING DONE NOTHING (it resolves full paths, no cwd exists) An application seeking to a computed offset and writing gets an APPEND AT EOF on exFAT where FAT32 fails loudly. One that chdirs then creates by bare name gets the file in ROOT on exFAT and in the cwd on FAT32. Filed on 08de1c7aaa, neither fixed - each is a genuine design decision about what the facade should promise, and the resolutions differ. NEITHER COULD HAVE BEEN FOUND BY THE DRIVER-BOUND LADDER. Each RT_ suite only ever runs against its own driver, so exFAT's seek and chdir paths had never been asserted on by anything. A GREEN COLUMN CAN MEAN A SUITE ASSERTED NOTHING, and that is recorded rather than glossed: FS_cogcwd's Test #2 - "one cog's cd does not affect another cog's CWD" - PASSES on exFAT precisely BECAUSE there is no cwd at all. And FS_fatchain passing does NOT mean exFAT's allocation structures are tested; its tests work through the file API, while the bitmap, up-case table and NoFatChain files remain untested (1891ce6d6a). This ticket warns against copying FAT32 chain tests onto exFAT and calling it coverage; that warning applies to its own green column. TWO MORE APPLICATION-API GAPS FILLED, both needed by the port and both worth having anyway: fs_sec_per_cluster() and fs_cluster_bytes(). Both drivers had published these through volumeGeometry() all along with an identical three-LONG contract, but the facade offered only the raw triple, so every caller wanting the commonest field had to know the layout. Three suites were blocked on exactly that. THE REMAINING WALL IS ERROR CONSTANTS, NOT METHODS. Twelve suites still cannot port, and most of their unmapped symbols are E_* codes whose VALUES DIFFER between the drivers (exFAT uses -80..-87). A suite asserting a specific code is asserting a driver's private numbering, so re-exporting FAT32's numbers would make suites pass against the wrong codes. Deciding what the FACADE promises for "file not found" - one code for both filesystems, translated where fs_chdir already reconciles cwd - is the next real design question, and it is a product decision rather than a test one. The white-box suites (raw sector access, debugGet*, defrag internals) should stay driver-bound and FAT32-only by design. docs/regression-report.md and the README both carry the facade table, the two divergences, and the caveat that a green exFAT column is not automatically coverage. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BsYQdKe1BGsfm1wqLw4KUo check-in: 450213b14a user: refaqtor tags: trunk | |
| 20:20 |
6110: FIRST SUITE PORTED TO THE fs_* FACADE AND RUN ON exFAT - 38/38 FAT32, 36/38 exFAT, and the two failures are a real driver divergence (tickets 8bcfaaa605, 08de1c7aaa)
FS_seek_tests on FAT32 (drive A) 38 of 38 PASS <- the control FS_seek_tests on exFAT (the SSD) 36 of 38 <- first ladder suite ever on exFAT THE FAT32 RUN IS WHAT MAKES THE exFAT NUMBER MEAN ANYTHING. FS_seek_tests is a behavioural port of RT_seek_tests - same tests, same order, same expected count - so a port that changed behaviour would show a different number and one that quietly dropped assertions would show a smaller one. 38 matching 38 says the port is faithful; only then is 36 of 38 a result rather than an artefact. THE TWO FAILURES ARE A GENUINE FACADE PARITY DEFECT, filed on 08de1c7aaa: fs_seek(h, past_EOF) FAT32 -> REFUSES, per its explicit contract "position must be <= file size (cannot seek past EOF)" (usb_fat32_fs.spin2:1119) exFAT -> SILENTLY CLAMPS to h_size and returns SUCCESS (usb_exfat_fs.spin2:380-383) The exFAT side is the dangerous one: a silent clamp does not merely differ, it MISLEADS. An application seeking to a computed offset and writing gets an APPEND AT EOF instead of an error, where the same caller bug on FAT32 surfaces immediately. Silent relocation of a write position is exactly the defect shape this project exists to catch. NOT FIXED - the choice is real (make exFAT refuse, make FAT32 clamp, or define and enforce the contract in usb_app the way fs_chdir already reconciles cwd semantics), and fs_seek's documentation currently says nothing about past-EOF at all. THIS ONE RESULT IS THE ARGUMENT FOR THE WHOLE PORT. Neither driver-bound ladder could ever have found it: RT_seek_tests only ever runs against usb_fat32_fs, so exFAT's seek path had never been asserted on by anything. One suite, pointed at the interface applications actually use, ran on both filesystems and produced the divergence on the first attempt. THE RECIPE, now mechanical rather than a design question: - every sd.<driver method> becomes fx.fs_<facade method>. RT_seek used twelve distinct symbols and all twelve map. - sd.SUCCESS becomes the literal 0 - the facade's contract is 0 for success, and importing the driver's constant would mean importing the driver. - the driver OBJ line goes. The suite stops referencing usb_fat32_fs at all, which is precisely why the same source runs on exFAT. THE FIXTURE FORWARDS, AND IT HAS TO. usb_app keeps state in VAR, so a suite declaring its own usb_app gets a SECOND INDEPENDENT DRIVER INSTANCE - mounting nothing while the fixture's instance holds the real session. rt_usb_fixture therefore gained plain fs_* passthroughs: each a single forward to the identically-named facade method, no branching, no error translation, no defaults. NOT the rt_fs shim that was rejected, which would have been a second COMMON INTERFACE re-implementing usb_app's semantic reconciliation somewhere free to drift. NAMED FS_ RATHER THAN RT_ DELIBERATELY: run_gate.sh globs RT_*_tests.spin2, so the 346/346 baseline is untouched by this file existing. The facade ladder grows beside the driver ladder until it is complete enough to replace it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BsYQdKe1BGsfm1wqLw4KUo check-in: 587ac0ca4d user: refaqtor tags: trunk | |
| 19:38 |
6130: THE GEOMETRY MATRIX IS COMPLETE for the 16 non-defrag suites - drive C's alignment-5 media, never run before today, is 332/332 clean. Plus a comprehensive regression report and a README summary (ticket 74f78296de)
drive cluster size alignment 16 non-defrag suites RT_defrag_tests A 32 KB 0 332 pass 0 fail 0 recov 14 pass (in the 346 baseline) B 4 KB 0 332 pass 0 fail 0 recov WEDGES THE DRIVE C 4 KB 5 332 pass 0 fail 0 recov WEDGES THE DRIVE ALL THREE GEOMETRIES PRODUCE THE IDENTICAL 332, zero failures and zero recovery events on each. Drive C had never had a single suite run against it before today, and its non-zero cluster alignment is the only geometry on the bench that reaches commit a359d54149's entryOffsetInCluster sign fix - so that fix is now EXERCISED ON LIVE MEDIA AND PASSING rather than argued from code reading. That is the reason this ticket existed. AND THE MATRIX SETTLES SOMETHING BY ELIMINATION: ALIGNMENT IS EXONERATED in the defrag wedge. Drives B (align 0) and C (align 5) behave identically in both directions - both pass all sixteen suites cleanly, both wedge on RT_defrag_tests. The variable they share is the 4 KB CLUSTER SIZE, not the alignment, and drive A's 32 KB clusters pass the same suite 14/14. Recorded as a correlation between suite, cluster size and wedge. NO MECHANISM PROPOSED - per operator direction the chase waits until the estate has been run. A PROCEDURAL FINDING WORTH AS MUCH AS THE DATA. run_gate.sh runs alphabetically, which puts RT_defrag_tests SECOND. On drive C that wedged the drive and cost the remaining fifteen suites, which then reported "pass=0 fail=1" apiece - AND THOSE WERE NOT TEST FAILURES. Every one was the fixture reporting "no FAT32 volume matched the selection among 3 device(s)": the drive was gone from the roster and the suite never started. That run's own total, "16 passed, 21 failed", is misleading and must not be read as an alignment-5 result. Re-running with an explicit sixteen-suite list, defrag excluded, produced the clean 332. On 4 KB media, run the explicit list and leave RT_defrag out or last - run_gate.sh already accepts a suite list, so this is discipline, not code. NEW: docs/regression-report.md. A full correctness-coverage report, companion to characterization-report.md which covers throughput. It carries the geometry matrix, the exFAT position (working by-serial fixture, freeSpace, tell/eof, and the structural reason the ladder cannot run there yet), the measured exFAT scan-storm limit, every defect found and fixed today, EVERY CLAIM WITHDRAWN, and a section titled "What is still untested" - because a coverage report that only lists successes is a marketing document. README gains a "Filesystem regression coverage" section with the matrix, what the 346 gate actually enforces (including that it fails on any recovery event), and the known gaps stated plainly: RT_defrag has never passed on 4 KB media, exFAT has no suite ladder, and multi-drive, hot-plug, malformed-media and exFAT-structural are not exercised at all. It closes with the same sentence the estate has always deserved - one drive at a time, one filesystem, well-formed media, no removals: a floor, not a finish line. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BsYQdKe1BGsfm1wqLw4KUo check-in: 2ea8927a2d user: refaqtor tags: trunk | |
| 16:49 |
6150: the settling run staged correctly - 16, 32 and 40 KB in ONE binary. Plus a reproducible number: 32 KB single-cluster files fail at op 20-21 (ticket 96a1cf0413)
THE ONE INTERNALLY-CONSISTENT RESULT SO FAR, and it argues against cluster count: 32 KB files - EXACTLY ONE CLUSTER on drive A's 32 KB clusters - failed at OP 20 (log headless_260826-122003, commit 42904687cd) and at OP 21 (log headless_260826-124228, commit 923d80e6ef). Two runs, same workload code, one op apart. Under the cluster-count reading a single-cluster file has no FAT chain and should not fail at all. It still cannot be compared against the old 16 KB / 120-op clean number, which came from the pre-two-pass harness. That is the whole point of the staged run. STAGED AND VERIFIED PRESENT THIS TIME: -D SOAK_CLUS_EDGE runs 16, 32 and 40 KB in ONE binary on one drive. 16 and 32 KB are both single-cluster; 40 KB is two clusters. 16 clean, 32 clean, 40 fails -> cluster count, within one drive AND one harness 16 clean, 32 fails -> byte size per file; cluster count is dead all three fail similarly -> neither; something else scales with file size A RESEAT WAS SPENT ON A FAILED EDIT, and that is worth writing down because the check I used was wrong rather than absent. The 16 KB phase had not actually been inserted - an earlier combined command was killed before its python ran - and I "verified" it with a grep for `run_phase_n(AX_SIZE, BASE_CHUNK, 16` that MATCHED A DIFFERENT BLOCK ENTIRELY (the SOAK_SIZE_SMALL arm, which legitimately contains that exact call). The grep returned 1, I read it as confirmation, and the run that followed tested the old binary and wedged a drive for nothing. The header line in the log said so plainly - "32 KB (1 clus) then 40 KB", with no 16 - and the first phase was 32 KB. THE LOG DISAGREED WITH WHAT I BELIEVED I HAD BUILT, and that disagreement was visible in the first four lines of output. Verification now greps WITHIN the block being edited (sed -n '/IFDEF SOAK_CLUS_EDGE/,/ELSEIF/p'), and the run's own header is read before trusting that a build is the build intended - the same discipline that already applies to comparing commit ids across logs. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BsYQdKe1BGsfm1wqLw4KUo check-in: 1dda9881d7 user: refaqtor tags: trunk | |
| 16:34 |
6150: WITHDRAWING the cluster-count conclusion - the comparison spanned a harness change. The build stamp made it checkable; I did not check (tickets 96a1cf0413, 1464ab6579)
THE CONFOUND: drive A, 16 KB, 120 ops CLEAN -> commit b76957865c drive B, 16 KB, failed at op 34 -> commit 42904687cd the two-pass rewrite landed at commit 1ddfa594d2, BETWEEN THEM Before that change an op wrote each chunk ONCE; after it, one_op writes the file sequentially AND rewrites every chunk, so an op moves 50% more data plus an extra seek per chunk. Drive B's ops were doing strictly more work than drive A's. The sentence that carried the conclusion - "same bytes, same op count, same chunk size, same code" - IS FALSE across those two runs. The two-pass change was made for good reasons (it makes access pattern a single axis and removed a seek-past-EOF contract violation) but it changed the WORKLOAD, and every number from before it is incomparable with every number after. WHAT IS UNDONE AND WHAT IS NOT: - Cluster count is UNSUPPORTED, not disproven. It may still be right; the evidence offered does not establish it. - The exclusions all stand, because each was measured WITHIN one harness: transfer size (4096 clean as phase 1, 512 failing as phase 4, same run), BOT transaction count (2x difference, identical op count at failure), cumulative ops and cumulative bytes (120 ops / 3.84 MB clean in the very run that then failed at 256 KB). AND ONE NEW INTERNALLY-CONSISTENT MEASUREMENT that argues against cluster count: 32 KB files - EXACTLY ONE CLUSTER on drive A's 32 KB clusters - FAILED AT OP 20 under the current harness. A single-cluster file should not fail at all under that reading. It stands alone until 16 KB is re-measured with the same binary. STAGED, NOT RUN: -D SOAK_CLUS_EDGE now runs 16, 32 and 40 KB IN ONE BINARY on one drive - 16 and 32 both single-cluster, 40 two clusters - so the only numbers compared are produced by identical code in one session. The estate is down to the SSD alone and it needs a reseat before that can run. THE PROCESS CHANGE IS THE MORE VALUABLE OUTCOME. A COMPARISON ACROSS TWO LOGS IS ONLY VALID IF THEIR COMMIT IDS MATCH. The comparability header exists exactly so this is checkable, and it worked - the ids differ and the difference was sitting in both logs the whole time. What was missing was the discipline of LOOKING before comparing. From here, every multi-run claim on 6150 names the commit id of each run it rests on, and any claim spanning two ids is unsupported until re-measured in one binary. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BsYQdKe1BGsfm1wqLw4KUo check-in: 923d80e6ef user: refaqtor tags: trunk | |
| 15:53 |
6150: scatter fix VERIFIED, and a constraint that cost a reseat - killing pnut-term-ts does NOT stop the P2 (ticket 96a1cf0413)
VERIFIED, not assumed (log harness/logs/headless_260826-114103.log, commit 1ddfa594d2 clean): the SCATTERED phase now runs 8 ops with 0 failures, 0 verify failures and 0 recoveries, with the sequential control clean in the same log. So the 8-of-8 CONTENT MISMATCHES reported earlier are confirmed as MY harness bugs and not media or driver faults, and the two-pass rewrite with checked seeks is correct. A fix to a verifier is now seen passing rather than believed. THE CONSTRAINT, worth more than the run it cost: KILLING pnut-term-ts DOES NOT STOP THE P2. The terminal only reads the debug stream; the chip keeps executing whatever was downloaded into it. The scatter-verification run was killed at the terminal DELIBERATELY and for a good reason - to protect the thumbs for the drive-B cluster-count experiment that needed them healthy. The soak carried on headless through its remaining phases, wedged every thumb on the hub, and the only thing the "protective" action achieved was to hide the damage while it happened. The next command found "target serial not found among 1 device(s)". TO ACTUALLY STOP A RUN: download something harmless (top_rig_probe.bin). The download resets the P2, and the reset is what ends the program. Recorded in the soak header where someone about to do this will read it, and in PROMPT-NEXT-SESSION.md's running-things constraints beside the "never run two pnut-term-ts at once" rule it sits next to. CONSEQUENCE: the drive-B experiment is built and staged but did not run. It is the discriminator between CLUSTER COUNT and BYTE SIZE PER FILE - 16 KB files on drive B's 4 KB clusters are FOUR clusters each, where the same 16 KB on drive A's 32 KB clusters is a single cluster with no FAT chain, and that single-cluster case is the only file size that has never failed. Next thing to run. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BsYQdKe1BGsfm1wqLw4KUo check-in: 42904687cd user: refaqtor tags: trunk | |
| 15:32 |
6150 arm 2: BOTH cumulative hypotheses are DEAD - and two harness bugs of mine were raising false CONTENT MISMATCHES (ticket 96a1cf0413)
PHASE 1 16 KB files, 120 ops -> 120 ok, 0 fail, 0 verify_fail, 0 RECOVERIES PHASE 3 128 KB files, 8 ops -> clean PHASE 4 256 KB files -> FAILED AT OP 1 120 OPS CLEAN KILLS THE ~25-OP ACCOUNT. 3.84 MB CLEAN KILLS THE ~3 MB ACCOUNT. Arm 1's first failure at op 7 with 256 KB files matched the byte prediction and looked like an answer; arm 2 existed because arm 1 alone could not separate "cumulative bytes" from "large files are simply harder". It is not bytes: 3.84 MB of 16 KB files passed without one failure or recovery, then 256 KB files failed on their FIRST op. file size ops before first failure bytes clean 16 KB 128+ (never failed) 4.1 MB 128 KB 8+ (never failed) 2 MB 256 KB 1 (after 136 clean ops) / 7 (from fresh) 64 KB ~25 (three earlier runs) No constant op count and no constant byte count fits any two rows. THE HYPOTHESIS THIS SUGGESTS, AS A HYPOTHESIS: CLUSTERS PER FILE. Drive A has 32 KB clusters, so 16 KB is a SINGLE-CLUSTER file with no FAT chain - the one size that never failed. 64 KB is 2 clusters, 128 KB 4, 256 KB 8, and failure arrives sooner as the chain lengthens. It would also explain why this project's signature corruption has always been multi-cluster: 6100's ~25 ops and the /TOOLS blob damage are both multi-cluster stories. THE DISCRIMINATOR IS CHEAP AND THE MEDIA ARE ALREADY ON THE RIG. Drives B and C have 4 KB clusters, so 16 KB there is FOUR clusters instead of one. Arm 2 unchanged on drive B: fails early => CLUSTER COUNT is the variable; runs 120 ops clean => it is byte size per file after all. One run, existing hardware. BYSTANDER WITNESS, FIRST READING: SSD at addr 2 before and after, and ALL FOUR DEVICES still present and mounted at the end - despite 33 recoveries. This run produced NO bystander loss, so whatever took the thumbs out in three previous runs is not an unavoidable consequence of recoveries. Ganged power is weakened, not dead: no witness re-enumeration was seen because no bystander was lost. TWO HARNESS BUGS, BOTH MINE, BOTH RAISING THE MOST SERIOUS SIGNAL THIS PROJECT HAS. Phase 5 reported CONTENT MISMATCH on 8 of 8 ops - "the right number of the WRONG BYTES". IT WAS NOT REAL. BUG 1: the scattered writer stamped chunk i then seeked to scatter_ix(i), so position p held the stamp of whichever i mapped there while the verifier expected stamp p. scatter_ix(64)=0, so chunk 0 held stamp 64 - precisely the "MISMATCH at chunk 0" reported eight times. BUG 2, found when fixing bug 1 moved the symptom to chunk 1 rather than curing it: the writer SEEKED PAST EOF in a freshly created empty file, against seekHandle's explicit contract "position must be <= file size (cannot seek past EOF)" (usb_fat32_fs.spin2:1119) - AND NEVER CHECKED fs_seek's RETURN, so the documented refusal was discarded and the write landed at the wrong offset. THE DRIVER WAS RIGHT BOTH TIMES. Once I had the arithmetic wrong; once I violated a documented precondition and then ignored the error that said so. A false content mismatch is worse here than a missed one - it is the signal everything else defers to. FIXED AND EXPLICITLY NOT VERIFIED: one_op now writes sequentially first (identical in both arms, which also makes access pattern a genuinely single axis instead of "one write pass versus two"), then rewrites each chunk at the same position - in order or scattered - with EVERY SEEK CHECKED. -D SOAK_SCATTER_ONLY exists to prove the scattered phase verifies clean against a sequential control in one log. It has not run: the drives wedged first. A fix to a VERIFIER is not to be believed until it is seen passing, so nothing about it is claimed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BsYQdKe1BGsfm1wqLw4KUo check-in: 1ddfa594d2 user: refaqtor tags: trunk | |
| 15:12 |
6150: a BYSTANDER WITNESS in the soak - the free discriminator between ganged hub power and cross-device contamination (tickets 96a1cf0413, fea65c4b2d)
THREE CONSECUTIVE RUNS have ended with every untouched thumb gone from the roster while the soak bound exactly one drive. That is 6140's bystander invariant with three independent observations behind it, and two candidate mechanisms have stood unseparated the whole time: cross-device contamination, or GANGED HUB POWER (drive_recover() cycles the bound device's port, and the driver's own escalation message names a ganged hub as a possibility). THE DISCRIMINATOR COSTS NOTHING AND IS NOW WIRED IN. A device only re-enumerates after losing power or being reset, so: - a ganged power cycle re-enumerates EVERY device, and a witness that never faulted comes back AT A NEW BUS ADDRESS; - cross-device damage leaves the witness's address alone. The soak now records the address of a device it will never touch - the exFAT SSD when soaking a thumb, drive A when soaking the SSD - at the start of the run and again in the summary, and states which reading the result supports: gone from the roster -> the bystander invariant again, mechanism unseparated address CHANGED -> evidence for ganged power, not cross-device damage address unchanged -> whatever happened to the other drives was NOT a bus-wide power cycle It costs one poll() at each end, which is traffic the idle loop does anyway. This is deliberately read out of runs that are HAPPENING REGARDLESS rather than given runs of its own, because every wedging experiment costs a physical reseat of three thumbs and that reseat is the real budget constraint on this line of work. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BsYQdKe1BGsfm1wqLw4KUo check-in: b76957865c user: refaqtor tags: trunk | |
| 14:17 |
6150: the OPS-vs-BYTES experiment - two arms predicting opposite directions from the same hypothesis (ticket 96a1cf0413)
The reverse-order run established that the wedge tracks cumulative WORK, and ruled out both transfer size and BOT transaction count. What it could NOT separate is OPS from BYTES: every phase compared used a 64 KB file, so bytes were strictly proportional to ops and the two moved together in lockstep. FILE SIZE SEPARATES THEM, and the predictions are an order of magnitude apart - which is what makes this decisive rather than suggestive. At 64 KB the wedge arrived at ~25 ops = ~3 MB moved (each op writes the file and reads it back). arm 1 -D SOAK_SIZE_BIG 256 KB files = 512 KB/op, 40 ops BYTE-limited -> first failure ~OP 6 OP-limited -> ~OP 25 arm 2 -D SOAK_SIZE_SMALL 16 KB files = 32 KB/op, 120 ops BYTE-limited -> first failure ~OP 96 OP-limited -> ~OP 25 TWO ARMS BECAUSE ONE WOULD NOT SETTLE IT. Arm 1 alone, failing early, is equally well explained by "large files are simply harder" - more clusters, longer chains, bigger allocations. Arm 2 is what rules that out: if the wedge is byte-limited it must arrive LATE with small files, not early. A hypothesis that predicts opposite directions in the two arms can be killed by either one. MEASUREMENT PLUMBING ADDED FOR IT: - run_phase_n() takes an explicit op count, so an experiment can run one long phase instead of being quantised to the 8-op default. run_phase() keeps its old signature and forwards, so every existing phase is untouched. - FIRST FAILING OP is recorded globally, once, at the first failure - a later cascade cannot overwrite it - and printed in the summary. That index IS the measurement here; everything else in the run is context. Both arms hold every other axis at baseline: chunk 512 B, gap 0, sequential, same drive, same content verification. NOT YET RUN. This commit is the experiment and its plumbing, compiled clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BsYQdKe1BGsfm1wqLw4KUo check-in: a61759d6af user: refaqtor tags: trunk | |
| 13:46 |
6150: -D SOAK_CHUNK_REVERSE - the experiment that separates SIZE from CUMULATIVE LOAD in the 4096-byte correlation (ticket 96a1cf0413)
Two runs found chunk=4096 B failing 2 of 2 while 512/1024/2048 were clean 6 of 6. That is a real correlation and it is NOT yet an attribution, because in both runs the axis was walked 512 -> 4096 and 4096 was ALWAYS THE FOURTH PHASE. Three explanations survive equally: (a) the SIZE 4096 is what breaks it (b) the FOURTH PHASE breaks it, whatever size it carries (c) CUMULATIVE transactions since power-on break it, and phase 4 is merely where the count arrives THE FIRST PHASE OF THIS BUILD IS THE WHOLE EXPERIMENT: 4096 B runs FIRST, on a freshly bound device, with nothing before it. fails at phase 1 -> SIZE. (b) and (c) both die - no prior phases, minimal prior transactions. clean at phase 1 -> not size alone, and then the repeat decides: PHASE 5 runs 4096 AGAIN after the three smaller phases. Failing there, having passed at phase 1, means CUMULATIVE LOAD or phase ordinal - the size is exonerated. The smaller sizes still run in between, so this build re-tests them too and the two orderings stay comparable phase-for-phase on everything except order. The header now prints which ordering is in force, and what each outcome would mean, because two logs differing only in a compiler flag are indistinguishable afterwards - and this pair differs ONLY in a compiler flag. THE BUILD STAMP DID ITS JOB ON THE WAY IN: the first build of this change was made from a dirty tree and the generator stamped it "d333673d64+ *** UNCOMMITTED CHANGES - dumps from this build are not reproducible from the repo ***". That is precisely the case it exists to catch, so this landed before the run rather than after. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BsYQdKe1BGsfm1wqLw4KUo check-in: 1f90083ee5 user: refaqtor tags: trunk | |
| 13:41 |
6150 soak run 2: the 4096-byte transfer-size correlation REPRODUCED (2 of 2), and the bystander question raised again by three untouched drives going dark (ticket 96a1cf0413)
THE RESULT. Driver commit ae549801a6 (clean), drive A, one axis moved per phase: PHASE 1 baseline chunk=512 B 8 ops 0 fail 0 verify_fail 0 RECOVERIES PHASE 2 TRANSFER SIZE chunk=1024 B 8 ops 0 fail 0 verify_fail 0 RECOVERIES PHASE 3 TRANSFER SIZE chunk=2048 B 8 ops 0 fail 0 verify_fail 0 RECOVERIES PHASE 4 TRANSFER SIZE chunk=4096 B 5 fail, 1 CONTENT MISMATCH PHASE 5 FILE SIZE chunk=512 B wedged; re-bind FAILED; RUN ABANDONED ACROSS BOTH RUNS: 512/1024/2048 clean in 6 of 6 phase-runs; 4096 FAILED IN 2 OF 2, with a content mismatch each time. The first axis correlation this project has been able to state, and precisely what the 6150 apparatus was built to produce. WHAT IS NOT ESTABLISHED AND MUST NOT BE ROUNDED UP: the chunk axis is always walked in the same ORDER and always as phases 2-4, so "4096" is not yet separated from "the fourth phase" or "cumulative transactions since power-on". Next run walks it in REVERSE (4096 first) and repeats it mid-run. If 4096 fails first-thing on a fresh device, SIZE is the variable; if it only fails after the smaller phases, CUMULATIVE LOAD is. THE FAILURE SHAPE DIFFERED BETWEEN RUNS, which is itself data: run 1's phase 4 took 2_915 ms with 16 recoveries; run 2's took 1_451_415 ms - 24 MINUTES - with ZERO recoveries, then a short read at chunk 59 and every later op failing at create. Same axis, same failure count, completely different profile. THE 6220 E_DISK_FULL MASKING SHOWED UP IN LIVE EVIDENCE. Ops 4-8 report "fs_create FAILED st=-60" - DISK FULL - on a 7.5 GB thumb holding one 64 KB file, because do_create() collapses any negative from allocateCluster() into E_DISK_FULL (usb_fat32_fs.spin2:3878). Filed from a synthetic test yesterday; here it is actively corrupting the reading of a real soak. It should be fixed. BOTH HARNESS FIXES FROM LAST NIGHT EARNED THEIR PLACE: - The abandon logic is why this run is readable at all. Phase 5 wedged, one re-bind by serial was tried, it failed, and the run STOPPED - saying "any axis beyond the last valid phase is UNTESTED, not proven clean". Run 1 without it produced eight further phases of artefact that read like four axes correlating. - The sliced gap wait means the multi-minute phases would actually have waited, had the run reached them. It did not, and the summary says so rather than leaving them looking untried-but-fine. COLLATERAL, AND IT RAISES 6140's BYSTANDER INVARIANT DIRECTLY. After the run the roster holds ONE device: the exFAT SSD. All three FAT32 thumbs are gone while ALL FOUR PORTS REPORT $0103 - connected AND enabled - which by the discriminator is the device-wedged class needing a physical reseat. THE SOAK ONLY EVER BOUND DRIVE A. B and C were enumerated at settle, never selected, never addressed, never written - and they went dark anyway. Two candidate mechanisms, NOT chosen between: cross-device contamination (6140/6200, and 22 recoveries ran this session), or GANGED HUB POWER, since drive_recover() cycles the bound port and the driver's own escalation message names a ganged hub as a possibility. Against the ganged reading: the SSD on port 1 survived and stayed mounted, though it may simply have re-enumerated where the flakier thumbs did not. The cheap discriminator is to watch the SSD's BUS ADDRESS across a soak that takes recoveries - a ganged cycle re-enumerates it at a NEW address even though it never faulted. ALSO FIXED HERE, found by re-reading my own output: the summary line labelled evt_count() as "held" and printed "7_648 held" against a 256-SLOT ring. count() returns records WRITTEN since arm, including overwritten ones. Now printed as written / overwritten / (256 held). A wrong label survives because nobody re-reads their own summary. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BsYQdKe1BGsfm1wqLw4KUo check-in: d333673d64 user: refaqtor tags: trunk | |
| 05:38 |
6150 soak, first run: THE WEDGE CORRELATES WITH TRANSFER SIZE - clean at 512/1024/2048 B, wedged at 4096 B. Plus two flaws in my own harness that the run exposed (ticket 96a1cf0413)
THE FINDING, AND IT IS EXACTLY WHAT THIS TICKET WAS BUILT TO PRODUCE. Log harness/logs/headless_260826-013051.log, drive A, every axis but one held at baseline: PHASE 1 baseline chunk=512 B 8 ops 0 fail 0 verify_fail 0 RECOVERIES PHASE 2 TRANSFER SIZE chunk=1024 B 8 ops 0 fail 0 verify_fail 0 RECOVERIES PHASE 3 TRANSFER SIZE chunk=2048 B 8 ops 0 fail 0 verify_fail 0 RECOVERIES PHASE 4 TRANSFER SIZE chunk=4096 B FAILED - 5 fail, 1 CONTENT MISMATCH, device WEDGED Three phases clean, the fourth wedges the drive, and the ONLY thing that moved is the maximum BOT data transfer size. "If the wedge correlates with an axis, THAT IS THE FINDING" - and this is the first time the estate has been able to say so, because it is the first workload that varied one axis at a time with the axis recorded into the evidence ring. IT IS ONE SAMPLE. n=1 on the 4096 phase, and the run that produced it was contaminated afterwards (below). It is a strong lead, NOT an established correlation, and it must be repeated - ideally with the chunk axis walked twice and the order reversed, so "4096" is separated from "the fourth phase" and from "cumulative reads". THE NEW FAILURE-CAUSE INSTRUMENTATION EARNED ITS KEEP IMMEDIATELY. The phase-4 failure reads: BOT data fail op=$28 hs=STALE(no handshake this xfer) phy_st=-2 sop=0 rxn=0 Same total-silence signature as 6200: PHY timeout, NO start-of-packet at all, and the handshake byte correctly labelled STALE instead of being reported as a NAK it never was. Then recovery gave up (rst=0 clrO=0 clrI=0), escalation to drive_recover failed repeatedly, and control transfers came back "CFG hdr fail hs=$100" - $100 being "no peer SOP ever seen". The device went completely silent to EVERYTHING, not just to bulk. TWO FLAWS IN MY OWN HARNESS, BOTH FOUND BY RUNNING IT, BOTH FIXED HERE: 1. THE LONG-GAP AXIS NEVER WAITED. waitms() computes WAITX((clkfreq/1000) * ms); at 200 MHz that is 200_000 * ms, which OVERFLOWS 32 BITS above ~21_474 ms. So waitms(60_000) does not wait a minute. MEASURED: phases 11 and 12, nominally 60 s and 300 s gaps over 8 ops each, both "completed" in 1_715 ms. THIS ONE MATTERED MORE THAN AN ORDINARY BUG. The multi-minute gap is the ONE axis 6200's idle hypothesis needs, and a silent zero-length wait would have produced a confident "long gaps make no difference" from a test that never waited - the worst kind of green. p2kb's "1-4,294,967 range" for waitms is the PARAMETER range, not the achievable delay, which is clock-dependent; the real ceiling here is the same ~21 s boundary the evidence ring already documents for getct at this clock. Now sliced into 10 s pieces. 2. THE RUN KEPT "MEASURING" AFTER THE DEVICE DIED. Phase 4 wedged the drive and the soak then ran EIGHT MORE PHASES against it, reporting per-axis failure counts that were purely an artefact of the wedge. A reader skimming that summary would see failures spread across four axes and conclude all four correlate, when only the first one did. That is exactly the "reproduction is not verification" trap in a new costume: the numbers looked like data. Now: if the device is still wedged at the end of a phase, ONE re-bind is attempted (by SERIAL - it returns at a new address), and if that fails the run is ABANDONED with an explicit note naming the last VALID phase and stating that later axes are UNTESTED, not proven clean. A short honest run beats a long contaminated one. The recovery counter added with the harness worked exactly as intended and is what made the contamination visible: phases 5-12 reported RECOVERIES=25..32 apiece, which is not a workload result, it is a dying drive. The drive recovered fully on its own afterwards - probe 4 of 4 devices, all mounted, no operator intervention. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BsYQdKe1BGsfm1wqLw4KUo check-in: ae549801a6 user: refaqtor tags: trunk | |
| 05:29 |
6150: the instrumented soak - comparability header, one-axis-at-a-time workload variation, unattended auto-recover. Gate green 346/346 (ticket 96a1cf0413)
THE RECORDER WAS LANDED AND PROVEN FIRST; THIS IS THE THING THAT DRIVES IT. All three pieces the ticket said were still owed, and explicitly NOT claimed until now: 1. THE COMPARABILITY HEADER. Every run prints driver commit id, stamp time, clock, media name and SERIAL, filesystem kind and label, geometry (sec/clus, first data sector, cluster alignment), capacity, and the workload baseline. A pattern is a statement about MANY runs; without this each dump is an anecdote and two dumps cannot be compared at all. THE COMMIT ID NEEDED A GENERATED FILE because pnut-ts has NO -D NAME=VALUE - its -D takes bare symbols only, so a hash cannot be passed on the command line. tools/gen_build_stamp.sh writes src/build_stamp.spin2 before the build, and DIRTY TREES ARE MARKED, NOT HIDDEN: a build from a modified checkout gets a '+' and an explicit "UNCOMMITTED CHANGES - not reproducible from the repo" line in the header. A dump claiming a clean commit id while the source had uncommitted edits is worse than one with no id, because it invites a reader to diff against a revision that was never built. The generated file is in .fossil-settings/ignore-glob rather than committed - committing it would make the tree dirty on every build and the id circular. 2. ONE AXIS AT A TIME, EACH CHANGE RECORDED INTO THE RING. Phases: baseline (nothing varied, and it runs FIRST because every later phase is read against it), then TRANSFER SIZE (512 -> 1024 -> 2048 -> 4096 B via the new disk_set_max_chunk), FILE SIZE (16/64/128/256 KB - it drives cluster COUNT, which is what makes a chain long enough to break in the middle), ACCESS PATTERN (sequential vs scattered-by-seek: same bytes, same file, ONLY the order differs), and INTER-OP GAP. Every phase stamps MK_PHASE/MK_AXIS into the ring before anything moves, so a dump says which axis was in effect without needing the debug log beside it - and a ring wrap cannot desynchronise them. If the wedge correlates with an axis, THAT IS THE FINDING - but only if exactly one thing moved. A soak that varies size and gap together produces failures nothing can be attributed to. THE GAP AXIS REACHES MINUTES, behind -D SOAK_LONG_GAPS (60 s and 300 s phases). 6200's evidence is that the fault appears on the first activity after the rig has been idle; a gap axis topping out at a second could never see it. It is a build gate because those two phases alone take longer than the whole rest of the run, and the default build says so explicitly rather than silently omitting them. 3. UNATTENDED, WITH disk_auto_recover(1, 3000) ON - the one place it belongs. A soak that stops at the first wedge collects ONE sample per session; one that recovers and continues collects many, and each recovery is itself a datum. WHICH REQUIRED COUNTING THEM. With auto-recover on, escalation happens INSIDE disk_read/disk_write and was invisible to the caller. usb_app now counts at the single site that can escalate (auto_recover_retry) and exposes disk_recoveries() / disk_recoveries_clear(), so the soak attributes recoveries to the PHASE they happened in. Every phase line ends with RECOVERIES=n, and a nonzero count prints "NOT the same result as a clean phase". A run that finished after three recoveries is not a clean run and must never be reported as one. CONTENT VERIFICATION, NEVER SIZE. Every 512-byte chunk is stamped with a marker, its OWN index, the phase and the file size, then read back and compared LONG BY LONG. The signature failure of this project is a perfect directory entry over a broken chain - THE RIGHT NUMBER OF THE WRONG BYTES - and a size comparison passes straight through it. Mismatches are counted separately from I/O failures and called out as the headline in the summary, because they mean something completely different. THE SCATTER IS DETERMINISTIC (odd chunks then even), not random. A random order would make two runs incomparable, which is the one thing this soak must not do. FACADE ADDITIONS, all needed by the above and all useful beyond it: disk_set_max_chunk() (the transfer-size axis was reachable only on the MSC object - an axis the harness cannot move is an axis that never gets tested), disk_recoveries(), disk_recoveries_clear(). GATE: 346 passed, 0 failed, 0 recovery events, 17 suites, RT_DRIVE=A, with all of this in. The baseline is intact. NOT YET RUN AS A SOAK. This commit is the harness and its plumbing, compiled and gate-clean. The overnight run and whatever it finds are the next entry. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BsYQdKe1BGsfm1wqLw4KUo check-in: d885de7742 user: refaqtor tags: trunk | |