Joey Eamigh
67beaede45
quantization + onnx sweeps
...
Phase 10.8: torchao/bnb quant sweep on iter1-independent. bf16 already
optimal; torchao int8-wo gives -19% VRAM at no F1 cost; all 4-bit
variants collapse (ModernBERT-large too quant-sensitive).
Phase 10.9: ONNX export + ORT eval. Legacy exporter only working path
(dynamo adds 56 Memcpy nodes); ORT fp32 -22% latency vs torch via
kernel fusion but bf16+flash-attn-2 still wins; fp16 broken on rotary;
dynamic int8 silently CPU-fallback + 0.5 F1 collapse.
Driver scripts wired to bun run py:quant / py:onnx; full reports at
results/eval/{quant,onnx}/REPORT.md.
2026-04-07 05:10:38 -04:00
Joey Eamigh
07dc3d6133
testing clspool and dapt on new architecture
2026-04-07 00:51:48 -04:00
Joey Eamigh
edcffbcc78
fix up status and deliverable docs
2026-04-06 17:02:05 -04:00
Joey Eamigh
4f5c88d94a
trying ensenble and nofilter versions of the model
2026-04-06 15:50:15 -04:00
Joey Eamigh
745172adb8
docs restructuring
2026-04-05 21:00:40 -04:00
Joey Eamigh
2e932bc327
working model!!!!!
2026-04-05 15:37:50 -04:00
Joey Eamigh
41df5923f2
adding old strategy notes for posterity (need to clean git history anyway)
2026-04-05 12:17:03 -04:00
Joey Eamigh
42f8849b14
first finetune attempt
2026-04-05 12:16:16 -04:00
Joey Eamigh
531317f7d4
corpus labeled
2026-04-05 01:30:39 -04:00
Joey Eamigh
6f4d6c57a4
labelapp updates v2
2026-04-05 00:55:53 -04:00
Joey Eamigh
160adc42ab
v2 holdout
2026-04-04 22:49:24 -04:00
Joey Eamigh
1f2d748a1d
new codebook and ethos
2026-04-04 15:01:20 -04:00
Joey Eamigh
d653ed9a20
pivot point
2026-04-03 14:43:53 -04:00
Joey Eamigh
26367a8e86
analyze gold
2026-04-02 09:28:44 -04:00
Joey Eamigh
c9497f5709
6 model panel benchmark
2026-04-02 02:02:36 -04:00
Joey Eamigh
e2c7a21c99
human labeling done
2026-04-02 00:28:31 -04:00
Joey Eamigh
b4319845e4
signoff deliverable draft
2026-03-31 16:58:29 -04:00
Joey Eamigh
96246d0197
docs & reference files
2026-03-31 16:27:47 -04:00
Joey Eamigh
32cd5ecfa8
opus golden set scaffolding
2026-03-30 22:02:52 -04:00
Joey Eamigh
7b660fe361
roll back to python 3.13 to fix everything lol
2026-03-30 21:25:46 -04:00
Joey Eamigh
8190950f1a
fix some bugs for tapt
2026-03-30 20:44:10 -04:00
Joey Eamigh
3292980d33
tapt setup
2026-03-30 19:46:20 -04:00
Joey Eamigh
1dce1ccb73
updating narrative at checkpoint resume
2026-03-30 15:20:59 -04:00
Joey Eamigh
313e14fb96
decisions for TAPT
2026-03-30 00:33:28 -04:00
Joey Eamigh
fe5155ab6d
pretraining config for run
2026-03-29 23:55:49 -04:00
Joey Eamigh
99cf4a606c
thread tokenization and chunking
2026-03-29 21:03:11 -04:00
Joey Eamigh
9d41dd199f
DAPT and precleaning for DAPT
2026-03-29 20:33:39 -04:00
Joey Eamigh
ca4bc288c9
labelapp timing fixes and migration
2026-03-29 16:37:51 -04:00
Joey Eamigh
a9a7d59603
DAPT/TAPT scaffolding
2026-03-29 16:12:19 -04:00
Joey Eamigh
3260a9c5d9
labelapp scaffold
2026-03-28 23:44:37 -04:00
Joey Eamigh
48e488933a
updating narrative and plan
2026-03-28 22:54:32 -04:00
Joey Eamigh
78d1f978de
initial scrape and tag
2026-03-28 20:39:36 -04:00
