diff --git a/README.fa.md b/README.fa.md index a4a23d1..fb241d8 100644 --- a/README.fa.md +++ b/README.fa.md @@ -65,12 +65,14 @@ print(doc.ents) # (محمدرضا شجریان, مشهد) | `sm` | ۵٬۴۸۴ | ۱۰٬۲۳۵ | | | `md` | ۵٬۴۰۸ | ۹٬۰۵۸ | | | `lg` | ۴٬۷۱۵ | ۹٬۲۱۵ | | -| `trf` | ۱۸۷ | | ۸٬۳۲۰ | +| `trf` | ۱۸۷ | ۱٬۱۵۸ | ۸٬۳۲۰ | ردهٔ `trf` روی یک پردازنده حدود ۲۹ برابر کندتر از `sm` است، و روی T4 نسبت به پردازندهٔ همان ماشین (۳۳۶ واژه بر ثانیه) ۲۵ برابر سریع‌تر، پس کارت گرافیک برای آن یک نیاز است نه بهینه‌سازی. -فاصلهٔ رده‌های پردازنده‌ای کمتر از ۱۵ درصد است، یعنی گلوگاه جست‌وجوی tok2vec نیست بلکه تجزیه‌گر -و واژه‌یاب است. پراکندگی اجراها روی لپ‌تاپ بسته به دمای دستگاه حدود ۱۰± درصد است. +حتی کارت ۲ گیگابایتی 940MX هم ۶٫۲ برابر سریع‌تر از پردازندهٔ همان دستگاه است و دستهٔ ۳۲ را بدون +کمبود حافظه اجرا می‌کند، اما به نسخهٔ `cu126` از torch نیاز دارد، چون هستهٔ sm_50 از نسخه‌های +`cu128`/`cu129` در ۲٫۸ حذف شده است. فاصلهٔ رده‌های پردازنده‌ای کمتر از ۱۵ درصد است، یعنی گلوگاه +جست‌وجوی tok2vec نیست بلکه تجزیه‌گر و واژه‌یاب است. پراکندگی اجراها روی لپ‌تاپ حدود ۱۰± درصد است. گام‌های تبدیل پیکره، آموزش، ارزیابی و بسته‌بندی در [`project.yml`](project.yml) تعریف شده‌اند. توضیح بیشتر دربارهٔ گزینش پیکره و پروانه‌ها در [`docs/MODELS.md`](docs/MODELS.md) و شرح انگلیسی diff --git a/README.md b/README.md index ea0a13e..d5eda7f 100644 --- a/README.md +++ b/README.md @@ -93,11 +93,13 @@ timing the pipe only, warmup discarded. Reproduce with | `sm` | 5,484 | 10,235 | | | `md` | 5,408 | 9,058 | | | `lg` | 4,715 | 9,215 | | -| `trf` | 187 | | 8,320 | +| `trf` | 187 | 1,158 | 8,320 | `trf` runs 29x slower than `sm` on the same CPU, and 25x faster on a T4 than on that VM's own -Xeon (336 words/s), so a GPU is a requirement rather than an optimization. The CPU tiers sit -within 15% of each other, so the tok2vec lookup is not the bottleneck; the parser and +Xeon (336 words/s), so a GPU is a requirement rather than an optimization. Even a 2 GB 940MX +gives 6.2x over its host CPU and fits batch 32 without running out of memory, though it needs +a `cu126` torch build: sm_50 kernels were dropped from the 2.8 `cu128`/`cu129` wheels. The CPU +tiers sit within 15% of each other, so the tok2vec lookup is not the bottleneck; the parser and lemmatizer are. Laptop spread is about 10% with thermal state. ## Named entity recognition diff --git a/docs/MODELS.md b/docs/MODELS.md index 8a9a33b..8d35a55 100644 --- a/docs/MODELS.md +++ b/docs/MODELS.md @@ -508,14 +508,20 @@ warmup discarded. Raw records in `metrics/throughput-*.json`. | `sm` | 5,484 | 10,235 | | | | `md` | 5,408 | 9,058 | | | | `lg` | 4,715 | 9,215 | | | -| `trf` | 187 | | 336 | 8,320 | +| `trf` | 187 | 1,158 | 336 | 8,320 | The CPU tiers sit within about 15% of each other, less than their vector-table sizes suggest, so the tok2vec lookup is not the bottleneck; the parser and lemmatizer are. Run-to-run spread on the laptop is roughly 10% either way with thermal state, and a background rsync halved every number, so treat small differences as noise. -`trf` is 29x slower than `sm` on the same CPU. The T4 column and the Xeon column come from -the same Colab VM, giving a clean 25x GPU speedup for the transformer. The 940MX column is -empty for `trf` because current PyTorch wheels dropped sm_50, so that GPU cannot run it at -all. +`trf` is 29x slower than `sm` on the same CPU. The T4 and Xeon columns come from the same Colab +VM, giving a clean 25x GPU speedup for the transformer. + +The 940MX runs `trf` at 1,158 words/s, 6.2x its host CPU, and fits batch 32 inside 2 GB without +running out of memory, so the GPU note in §3.4 that dismissed this card for transformer work +holds only for training, not inference. It does need a `cu126` build of torch: Maxwell sm_50 +kernels were dropped from the `cu128` and `cu129` wheels starting torch 2.8, and `pip install +torch` now resolves to one of those. `.venv-trf-gpu` pins `torch==2.7.1+cu126` for this reason, +and is kept separate from `.venv` because torch's pinned `nvidia-*` wheels would downgrade the +CUDA libraries cupy runs on there from 12.9 to 12.6. diff --git a/scripts/make_model_card.py b/scripts/make_model_card.py index aa5e6d6..49adcf7 100644 --- a/scripts/make_model_card.py +++ b/scripts/make_model_card.py @@ -114,15 +114,6 @@ def main(): for device, batch, wps in rows: lines.append(f"| {device} | {batch} | {wps:,.0f} |") lines.append("") - gpu = next((r for r in rows if r[0].startswith("gpu")), None) - cpu = next((r for r in rows if r[0].startswith("cpu")), None) - if gpu and cpu: - lines += [ - f"A transformer pipeline is GPU-bound: the T4 is {gpu[2] / cpu[2]:.0f}x the " - f"CPU on the same machine. On CPU this runs roughly 25x slower than the " - f"`sm`/`md`/`lg` tiers, which is the price of the accuracy below.", - "", - ] lines += ["## Sources", "", "| Source | Author | Licence |", "| --- | --- | --- |"] for s in meta.get("sources", []):