Fill in trf throughput on the 940MX

The 940MX does run trf, at 1,158 words/s with batch 32 inside 2 GB, 6.2x its host
CPU. The earlier claim that current PyTorch wheels cannot target sm_50 was only
half right: Maxwell kernels were dropped from the cu128 and cu129 builds starting
torch 2.8, which is what `pip install torch` now resolves to, but the cu126 build
of 2.7.1 still ships sm_50 and works.

.venv-trf-gpu pins torch==2.7.1+cu126 for this and stays separate from .venv,
whose cupy runs on nvidia-* 12.9 wheels that torch would downgrade to 12.6.

Drop the editorial sentence from the generated model card; the table states it.
This commit is contained in:
Mohamad Fazeli 2026-08-13 17:11:25 +03:30
parent 6fa8aa70b8
commit 4135142e72
Signed by: fazel
GPG Key ID: 05E227BF4D6736DE
4 changed files with 21 additions and 20 deletions

View File

@ -65,12 +65,14 @@ print(doc.ents) # (محمدرضا شجریان, مشهد)
| `sm` | ۵٬۴۸۴ | ۱۰٬۲۳۵ | |
| `md` | ۵٬۴۰۸ | ۹٬۰۵۸ | |
| `lg` | ۴٬۷۱۵ | ۹٬۲۱۵ | |
| `trf` | ۱۸۷ | | ۸٬۳۲۰ |
| `trf` | ۱۸۷ | ۱٬۱۵۸ | ۸٬۳۲۰ |
ردهٔ `trf` روی یک پردازنده حدود ۲۹ برابر کندتر از `sm` است، و روی T4 نسبت به پردازندهٔ همان
ماشین (۳۳۶ واژه بر ثانیه) ۲۵ برابر سریع‌تر، پس کارت گرافیک برای آن یک نیاز است نه بهینه‌سازی.
فاصلهٔ رده‌های پردازنده‌ای کمتر از ۱۵ درصد است، یعنی گلوگاه جست‌وجوی tok2vec نیست بلکه تجزیه‌گر
و واژه‌یاب است. پراکندگی اجراها روی لپ‌تاپ بسته به دمای دستگاه حدود ۱۰± درصد است.
حتی کارت ۲ گیگابایتی 940MX هم ۶٫۲ برابر سریع‌تر از پردازندهٔ همان دستگاه است و دستهٔ ۳۲ را بدون
کمبود حافظه اجرا می‌کند، اما به نسخهٔ `cu126` از torch نیاز دارد، چون هستهٔ sm_50 از نسخه‌های
`cu128`/`cu129` در ۲٫۸ حذف شده است. فاصلهٔ رده‌های پردازنده‌ای کمتر از ۱۵ درصد است، یعنی گلوگاه
جست‌وجوی tok2vec نیست بلکه تجزیه‌گر و واژه‌یاب است. پراکندگی اجراها روی لپ‌تاپ حدود ۱۰± درصد است.
گام‌های تبدیل پیکره، آموزش، ارزیابی و بسته‌بندی در [`project.yml`](project.yml) تعریف شده‌اند.
توضیح بیشتر دربارهٔ گزینش پیکره و پروانه‌ها در [`docs/MODELS.md`](docs/MODELS.md) و شرح انگلیسی

View File

@ -93,11 +93,13 @@ timing the pipe only, warmup discarded. Reproduce with
| `sm` | 5,484 | 10,235 | |
| `md` | 5,408 | 9,058 | |
| `lg` | 4,715 | 9,215 | |
| `trf` | 187 | | 8,320 |
| `trf` | 187 | 1,158 | 8,320 |
`trf` runs 29x slower than `sm` on the same CPU, and 25x faster on a T4 than on that VM's own
Xeon (336 words/s), so a GPU is a requirement rather than an optimization. The CPU tiers sit
within 15% of each other, so the tok2vec lookup is not the bottleneck; the parser and
Xeon (336 words/s), so a GPU is a requirement rather than an optimization. Even a 2 GB 940MX
gives 6.2x over its host CPU and fits batch 32 without running out of memory, though it needs
a `cu126` torch build: sm_50 kernels were dropped from the 2.8 `cu128`/`cu129` wheels. The CPU
tiers sit within 15% of each other, so the tok2vec lookup is not the bottleneck; the parser and
lemmatizer are. Laptop spread is about 10% with thermal state.
## Named entity recognition

View File

@ -508,14 +508,20 @@ warmup discarded. Raw records in `metrics/throughput-*.json`.
| `sm` | 5,484 | 10,235 | | |
| `md` | 5,408 | 9,058 | | |
| `lg` | 4,715 | 9,215 | | |
| `trf` | 187 | | 336 | 8,320 |
| `trf` | 187 | 1,158 | 336 | 8,320 |
The CPU tiers sit within about 15% of each other, less than their vector-table sizes suggest,
so the tok2vec lookup is not the bottleneck; the parser and lemmatizer are. Run-to-run spread
on the laptop is roughly 10% either way with thermal state, and a background rsync halved
every number, so treat small differences as noise.
`trf` is 29x slower than `sm` on the same CPU. The T4 column and the Xeon column come from
the same Colab VM, giving a clean 25x GPU speedup for the transformer. The 940MX column is
empty for `trf` because current PyTorch wheels dropped sm_50, so that GPU cannot run it at
all.
`trf` is 29x slower than `sm` on the same CPU. The T4 and Xeon columns come from the same Colab
VM, giving a clean 25x GPU speedup for the transformer.
The 940MX runs `trf` at 1,158 words/s, 6.2x its host CPU, and fits batch 32 inside 2 GB without
running out of memory, so the GPU note in §3.4 that dismissed this card for transformer work
holds only for training, not inference. It does need a `cu126` build of torch: Maxwell sm_50
kernels were dropped from the `cu128` and `cu129` wheels starting torch 2.8, and `pip install
torch` now resolves to one of those. `.venv-trf-gpu` pins `torch==2.7.1+cu126` for this reason,
and is kept separate from `.venv` because torch's pinned `nvidia-*` wheels would downgrade the
CUDA libraries cupy runs on there from 12.9 to 12.6.

View File

@ -114,15 +114,6 @@ def main():
for device, batch, wps in rows:
lines.append(f"| {device} | {batch} | {wps:,.0f} |")
lines.append("")
gpu = next((r for r in rows if r[0].startswith("gpu")), None)
cpu = next((r for r in rows if r[0].startswith("cpu")), None)
if gpu and cpu:
lines += [
f"A transformer pipeline is GPU-bound: the T4 is {gpu[2] / cpu[2]:.0f}x the "
f"CPU on the same machine. On CPU this runs roughly 25x slower than the "
f"`sm`/`md`/`lg` tiers, which is the price of the accuracy below.",
"",
]
lines += ["## Sources", "", "| Source | Author | Licence |", "| --- | --- | --- |"]
for s in meta.get("sources", []):