Fill in trf throughput on the 940MX
The 940MX does run trf, at 1,158 words/s with batch 32 inside 2 GB, 6.2x its host CPU. The earlier claim that current PyTorch wheels cannot target sm_50 was only half right: Maxwell kernels were dropped from the cu128 and cu129 builds starting torch 2.8, which is what `pip install torch` now resolves to, but the cu126 build of 2.7.1 still ships sm_50 and works. .venv-trf-gpu pins torch==2.7.1+cu126 for this and stays separate from .venv, whose cupy runs on nvidia-* 12.9 wheels that torch would downgrade to 12.6. Drop the editorial sentence from the generated model card; the table states it.
This commit is contained in:
parent
6fa8aa70b8
commit
4135142e72
|
|
@ -65,12 +65,14 @@ print(doc.ents) # (محمدرضا شجریان, مشهد)
|
||||||
| `sm` | ۵٬۴۸۴ | ۱۰٬۲۳۵ | |
|
| `sm` | ۵٬۴۸۴ | ۱۰٬۲۳۵ | |
|
||||||
| `md` | ۵٬۴۰۸ | ۹٬۰۵۸ | |
|
| `md` | ۵٬۴۰۸ | ۹٬۰۵۸ | |
|
||||||
| `lg` | ۴٬۷۱۵ | ۹٬۲۱۵ | |
|
| `lg` | ۴٬۷۱۵ | ۹٬۲۱۵ | |
|
||||||
| `trf` | ۱۸۷ | | ۸٬۳۲۰ |
|
| `trf` | ۱۸۷ | ۱٬۱۵۸ | ۸٬۳۲۰ |
|
||||||
|
|
||||||
ردهٔ `trf` روی یک پردازنده حدود ۲۹ برابر کندتر از `sm` است، و روی T4 نسبت به پردازندهٔ همان
|
ردهٔ `trf` روی یک پردازنده حدود ۲۹ برابر کندتر از `sm` است، و روی T4 نسبت به پردازندهٔ همان
|
||||||
ماشین (۳۳۶ واژه بر ثانیه) ۲۵ برابر سریعتر، پس کارت گرافیک برای آن یک نیاز است نه بهینهسازی.
|
ماشین (۳۳۶ واژه بر ثانیه) ۲۵ برابر سریعتر، پس کارت گرافیک برای آن یک نیاز است نه بهینهسازی.
|
||||||
فاصلهٔ ردههای پردازندهای کمتر از ۱۵ درصد است، یعنی گلوگاه جستوجوی tok2vec نیست بلکه تجزیهگر
|
حتی کارت ۲ گیگابایتی 940MX هم ۶٫۲ برابر سریعتر از پردازندهٔ همان دستگاه است و دستهٔ ۳۲ را بدون
|
||||||
و واژهیاب است. پراکندگی اجراها روی لپتاپ بسته به دمای دستگاه حدود ۱۰± درصد است.
|
کمبود حافظه اجرا میکند، اما به نسخهٔ `cu126` از torch نیاز دارد، چون هستهٔ sm_50 از نسخههای
|
||||||
|
`cu128`/`cu129` در ۲٫۸ حذف شده است. فاصلهٔ ردههای پردازندهای کمتر از ۱۵ درصد است، یعنی گلوگاه
|
||||||
|
جستوجوی tok2vec نیست بلکه تجزیهگر و واژهیاب است. پراکندگی اجراها روی لپتاپ حدود ۱۰± درصد است.
|
||||||
|
|
||||||
گامهای تبدیل پیکره، آموزش، ارزیابی و بستهبندی در [`project.yml`](project.yml) تعریف شدهاند.
|
گامهای تبدیل پیکره، آموزش، ارزیابی و بستهبندی در [`project.yml`](project.yml) تعریف شدهاند.
|
||||||
توضیح بیشتر دربارهٔ گزینش پیکره و پروانهها در [`docs/MODELS.md`](docs/MODELS.md) و شرح انگلیسی
|
توضیح بیشتر دربارهٔ گزینش پیکره و پروانهها در [`docs/MODELS.md`](docs/MODELS.md) و شرح انگلیسی
|
||||||
|
|
|
||||||
|
|
@ -93,11 +93,13 @@ timing the pipe only, warmup discarded. Reproduce with
|
||||||
| `sm` | 5,484 | 10,235 | |
|
| `sm` | 5,484 | 10,235 | |
|
||||||
| `md` | 5,408 | 9,058 | |
|
| `md` | 5,408 | 9,058 | |
|
||||||
| `lg` | 4,715 | 9,215 | |
|
| `lg` | 4,715 | 9,215 | |
|
||||||
| `trf` | 187 | | 8,320 |
|
| `trf` | 187 | 1,158 | 8,320 |
|
||||||
|
|
||||||
`trf` runs 29x slower than `sm` on the same CPU, and 25x faster on a T4 than on that VM's own
|
`trf` runs 29x slower than `sm` on the same CPU, and 25x faster on a T4 than on that VM's own
|
||||||
Xeon (336 words/s), so a GPU is a requirement rather than an optimization. The CPU tiers sit
|
Xeon (336 words/s), so a GPU is a requirement rather than an optimization. Even a 2 GB 940MX
|
||||||
within 15% of each other, so the tok2vec lookup is not the bottleneck; the parser and
|
gives 6.2x over its host CPU and fits batch 32 without running out of memory, though it needs
|
||||||
|
a `cu126` torch build: sm_50 kernels were dropped from the 2.8 `cu128`/`cu129` wheels. The CPU
|
||||||
|
tiers sit within 15% of each other, so the tok2vec lookup is not the bottleneck; the parser and
|
||||||
lemmatizer are. Laptop spread is about 10% with thermal state.
|
lemmatizer are. Laptop spread is about 10% with thermal state.
|
||||||
|
|
||||||
## Named entity recognition
|
## Named entity recognition
|
||||||
|
|
|
||||||
|
|
@ -508,14 +508,20 @@ warmup discarded. Raw records in `metrics/throughput-*.json`.
|
||||||
| `sm` | 5,484 | 10,235 | | |
|
| `sm` | 5,484 | 10,235 | | |
|
||||||
| `md` | 5,408 | 9,058 | | |
|
| `md` | 5,408 | 9,058 | | |
|
||||||
| `lg` | 4,715 | 9,215 | | |
|
| `lg` | 4,715 | 9,215 | | |
|
||||||
| `trf` | 187 | | 336 | 8,320 |
|
| `trf` | 187 | 1,158 | 336 | 8,320 |
|
||||||
|
|
||||||
The CPU tiers sit within about 15% of each other, less than their vector-table sizes suggest,
|
The CPU tiers sit within about 15% of each other, less than their vector-table sizes suggest,
|
||||||
so the tok2vec lookup is not the bottleneck; the parser and lemmatizer are. Run-to-run spread
|
so the tok2vec lookup is not the bottleneck; the parser and lemmatizer are. Run-to-run spread
|
||||||
on the laptop is roughly 10% either way with thermal state, and a background rsync halved
|
on the laptop is roughly 10% either way with thermal state, and a background rsync halved
|
||||||
every number, so treat small differences as noise.
|
every number, so treat small differences as noise.
|
||||||
|
|
||||||
`trf` is 29x slower than `sm` on the same CPU. The T4 column and the Xeon column come from
|
`trf` is 29x slower than `sm` on the same CPU. The T4 and Xeon columns come from the same Colab
|
||||||
the same Colab VM, giving a clean 25x GPU speedup for the transformer. The 940MX column is
|
VM, giving a clean 25x GPU speedup for the transformer.
|
||||||
empty for `trf` because current PyTorch wheels dropped sm_50, so that GPU cannot run it at
|
|
||||||
all.
|
The 940MX runs `trf` at 1,158 words/s, 6.2x its host CPU, and fits batch 32 inside 2 GB without
|
||||||
|
running out of memory, so the GPU note in §3.4 that dismissed this card for transformer work
|
||||||
|
holds only for training, not inference. It does need a `cu126` build of torch: Maxwell sm_50
|
||||||
|
kernels were dropped from the `cu128` and `cu129` wheels starting torch 2.8, and `pip install
|
||||||
|
torch` now resolves to one of those. `.venv-trf-gpu` pins `torch==2.7.1+cu126` for this reason,
|
||||||
|
and is kept separate from `.venv` because torch's pinned `nvidia-*` wheels would downgrade the
|
||||||
|
CUDA libraries cupy runs on there from 12.9 to 12.6.
|
||||||
|
|
|
||||||
|
|
@ -114,15 +114,6 @@ def main():
|
||||||
for device, batch, wps in rows:
|
for device, batch, wps in rows:
|
||||||
lines.append(f"| {device} | {batch} | {wps:,.0f} |")
|
lines.append(f"| {device} | {batch} | {wps:,.0f} |")
|
||||||
lines.append("")
|
lines.append("")
|
||||||
gpu = next((r for r in rows if r[0].startswith("gpu")), None)
|
|
||||||
cpu = next((r for r in rows if r[0].startswith("cpu")), None)
|
|
||||||
if gpu and cpu:
|
|
||||||
lines += [
|
|
||||||
f"A transformer pipeline is GPU-bound: the T4 is {gpu[2] / cpu[2]:.0f}x the "
|
|
||||||
f"CPU on the same machine. On CPU this runs roughly 25x slower than the "
|
|
||||||
f"`sm`/`md`/`lg` tiers, which is the price of the accuracy below.",
|
|
||||||
"",
|
|
||||||
]
|
|
||||||
|
|
||||||
lines += ["## Sources", "", "| Source | Author | Licence |", "| --- | --- | --- |"]
|
lines += ["## Sources", "", "| Source | Author | Licence |", "| --- | --- | --- |"]
|
||||||
for s in meta.get("sources", []):
|
for s in meta.get("sources", []):
|
||||||
|
|
|
||||||
Loading…
Reference in New Issue