README: dedicated NER section, drop AI-tell formatting

- New ## Named entity recognition section: labels, silver-provenance,
  per-label breakdown table, package options. Moved out of ## Results
  (which now just points to it) and out of ## Caveats (silver-label and
  thin-label bullets folded into the new section's prose instead of
  repeating them).
- Added fa_ent_news_sm to the package table (built, was undocumented).
- Design decision 4 no longer repeats the NER transfer methodology now
  that it has a dedicated home.
- Dropped the warning-emoji decoration on the comparability note.
This commit is contained in:
Mohamad Fazeli 2026-08-12 13:49:05 +03:30
parent d39c09daca
commit c3cb02d9c3
Signed by: fazel
GPG Key ID: 05E227BF4D6736DE
1 changed files with 27 additions and 20 deletions

View File

@ -36,7 +36,7 @@ Compared against Hazm (the most-used Persian toolkit) and `en_core_web_sm` (Engl
> **¹** Hazm scores from its official README
> **²** `en_core_web_sm` scores from spaCy's official model card
> ⚠️ **Note on comparability:** These benchmarks come from *different evaluation sets, treebanks, and test splits*.
> **Note on comparability:** These benchmarks come from *different evaluation sets, treebanks, and test splits*.
From `spacy benchmark accuracy`, stored in `metrics/`.
@ -44,6 +44,7 @@ From `spacy benchmark accuracy`, stored in `metrics/`.
| --- | --- | --- | --- | --- |
| `fa_dep_news_sm` | tok2vec, tagger, morphologizer, trainable_lemmatizer, parser | CC BY-SA 4.0 | LEMMA 97.91 | 7.5 MB |
| `fa_core_news_sm` | the above plus ner | CC BY-SA 4.0 | ENTS_F 71.87 | 13 MB |
| `fa_ent_news_sm` | `ner` alone (own embedded tok2vec) | CC BY-SA 4.0 | ENTS_F 71.87 | 5.6 MB |
| `fa_dep_news_md` | same as `fa_dep_news_sm`, plus floret vectors | CC BY-SA 4.0 | LEMMA 97.96 | 62 MB |
| `fa_core_news_md` | same as `fa_core_news_sm`, plus floret vectors | CC BY-SA 4.0 | ENTS_F 74.71 | 68 MB |
| `fa_ent_news_md` | `ner` alone (own embedded tok2vec), plus floret vectors | CC BY-SA 4.0 | ENTS_F 74.71 | 58 MB |
@ -67,9 +68,26 @@ isolate what the vectors buy. Full breakdown in `docs/MODELS.md` §6.
| `ENTS_F` | 71.87 | 74.71 | |
| Speed | ~9,250 words/s | ~7,700 words/s | |
Entity scores are `fa_core_news_*` on the PerDT NER test split. The `md` gain is almost
entirely recall (+6.08): static vectors give the model a lexical prior for rare proper nouns
that hash embeddings never had.
Entity scores are `fa_core_news_*` on the PerDT NER test split; per-label breakdown and
caveats are in [Named entity recognition](#named-entity-recognition).
For comparison, `en_core_web_sm` scores TAG 97, LAS 90, ENTS_F 84 on a larger, cleaner corpus.
Trained on a 4-core i5-7200U with no GPU: `sm` 1h27m syntax + 17 min NER, `md` 1h54m syntax
+ 25 min NER (the two `md` runs overlapped, so wall clock overstates each).
## Named entity recognition
Seven labels: `LOC`, `PER`, `ORG`, `DAT`, `MON`, `TIM`, `PCT`. They come from PerDT's own
`not-to-release/Dadegan with NER tag/` layer, transferred onto this pipeline's tokenization
by difflib at a 99.86% alignment rate; spans that could not be aligned exactly were dropped
rather than guessed (`scripts/transfer_perdt_ner.py`). That layer is silver: PerDT's README
states it was produced by the BERT-based Beheshti-NER tagger with manual corrections for
recall, so the `ENTS_F` numbers below partly reflect agreement with that tagger, not with
human annotation.
`ner` runs standalone with its own embedded tok2vec (`fa_ent_news_sm`, `fa_ent_news_md`), or
bundled into `fa_core_news_sm`/`fa_core_news_md` alongside the syntax pipeline.
| Label | Gold in test | `sm` F | `md` F | Train examples |
| --- | --- | --- | --- | --- |
@ -82,14 +100,12 @@ that hash embeddings never had.
| `PCT` | 4 | 57.14 | 33.33 | 121 |
`MON`, `TIM` and `PCT` have single-digit support in the test split, so their deltas are one
or two entities changing hands, not signal. The three labels that carry the split (`PER`,
`LOC`, `ORG`) all improve.
or two entities changing hands, not signal. `PER`, `LOC` and `ORG` carry the split and all
improve with floret vectors; the `md` gain over `sm` (`ENTS_F` 71.87 to 74.71) is almost
entirely recall (+6.08), the lexical prior static vectors give rare proper nouns that hash
embeddings never had.
For comparison, `en_core_web_sm` scores TAG 97, LAS 90, ENTS_F 84 on a larger, cleaner corpus.
Trained on a 4-core i5-7200U with no GPU: `sm` 1h27m syntax + 17 min NER, `md` 1h54m syntax
+ 25 min NER (the two `md` runs overlapped, so wall clock overstates each).
## Install
```bash
@ -100,12 +116,6 @@ pip install https://huggingface.co/Phazel/fa_dep_news_sm/resolve/main/fa_dep_new
## Caveats
- **The entity labels are silver.** They come from the treebank's own
`not-to-release/Dadegan with NER tag/` layer, which its README states was produced by the
BERT-based Beheshti-NER tagger with manual corrections for recall. `ENTS_F 71.87` is measured
against a silver test split and partly reflects agreement with that tagger.
- **Three entity labels are thin.** `MON` (205 training examples), `TIM` (135) and `PCT` (121)
rest on 4 to 11 test entities each. `PER`, `LOC`, `ORG` and `DAT` have 1,300 or more.
- **Some lemmas contain a space.** Multiword tokens were merged, so `کتاب‌هایش` is one token
tagged `N_IANM_PR_JOPER` with lemma `کتاب او`. This affects about 1.5% of tokens.
- **`doc.noun_chunks` under-fires.** `spacy/lang/fa/syntax_iterators.py` upstream matches
@ -159,10 +169,7 @@ The two training runs are single-threaded and independent, so they can run concu
English pipelines derive UPOS from PTB tags by rule because OntoNotes has no UPOS. UD gives
gold UPOS, FEATS and lemmas, which yields real `pos_acc`, `morph_acc` and `lemma_acc` numbers
instead of unmeasurable rule coverage.
4. PerDT, not Seraji: 3.7x more tokens, and Seraji has no `PROPN` tag. Entity spans were
transferred onto this pipeline's tokenization by difflib at a 99.86% rate, and spans that
could not be aligned exactly were dropped rather than guessed
(`scripts/transfer_perdt_ner.py`).
4. PerDT, not Seraji: 3.7x more tokens, and Seraji has no `PROPN` tag.
## Why not hazm's own models