diff --git a/configs/fa_ner_sm.cfg b/configs/fa_ner_sm.cfg index 8e6f209..d32bc43 100644 --- a/configs/fa_ner_sm.cfg +++ b/configs/fa_ner_sm.cfg @@ -1,10 +1,11 @@ # fa_ent_news_sm — Persian NER, CPU size (sm). # -# A standalone artifact, not part of the shipping `dep` pipeline. It is trained on -# ParsTwiNER (MIT, tweets) while the rest of the project trains on UD_Persian-PerDT -# (CC BY-SA 4.0, edited prose), and it scores 67.22 F against 85-98 for the UD -# components — different corpus, different genre, different quality tier, so it gets its -# own package rather than being hidden inside a `core` one. Built by `spacy project run ner`. +# Standalone package, built by `spacy project run ner`, sitting alongside (not inside) +# fa_dep_news_sm. Trained on the PerDT treebank's own NER layer (CC BY-SA 4.0, edited +# prose), transferred onto this project's tokenization by scripts/transfer_perdt_ner.py. +# Scores ENTS_F 71.87 on the PerDT test split; see docs/MODELS.md §3.2 for the corpus +# survey (ParsTwiNER, ARMAN, PEYMA and others were considered and rejected on licence or +# genre grounds). # # Deliberate deviation from `spacy init config --pipeline ner`: the tok2vec is # EMBEDDED inside components.ner.model instead of being a separate `tok2vec` @@ -12,8 +13,8 @@ # it was trained in; embedding makes the component self-contained and therefore # sourceable into another pipeline via `nlp.add_pipe("ner", source=...)`. This is the same # design as en_core_web_sm, whose `ner` "has its own independent internal tok2vec" -# (https://spacy.io/models#design). That is what will let a future fa_core_news_sm combine -# this component (or its ../ner_dataset replacement) with the dep pipeline. +# (https://spacy.io/models#design). `assemble-core` in project.yml uses exactly that to +# source this component into fa_core_news_sm alongside the dep pipeline. [paths] train = null