Baseline models
There are non-empty training data for 73 treebanks. (Note: The link has been updated and now points to the official release 2.2 of Universal Dependencies, which also contains the test data and some non-shared-task treebanks.) Most of the treebanks also have development data but some do not. If there are no development data, we cut a small portion of the training data and call it dev data for the purpose of tuning hyperparameters of the baseline models. We only use the remainder of the original training data for the actual training.
In addition to the treebank-specific models, there is also a “mixed model”, trained on samples from all treebanks. The first 200 training sentences of each treebank are included in the mixed training data. If there are fewer than 200 sentences (after a portion has been reserved to be used as dev data), all training sentences are included. Hyperparameters are tuned on mixed development data, consisting of at most 20 sentences from each treebank’s dev data. The mixed model is used as a baseline solution for languages where no training data are available.
The baseline models were trained using UDPipe 1.2 and they cover all levels of processing: tokenization, word segmentation, morphology (UPOS tags, features, lemmas) and syntax.
During the shared task, the baseline models were downloadable from a temporary site. After the shared task, they were published in Lindat. Besides the actual models the package also contains information needed to replicate them: the hyperparameters, the modified train-dev split where applicable, and pre-computed word embeddings for the parser.
Preprocessed development and test data on TIRA
By default, the participating systems are expected to process untokenized text, i.e., they must take care of all levels of processing (tokenization, morphology, and syntax). To make the task easier for teams that only want to focus on syntax, we provide input files preprocessed by UDPipe with the baseline models. All test sets and all official dev sets (where they exist) will be available in TIRA as CoNLL-U files with predicted tokenization, word segmentation, sentence segmentation, lemmas, UPOS tags and features (but not syntactic relations).
For treebanks that have non-empty training data (including treebanks whose training set is very small), the preprocessing will be done using the baseline model trained on that treebank (see above). For the nine treebanks that do not contain training data, a substitute model will be used:
- Breton KEB ← mixed model
- Czech PUD ← Czech PDT
- English PUD ← English EWT
- Faroese OFT ← mixed model
- Finnish PUD ← Finnish TDT
- Japanese Modern ← Japanese GSD
- Naija NSC ← mixed model
- Swedish PUD ← Swedish Talbanken
- Thai PUD ← mixed model
Preprocessed morphology in training and development data
Some participants may want to train their parser on data with predicted morphology rather than the gold standard. To facilitate that, we provide a version of the training and development data where morphology (lemmas, UPOS, and features) was predicted by UDPipe. Syntactic annotation is manual. Note that word and sentence segmentation must be manual too, because the syntactic relations depend on it.
Morphology in development data is predicted using the baseline models (see above). Morphology in training data is predicted via “jack-knifing” (split the training set into 10 parts, train a model on 9 parts, use it to predict morphology in the tenth part, repeat for all 10 target parts). Same hyperparameters are used as those used to train the baseline model on the entire training set.
The data with predicted morphology can be downloaded from Lindat.
Baseline system on TIRA
The baseline system will consist of UDPipe 1.2 and the baseline models (see above). For the nine treebanks that do not have their own training data, a substitution model will be used (the same substitution as for preprocessing the dev/test data, see the table above).
Baseline results
Development data
The official evaluation of the baseline system after it had been run on Tira with the development data as input:
af_afribooms LAS= 80.19% MLAS= 65.98% BLEX= 70.40% (OK: Result F1 scores rounded to 5% are LAS=80% MLAS=65% BLEX=70%) grc_perseus LAS= 57.89% MLAS= 30.80% BLEX= 40.49% (OK: Result F1 scores rounded to 5% are LAS=60% MLAS=30% BLEX=40%) grc_proiel LAS= 69.13% MLAS= 52.42% BLEX= 57.92% (OK: Result F1 scores rounded to 5% are LAS=70% MLAS=50% BLEX=60%) ar_padt LAS= 66.81% MLAS= 55.67% BLEX= 57.90% (OK: Result F1 scores rounded to 5% are LAS=65% MLAS=55% BLEX=60%) eu_bdt LAS= 70.06% MLAS= 57.46% BLEX= 63.39% (OK: Result F1 scores rounded to 5% are LAS=70% MLAS=55% BLEX=65%) bg_btb LAS= 84.67% MLAS= 74.54% BLEX= 73.78% (OK: Result F1 scores rounded to 5% are LAS=85% MLAS=75% BLEX=75%) ca_ancora LAS= 85.63% MLAS= 77.04% BLEX= 77.56% (OK: Result F1 scores rounded to 5% are LAS=85% MLAS=75% BLEX=80%) hr_set LAS= 77.84% MLAS= 59.60% BLEX= 69.99% (OK: Result F1 scores rounded to 5% are LAS=80% MLAS=60% BLEX=70%) cs_cac LAS= 84.42% MLAS= 72.17% BLEX= 78.29% (OK: Result F1 scores rounded to 5% are LAS=85% MLAS=70% BLEX=80%) cs_fictree LAS= 83.16% MLAS= 70.72% BLEX= 75.80% (OK: Result F1 scores rounded to 5% are LAS=85% MLAS=70% BLEX=75%) cs_pdt LAS= 84.85% MLAS= 75.35% BLEX= 80.55% (OK: Result F1 scores rounded to 5% are LAS=85% MLAS=75% BLEX=80%) da_ddt LAS= 75.16% MLAS= 65.29% BLEX= 66.07% (OK: Result F1 scores rounded to 5% are LAS=75% MLAS=65% BLEX=65%) nl_alpino LAS= 80.21% MLAS= 67.14% BLEX= 69.77% (OK: Result F1 scores rounded to 5% are LAS=80% MLAS=65% BLEX=70%) nl_lassysmall LAS= 73.61% MLAS= 59.99% BLEX= 61.71% (OK: Result F1 scores rounded to 5% are LAS=75% MLAS=60% BLEX=60%) en_ewt LAS= 77.62% MLAS= 68.58% BLEX= 70.98% (OK: Result F1 scores rounded to 5% are LAS=80% MLAS=70% BLEX=70%) en_gum LAS= 76.63% MLAS= 65.57% BLEX= 67.20% (OK: Result F1 scores rounded to 5% are LAS=75% MLAS=65% BLEX=65%) en_lines LAS= 75.78% MLAS= 66.29% BLEX= 68.57% (OK: Result F1 scores rounded to 5% are LAS=75% MLAS=65% BLEX=70%) et_edt LAS= 76.50% MLAS= 68.27% BLEX= 64.17% (OK: Result F1 scores rounded to 5% are LAS=75% MLAS=70% BLEX=65%) fi_ftb LAS= 75.76% MLAS= 65.72% BLEX= 62.68% (OK: Result F1 scores rounded to 5% are LAS=75% MLAS=65% BLEX=65%) fi_tdt LAS= 76.39% MLAS= 68.60% BLEX= 62.33% (OK: Result F1 scores rounded to 5% are LAS=75% MLAS=70% BLEX=60%) fr_gsd LAS= 85.81% MLAS= 77.80% BLEX= 79.16% (OK: Result F1 scores rounded to 5% are LAS=85% MLAS=80% BLEX=80%) fr_sequoia LAS= 82.72% MLAS= 74.13% BLEX= 76.34% (OK: Result F1 scores rounded to 5% are LAS=85% MLAS=75% BLEX=75%) fr_spoken LAS= 65.09% MLAS= 54.00% BLEX= 55.42% (OK: Result F1 scores rounded to 5% are LAS=65% MLAS=55% BLEX=55%) gl_ctg LAS= 76.32% MLAS= 62.58% BLEX= 65.57% (OK: Result F1 scores rounded to 5% are LAS=75% MLAS=65% BLEX=65%) de_gsd LAS= 75.55% MLAS= 38.52% BLEX= 65.39% (OK: Result F1 scores rounded to 5% are LAS=75% MLAS=40% BLEX=65%) got_proiel LAS= 62.03% MLAS= 48.16% BLEX= 54.39% (OK: Result F1 scores rounded to 5% are LAS=60% MLAS=50% BLEX=55%) el_gdt LAS= 81.37% MLAS= 63.92% BLEX= 65.21% (OK: Result F1 scores rounded to 5% are LAS=80% MLAS=65% BLEX=65%) he_htb LAS= 61.95% MLAS= 49.28% BLEX= 51.45% (OK: Result F1 scores rounded to 5% are LAS=60% MLAS=50% BLEX=50%) hi_hdtb LAS= 87.26% MLAS= 69.78% BLEX= 80.59% (OK: Result F1 scores rounded to 5% are LAS=85% MLAS=70% BLEX=80%) hu_szeged LAS= 68.41% MLAS= 56.47% BLEX= 60.17% (OK: Result F1 scores rounded to 5% are LAS=70% MLAS=55% BLEX=60%) zh_gsd LAS= 57.39% MLAS= 48.19% BLEX= 52.84% (OK: Result F1 scores rounded to 5% are LAS=55% MLAS=50% BLEX=55%) id_gsd LAS= 74.40% MLAS= 63.51% BLEX= 63.29% (OK: Result F1 scores rounded to 5% are LAS=75% MLAS=65% BLEX=65%) it_isdt LAS= 85.95% MLAS= 77.20% BLEX= 77.37% (OK: Result F1 scores rounded to 5% are LAS=85% MLAS=75% BLEX=75%) it_postwita LAS= 65.85% MLAS= 52.14% BLEX= 52.90% (OK: Result F1 scores rounded to 5% are LAS=65% MLAS=50% BLEX=55%) ja_gsd LAS= 75.48% MLAS= 62.39% BLEX= 64.58% (OK: Result F1 scores rounded to 5% are LAS=75% MLAS=60% BLEX=65%) ko_gsd LAS= 57.25% MLAS= 49.06% BLEX= 44.24% (OK: Result F1 scores rounded to 5% are LAS=55% MLAS=50% BLEX=45%) ko_kaist LAS= 71.00% MLAS= 63.32% BLEX= 59.19% (OK: Result F1 scores rounded to 5% are LAS=70% MLAS=65% BLEX=60%) la_ittb LAS= 73.23% MLAS= 59.94% BLEX= 67.43% (OK: Result F1 scores rounded to 5% are LAS=75% MLAS=60% BLEX=65%) la_proiel LAS= 61.33% MLAS= 48.40% BLEX= 55.10% (OK: Result F1 scores rounded to 5% are LAS=60% MLAS=50% BLEX=55%) lv_lvtb LAS= 70.67% MLAS= 57.79% BLEX= 60.96% (OK: Result F1 scores rounded to 5% are LAS=70% MLAS=60% BLEX=60%) no_bokmaal LAS= 84.56% MLAS= 75.95% BLEX= 78.04% (OK: Result F1 scores rounded to 5% are LAS=85% MLAS=75% BLEX=80%) no_nynorsk LAS= 82.75% MLAS= 73.88% BLEX= 75.76% (OK: Result F1 scores rounded to 5% are LAS=85% MLAS=75% BLEX=75%) fro_srcmf LAS= 79.15% MLAS= 70.43% BLEX= 74.27% (OK: Result F1 scores rounded to 5% are LAS=80% MLAS=70% BLEX=75%) cu_proiel LAS= 66.12% MLAS= 54.48% BLEX= 59.16% (OK: Result F1 scores rounded to 5% are LAS=65% MLAS=55% BLEX=60%) fa_seraji LAS= 79.78% MLAS= 73.03% BLEX= 73.35% (OK: Result F1 scores rounded to 5% are LAS=80% MLAS=75% BLEX=75%) pl_lfg LAS= 88.79% MLAS= 75.15% BLEX= 79.18% (OK: Result F1 scores rounded to 5% are LAS=90% MLAS=75% BLEX=80%) pl_sz LAS= 82.65% MLAS= 63.92% BLEX= 72.59% (OK: Result F1 scores rounded to 5% are LAS=85% MLAS=65% BLEX=75%) pt_bosque LAS= 84.93% MLAS= 73.22% BLEX= 76.02% (OK: Result F1 scores rounded to 5% are LAS=85% MLAS=75% BLEX=75%) ro_rrt LAS= 80.32% MLAS= 71.21% BLEX= 71.82% (OK: Result F1 scores rounded to 5% are LAS=80% MLAS=70% BLEX=70%) ru_syntagrus LAS= 83.87% MLAS= 75.78% BLEX= 77.27% (OK: Result F1 scores rounded to 5% are LAS=85% MLAS=75% BLEX=75%) sr_set LAS= 82.12% MLAS= 69.12% BLEX= 73.06% (OK: Result F1 scores rounded to 5% are LAS=80% MLAS=70% BLEX=75%) sk_snk LAS= 75.73% MLAS= 54.34% BLEX= 59.71% (OK: Result F1 scores rounded to 5% are LAS=75% MLAS=55% BLEX=60%) sl_ssj LAS= 77.72% MLAS= 63.96% BLEX= 68.97% (OK: Result F1 scores rounded to 5% are LAS=80% MLAS=65% BLEX=70%) es_ancora LAS= 85.08% MLAS= 76.81% BLEX= 77.48% (OK: Result F1 scores rounded to 5% are LAS=85% MLAS=75% BLEX=75%) sv_lines LAS= 76.23% MLAS= 62.16% BLEX= 67.63% (OK: Result F1 scores rounded to 5% are LAS=75% MLAS=60% BLEX=70%) sv_talbanken LAS= 75.39% MLAS= 66.87% BLEX= 68.25% (OK: Result F1 scores rounded to 5% are LAS=75% MLAS=65% BLEX=70%) tr_imst LAS= 54.83% MLAS= 44.25% BLEX= 45.81% (OK: Result F1 scores rounded to 5% are LAS=55% MLAS=45% BLEX=45%) uk_iu LAS= 77.94% MLAS= 59.66% BLEX= 68.07% (OK: Result F1 scores rounded to 5% are LAS=80% MLAS=60% BLEX=70%) ur_udtb LAS= 77.44% MLAS= 49.91% BLEX= 63.55% (OK: Result F1 scores rounded to 5% are LAS=75% MLAS=50% BLEX=65%) ug_udt LAS= 56.88% MLAS= 37.43% BLEX= 43.34% (OK: Result F1 scores rounded to 5% are LAS=55% MLAS=35% BLEX=45%) vi_vtb LAS= 43.65% MLAS= 37.39% BLEX= 39.18% (OK: Result F1 scores rounded to 5% are LAS=45% MLAS=35% BLEX=40%)
Test data
The official evaluation of the baseline system after it had been run on Tira with the test data as input:
af_afribooms LAS= 77.88% MLAS= 64.48% BLEX= 66.60% (OK: Result F1 scores rounded to 5% are LAS=80% MLAS=65% BLEX=65%) grc_perseus LAS= 57.75% MLAS= 31.05% BLEX= 38.74% (OK: Result F1 scores rounded to 5% are LAS=60% MLAS=30% BLEX=40%) grc_proiel LAS= 67.57% MLAS= 49.51% BLEX= 55.85% (OK: Result F1 scores rounded to 5% are LAS=70% MLAS=50% BLEX=55%) ar_padt LAS= 66.41% MLAS= 55.01% BLEX= 57.60% (OK: Result F1 scores rounded to 5% are LAS=65% MLAS=55% BLEX=60%) hy_armtdp LAS= 21.79% MLAS= 6.84% BLEX= 11.94% (OK: Result F1 scores rounded to 5% are LAS=20% MLAS=5% BLEX=10%) eu_bdt LAS= 70.13% MLAS= 57.65% BLEX= 63.50% (OK: Result F1 scores rounded to 5% are LAS=70% MLAS=60% BLEX=65%) br_keb LAS= 10.25% MLAS= 0.37% BLEX= 2.10% (OK: Result F1 scores rounded to 5% are LAS=10% MLAS=0% BLEX=0%) bg_btb LAS= 84.91% MLAS= 75.30% BLEX= 73.78% (OK: Result F1 scores rounded to 5% are LAS=85% MLAS=75% BLEX=75%) bxr_bdt LAS= 12.61% MLAS= 2.09% BLEX= 4.41% (OK: Result F1 scores rounded to 5% are LAS=15% MLAS=0% BLEX=5%) ca_ancora LAS= 85.61% MLAS= 76.74% BLEX= 77.27% (OK: Result F1 scores rounded to 5% are LAS=85% MLAS=75% BLEX=75%) hr_set LAS= 78.61% MLAS= 58.72% BLEX= 70.26% (OK: Result F1 scores rounded to 5% are LAS=80% MLAS=60% BLEX=70%) cs_cac LAS= 83.72% MLAS= 70.89% BLEX= 77.65% (OK: Result F1 scores rounded to 5% are LAS=85% MLAS=70% BLEX=80%) cs_fictree LAS= 82.49% MLAS= 69.26% BLEX= 74.96% (OK: Result F1 scores rounded to 5% are LAS=80% MLAS=70% BLEX=75%) cs_pdt LAS= 83.94% MLAS= 74.32% BLEX= 79.39% (OK: Result F1 scores rounded to 5% are LAS=85% MLAS=75% BLEX=80%) cs_pud LAS= 80.08% MLAS= 66.53% BLEX= 73.79% (OK: Result F1 scores rounded to 5% are LAS=80% MLAS=65% BLEX=75%) da_ddt LAS= 75.43% MLAS= 65.41% BLEX= 66.04% (OK: Result F1 scores rounded to 5% are LAS=75% MLAS=65% BLEX=65%) nl_alpino LAS= 77.60% MLAS= 61.55% BLEX= 64.76% (OK: Result F1 scores rounded to 5% are LAS=80% MLAS=60% BLEX=65%) nl_lassysmall LAS= 74.56% MLAS= 61.85% BLEX= 63.14% (OK: Result F1 scores rounded to 5% are LAS=75% MLAS=60% BLEX=65%) en_ewt LAS= 77.56% MLAS= 68.70% BLEX= 71.02% (OK: Result F1 scores rounded to 5% are LAS=80% MLAS=70% BLEX=70%) en_gum LAS= 74.20% MLAS= 62.66% BLEX= 62.14% (OK: Result F1 scores rounded to 5% are LAS=75% MLAS=65% BLEX=60%) en_lines LAS= 73.10% MLAS= 64.03% BLEX= 65.42% (OK: Result F1 scores rounded to 5% are LAS=75% MLAS=65% BLEX=65%) en_pud LAS= 79.56% MLAS= 67.59% BLEX= 71.14% (OK: Result F1 scores rounded to 5% are LAS=80% MLAS=70% BLEX=70%) et_edt LAS= 75.02% MLAS= 67.12% BLEX= 63.85% (OK: Result F1 scores rounded to 5% are LAS=75% MLAS=65% BLEX=65%) fo_oft LAS= 25.19% MLAS= 0.36% BLEX= 5.56% (OK: Result F1 scores rounded to 5% are LAS=25% MLAS=0% BLEX=5%) fi_ftb LAS= 75.64% MLAS= 65.22% BLEX= 61.76% (OK: Result F1 scores rounded to 5% are LAS=75% MLAS=65% BLEX=60%) fi_pud LAS= 80.15% MLAS= 73.16% BLEX= 65.46% (OK: Result F1 scores rounded to 5% are LAS=80% MLAS=75% BLEX=65%) fi_tdt LAS= 76.45% MLAS= 68.58% BLEX= 62.19% (OK: Result F1 scores rounded to 5% are LAS=75% MLAS=70% BLEX=60%) fr_gsd LAS= 81.05% MLAS= 72.16% BLEX= 74.22% (OK: Result F1 scores rounded to 5% are LAS=80% MLAS=70% BLEX=75%) fr_sequoia LAS= 81.12% MLAS= 71.34% BLEX= 74.41% (OK: Result F1 scores rounded to 5% are LAS=80% MLAS=70% BLEX=75%) fr_spoken LAS= 65.56% MLAS= 53.46% BLEX= 54.67% (OK: Result F1 scores rounded to 5% are LAS=65% MLAS=55% BLEX=55%) gl_ctg LAS= 76.10% MLAS= 62.11% BLEX= 65.29% (OK: Result F1 scores rounded to 5% are LAS=75% MLAS=60% BLEX=65%) gl_treegal LAS= 66.16% MLAS= 49.13% BLEX= 51.60% (OK: Result F1 scores rounded to 5% are LAS=65% MLAS=50% BLEX=50%) de_gsd LAS= 70.85% MLAS= 34.09% BLEX= 60.56% (OK: Result F1 scores rounded to 5% are LAS=70% MLAS=35% BLEX=60%) got_proiel LAS= 62.16% MLAS= 48.57% BLEX= 55.02% (OK: Result F1 scores rounded to 5% are LAS=60% MLAS=50% BLEX=55%) el_gdt LAS= 82.11% MLAS= 65.33% BLEX= 68.67% (OK: Result F1 scores rounded to 5% are LAS=80% MLAS=65% BLEX=70%) he_htb LAS= 57.86% MLAS= 44.09% BLEX= 46.51% (OK: Result F1 scores rounded to 5% are LAS=60% MLAS=45% BLEX=45%) hi_hdtb LAS= 87.15% MLAS= 69.09% BLEX= 79.93% (OK: Result F1 scores rounded to 5% are LAS=85% MLAS=70% BLEX=80%) hu_szeged LAS= 66.76% MLAS= 52.82% BLEX= 56.92% (OK: Result F1 scores rounded to 5% are LAS=65% MLAS=55% BLEX=55%) zh_gsd LAS= 57.91% MLAS= 48.49% BLEX= 52.92% (OK: Result F1 scores rounded to 5% are LAS=60% MLAS=50% BLEX=55%) id_gsd LAS= 74.37% MLAS= 63.42% BLEX= 62.50% (OK: Result F1 scores rounded to 5% are LAS=75% MLAS=65% BLEX=60%) ga_idt LAS= 62.93% MLAS= 37.66% BLEX= 42.06% (OK: Result F1 scores rounded to 5% are LAS=65% MLAS=40% BLEX=40%) it_isdt LAS= 86.26% MLAS= 77.06% BLEX= 77.12% (OK: Result F1 scores rounded to 5% are LAS=85% MLAS=75% BLEX=75%) it_postwita LAS= 66.81% MLAS= 53.64% BLEX= 53.99% (OK: Result F1 scores rounded to 5% are LAS=65% MLAS=55% BLEX=55%) ja_gsd LAS= 72.32% MLAS= 58.35% BLEX= 60.17% (OK: Result F1 scores rounded to 5% are LAS=70% MLAS=60% BLEX=60%) ja_modern LAS= 22.71% MLAS= 8.10% BLEX= 9.49% (OK: Result F1 scores rounded to 5% are LAS=25% MLAS=10% BLEX=10%) kk_ktb LAS= 24.21% MLAS= 7.62% BLEX= 9.79% (OK: Result F1 scores rounded to 5% are LAS=25% MLAS=10% BLEX=10%) ko_gsd LAS= 61.40% MLAS= 54.10% BLEX= 50.50% (OK: Result F1 scores rounded to 5% are LAS=60% MLAS=55% BLEX=50%) ko_kaist LAS= 70.25% MLAS= 61.49% BLEX= 57.68% (OK: Result F1 scores rounded to 5% are LAS=70% MLAS=60% BLEX=60%) kmr_mg LAS= 23.92% MLAS= 5.47% BLEX= 11.86% (OK: Result F1 scores rounded to 5% are LAS=25% MLAS=5% BLEX=10%) la_ittb LAS= 75.95% MLAS= 66.08% BLEX= 71.87% (OK: Result F1 scores rounded to 5% are LAS=75% MLAS=65% BLEX=70%) la_perseus LAS= 47.61% MLAS= 30.16% BLEX= 32.19% (OK: Result F1 scores rounded to 5% are LAS=50% MLAS=30% BLEX=30%) la_proiel LAS= 59.66% MLAS= 47.05% BLEX= 53.65% (OK: Result F1 scores rounded to 5% are LAS=60% MLAS=45% BLEX=55%) lv_lvtb LAS= 69.43% MLAS= 54.96% BLEX= 58.25% (OK: Result F1 scores rounded to 5% are LAS=70% MLAS=55% BLEX=60%) pcm_nsc LAS= 12.18% MLAS= 4.60% BLEX= 10.87% (OK: Result F1 scores rounded to 5% are LAS=10% MLAS=5% BLEX=10%) sme_giella LAS= 56.98% MLAS= 46.05% BLEX= 42.35% (OK: Result F1 scores rounded to 5% are LAS=55% MLAS=45% BLEX=40%) no_bokmaal LAS= 83.47% MLAS= 74.65% BLEX= 76.32% (OK: Result F1 scores rounded to 5% are LAS=85% MLAS=75% BLEX=75%) no_nynorsk LAS= 82.13% MLAS= 72.40% BLEX= 74.22% (OK: Result F1 scores rounded to 5% are LAS=80% MLAS=70% BLEX=75%) no_nynorsklia LAS= 48.95% MLAS= 37.60% BLEX= 40.69% (OK: Result F1 scores rounded to 5% are LAS=50% MLAS=40% BLEX=40%) fro_srcmf LAS= 79.27% MLAS= 70.70% BLEX= 74.45% (OK: Result F1 scores rounded to 5% are LAS=80% MLAS=70% BLEX=75%) cu_proiel LAS= 65.46% MLAS= 53.96% BLEX= 58.39% (OK: Result F1 scores rounded to 5% are LAS=65% MLAS=55% BLEX=60%) fa_seraji LAS= 79.10% MLAS= 72.20% BLEX= 69.43% (OK: Result F1 scores rounded to 5% are LAS=80% MLAS=70% BLEX=70%) pl_lfg LAS= 87.53% MLAS= 74.54% BLEX= 78.58% (OK: Result F1 scores rounded to 5% are LAS=90% MLAS=75% BLEX=80%) pl_sz LAS= 81.90% MLAS= 63.84% BLEX= 71.98% (OK: Result F1 scores rounded to 5% are LAS=80% MLAS=65% BLEX=70%) pt_bosque LAS= 82.07% MLAS= 67.40% BLEX= 72.04% (OK: Result F1 scores rounded to 5% are LAS=80% MLAS=65% BLEX=70%) ro_rrt LAS= 80.27% MLAS= 71.48% BLEX= 71.87% (OK: Result F1 scores rounded to 5% are LAS=80% MLAS=70% BLEX=70%) ru_syntagrus LAS= 84.59% MLAS= 76.87% BLEX= 78.01% (OK: Result F1 scores rounded to 5% are LAS=85% MLAS=75% BLEX=80%) ru_taiga LAS= 55.51% MLAS= 36.79% BLEX= 39.79% (OK: Result F1 scores rounded to 5% are LAS=55% MLAS=35% BLEX=40%) sr_set LAS= 82.07% MLAS= 70.04% BLEX= 74.12% (OK: Result F1 scores rounded to 5% are LAS=80% MLAS=70% BLEX=75%) sk_snk LAS= 75.41% MLAS= 54.38% BLEX= 60.35% (OK: Result F1 scores rounded to 5% are LAS=75% MLAS=55% BLEX=60%) sl_ssj LAS= 77.33% MLAS= 63.47% BLEX= 68.93% (OK: Result F1 scores rounded to 5% are LAS=75% MLAS=65% BLEX=70%) sl_sst LAS= 46.95% MLAS= 34.19% BLEX= 38.73% (OK: Result F1 scores rounded to 5% are LAS=45% MLAS=35% BLEX=40%) es_ancora LAS= 84.43% MLAS= 76.01% BLEX= 76.43% (OK: Result F1 scores rounded to 5% are LAS=85% MLAS=75% BLEX=75%) sv_lines LAS= 74.06% MLAS= 58.62% BLEX= 66.39% (OK: Result F1 scores rounded to 5% are LAS=75% MLAS=60% BLEX=65%) sv_pud LAS= 70.63% MLAS= 43.38% BLEX= 54.47% (OK: Result F1 scores rounded to 5% are LAS=70% MLAS=45% BLEX=55%) sv_talbanken LAS= 77.91% MLAS= 69.22% BLEX= 70.01% (OK: Result F1 scores rounded to 5% are LAS=80% MLAS=70% BLEX=70%) th_pud LAS= 0.70% MLAS= 0.03% BLEX= 0.42% (OK: Result F1 scores rounded to 5% are LAS=0% MLAS=0% BLEX=0%) tr_imst LAS= 54.04% MLAS= 44.50% BLEX= 45.91% (OK: Result F1 scores rounded to 5% are LAS=55% MLAS=45% BLEX=45%) uk_iu LAS= 74.91% MLAS= 56.78% BLEX= 63.72% (OK: Result F1 scores rounded to 5% are LAS=75% MLAS=55% BLEX=65%) hsb_ufal LAS= 23.64% MLAS= 3.55% BLEX= 11.72% (OK: Result F1 scores rounded to 5% are LAS=25% MLAS=5% BLEX=10%) ur_udtb LAS= 77.29% MLAS= 50.31% BLEX= 63.74% (OK: Result F1 scores rounded to 5% are LAS=75% MLAS=50% BLEX=65%) ug_udt LAS= 56.26% MLAS= 36.82% BLEX= 43.53% (OK: Result F1 scores rounded to 5% are LAS=55% MLAS=35% BLEX=45%) vi_vtb LAS= 39.63% MLAS= 33.49% BLEX= 35.72% (OK: Result F1 scores rounded to 5% are LAS=40% MLAS=35% BLEX=35%)