Baseline models

There are non-empty training data for 73 treebanks. Most of the treebanks also have development data but some do not. If there are no development data, we cut a small portion of the training data and call it dev data for the purpose of tuning hyperparameters of the baseline models. We only use the remainder of the original training data for the actual training.

In addition to the treebank-specific models, there is also a “mixed model”, trained on samples from all treebanks. The first 200 training sentences of each treebank are included in the mixed training data. If there are fewer than 200 sentences (after a portion has been reserved to be used as dev data), all training sentences are included. Hyperparameters are tuned on mixed development data, consisting of at most 20 sentences from each treebank’s dev data. The mixed model is used as a baseline solution for languages where no training data are available.

The baseline models were trained using UDPipe 1.2 and they cover all levels of processing: tokenization, word segmentation, morphology (UPOS tags, features, lemmas) and syntax.

During the shared task, the baseline models will be downloadable from here (1232 MB). After the shared task, they will be published in Lindat. Besides the actual models the package also contains information needed to replicate them: the hyperparameters, the modified train-dev split where applicable, and pre-computed word embeddings for the parser.

Preprocessed development and test data on TIRA

By default, the participating systems are expected to process untokenized text, i.e., they must take care of all levels of processing (tokenization, morphology, and syntax). To make the task easier for teams that only want to focus on syntax, we provide input files preprocessed by UDPipe with the baseline models. All test sets and all official dev sets (where they exist) will be available in TIRA as CoNLL-U files with predicted tokenization, word segmentation, sentence segmentation, lemmas, UPOS tags and features (but not syntactic relations).

For treebanks that have non-empty training data (including treebanks whose training set is very small), the preprocessing will be done using the baseline model trained on that treebank (see above). For the nine treebanks that do not contain training data, a substitute model will be used:

  • Breton KEB ← mixed model
  • Czech PUD ← Czech PDT
  • English PUD ← English EWT
  • Faroese OFT ← mixed model
  • Finnish PUD ← Finnish TDT
  • Japanese Modern ← Japanese GSD
  • Naija NSC ← mixed model
  • Swedish PUD ← Swedish Talbanken
  • Thai PUD ← mixed model

Preprocessed morphology in training and development data

Some participants may want to train their parser on data with predicted morphology rather than the gold standard. To facilitate that, we provide a version of the training and development data where morphology (lemmas, UPOS, and features) was predicted by UDPipe. Syntactic annotation is manual. Note that word and sentence segmentation must be manual too, because the syntactic relations depend on it.

Morphology in development data is predicted using the baseline models (see above). Morphology in training data is predicted via “jack-knifing” (split the training set into 10 parts, train a model on 9 parts, use it to predict morphology in the tenth part, repeat for all 10 target parts). Same hyperparameters are used as those used to train the baseline model on the entire training set.

The data with predicted morphology can be downloaded from here (95 MB).

Baseline system on TIRA

The baseline system will consist of UDPipe 1.2 and the baseline models (see above). For the nine treebanks that do not have their own training data, a substitution model will be used (the same substitution as for preprocessing the dev/test data, see the table above).

Baseline results

Development data

The official evaluation of the baseline system after it had been run on Tira with the development data as input:

af_afribooms  LAS= 80.19% MLAS= 65.98% BLEX= 70.40% (OK: Result F1 scores rounded to 5% are LAS=80% MLAS=65% BLEX=70%)
grc_perseus   LAS= 57.89% MLAS= 30.80% BLEX= 40.49% (OK: Result F1 scores rounded to 5% are LAS=60% MLAS=30% BLEX=40%)
grc_proiel    LAS= 69.13% MLAS= 52.42% BLEX= 57.92% (OK: Result F1 scores rounded to 5% are LAS=70% MLAS=50% BLEX=60%)
ar_padt       LAS= 66.81% MLAS= 55.67% BLEX= 57.90% (OK: Result F1 scores rounded to 5% are LAS=65% MLAS=55% BLEX=60%)
eu_bdt        LAS= 70.06% MLAS= 57.46% BLEX= 63.39% (OK: Result F1 scores rounded to 5% are LAS=70% MLAS=55% BLEX=65%)
bg_btb        LAS= 84.67% MLAS= 74.54% BLEX= 73.78% (OK: Result F1 scores rounded to 5% are LAS=85% MLAS=75% BLEX=75%)
ca_ancora     LAS= 85.63% MLAS= 77.04% BLEX= 77.56% (OK: Result F1 scores rounded to 5% are LAS=85% MLAS=75% BLEX=80%)
hr_set        LAS= 77.84% MLAS= 59.60% BLEX= 69.99% (OK: Result F1 scores rounded to 5% are LAS=80% MLAS=60% BLEX=70%)
cs_cac        LAS= 84.42% MLAS= 72.17% BLEX= 78.29% (OK: Result F1 scores rounded to 5% are LAS=85% MLAS=70% BLEX=80%)
cs_fictree    LAS= 83.16% MLAS= 70.72% BLEX= 75.80% (OK: Result F1 scores rounded to 5% are LAS=85% MLAS=70% BLEX=75%)
cs_pdt        LAS= 84.85% MLAS= 75.35% BLEX= 80.55% (OK: Result F1 scores rounded to 5% are LAS=85% MLAS=75% BLEX=80%)
da_ddt        LAS= 75.16% MLAS= 65.29% BLEX= 66.07% (OK: Result F1 scores rounded to 5% are LAS=75% MLAS=65% BLEX=65%)
nl_alpino     LAS= 80.21% MLAS= 67.14% BLEX= 69.77% (OK: Result F1 scores rounded to 5% are LAS=80% MLAS=65% BLEX=70%)
nl_lassysmall LAS= 73.61% MLAS= 59.99% BLEX= 61.71% (OK: Result F1 scores rounded to 5% are LAS=75% MLAS=60% BLEX=60%)
en_ewt        LAS= 77.62% MLAS= 68.58% BLEX= 70.98% (OK: Result F1 scores rounded to 5% are LAS=80% MLAS=70% BLEX=70%)
en_gum        LAS= 76.63% MLAS= 65.57% BLEX= 67.20% (OK: Result F1 scores rounded to 5% are LAS=75% MLAS=65% BLEX=65%)
en_lines      LAS= 75.78% MLAS= 66.29% BLEX= 68.57% (OK: Result F1 scores rounded to 5% are LAS=75% MLAS=65% BLEX=70%)
et_edt        LAS= 76.50% MLAS= 68.27% BLEX= 64.17% (OK: Result F1 scores rounded to 5% are LAS=75% MLAS=70% BLEX=65%)
fi_ftb        LAS= 75.76% MLAS= 65.72% BLEX= 62.68% (OK: Result F1 scores rounded to 5% are LAS=75% MLAS=65% BLEX=65%)
fi_tdt        LAS= 76.39% MLAS= 68.60% BLEX= 62.33% (OK: Result F1 scores rounded to 5% are LAS=75% MLAS=70% BLEX=60%)
fr_gsd        LAS= 85.81% MLAS= 77.80% BLEX= 79.16% (OK: Result F1 scores rounded to 5% are LAS=85% MLAS=80% BLEX=80%)
fr_sequoia    LAS= 82.72% MLAS= 74.13% BLEX= 76.34% (OK: Result F1 scores rounded to 5% are LAS=85% MLAS=75% BLEX=75%)
fr_spoken     LAS= 65.09% MLAS= 54.00% BLEX= 55.42% (OK: Result F1 scores rounded to 5% are LAS=65% MLAS=55% BLEX=55%)
gl_ctg        LAS= 76.32% MLAS= 62.58% BLEX= 65.57% (OK: Result F1 scores rounded to 5% are LAS=75% MLAS=65% BLEX=65%)
de_gsd        LAS= 75.55% MLAS= 38.52% BLEX= 65.39% (OK: Result F1 scores rounded to 5% are LAS=75% MLAS=40% BLEX=65%)
got_proiel    LAS= 62.03% MLAS= 48.16% BLEX= 54.39% (OK: Result F1 scores rounded to 5% are LAS=60% MLAS=50% BLEX=55%)
el_gdt        LAS= 81.37% MLAS= 63.92% BLEX= 65.21% (OK: Result F1 scores rounded to 5% are LAS=80% MLAS=65% BLEX=65%)
he_htb        LAS= 61.95% MLAS= 49.28% BLEX= 51.45% (OK: Result F1 scores rounded to 5% are LAS=60% MLAS=50% BLEX=50%)
hi_hdtb       LAS= 87.26% MLAS= 69.78% BLEX= 80.59% (OK: Result F1 scores rounded to 5% are LAS=85% MLAS=70% BLEX=80%)
hu_szeged     LAS= 68.41% MLAS= 56.47% BLEX= 60.17% (OK: Result F1 scores rounded to 5% are LAS=70% MLAS=55% BLEX=60%)
zh_gsd        LAS= 57.39% MLAS= 48.19% BLEX= 52.84% (OK: Result F1 scores rounded to 5% are LAS=55% MLAS=50% BLEX=55%)
id_gsd        LAS= 74.40% MLAS= 63.51% BLEX= 63.29% (OK: Result F1 scores rounded to 5% are LAS=75% MLAS=65% BLEX=65%)
it_isdt       LAS= 85.95% MLAS= 77.20% BLEX= 77.37% (OK: Result F1 scores rounded to 5% are LAS=85% MLAS=75% BLEX=75%)
it_postwita   LAS= 65.85% MLAS= 52.14% BLEX= 52.90% (OK: Result F1 scores rounded to 5% are LAS=65% MLAS=50% BLEX=55%)
ja_gsd        LAS= 75.48% MLAS= 62.39% BLEX= 64.58% (OK: Result F1 scores rounded to 5% are LAS=75% MLAS=60% BLEX=65%)
ko_gsd        LAS= 57.25% MLAS= 49.06% BLEX= 44.24% (OK: Result F1 scores rounded to 5% are LAS=55% MLAS=50% BLEX=45%)
ko_kaist      LAS= 71.00% MLAS= 63.32% BLEX= 59.19% (OK: Result F1 scores rounded to 5% are LAS=70% MLAS=65% BLEX=60%)
la_ittb       LAS= 73.23% MLAS= 59.94% BLEX= 67.43% (OK: Result F1 scores rounded to 5% are LAS=75% MLAS=60% BLEX=65%)
la_proiel     LAS= 61.33% MLAS= 48.40% BLEX= 55.10% (OK: Result F1 scores rounded to 5% are LAS=60% MLAS=50% BLEX=55%)
lv_lvtb       LAS= 70.67% MLAS= 57.79% BLEX= 60.96% (OK: Result F1 scores rounded to 5% are LAS=70% MLAS=60% BLEX=60%)
no_bokmaal    LAS= 84.56% MLAS= 75.95% BLEX= 78.04% (OK: Result F1 scores rounded to 5% are LAS=85% MLAS=75% BLEX=80%)
no_nynorsk    LAS= 82.75% MLAS= 73.88% BLEX= 75.76% (OK: Result F1 scores rounded to 5% are LAS=85% MLAS=75% BLEX=75%)
fro_srcmf     LAS= 79.15% MLAS= 70.43% BLEX= 74.27% (OK: Result F1 scores rounded to 5% are LAS=80% MLAS=70% BLEX=75%)
cu_proiel     LAS= 66.12% MLAS= 54.48% BLEX= 59.16% (OK: Result F1 scores rounded to 5% are LAS=65% MLAS=55% BLEX=60%)
fa_seraji     LAS= 79.78% MLAS= 73.03% BLEX= 73.35% (OK: Result F1 scores rounded to 5% are LAS=80% MLAS=75% BLEX=75%)
pl_lfg        LAS= 88.79% MLAS= 75.15% BLEX= 79.18% (OK: Result F1 scores rounded to 5% are LAS=90% MLAS=75% BLEX=80%)
pl_sz         LAS= 82.65% MLAS= 63.92% BLEX= 72.59% (OK: Result F1 scores rounded to 5% are LAS=85% MLAS=65% BLEX=75%)
pt_bosque     LAS= 84.93% MLAS= 73.22% BLEX= 76.02% (OK: Result F1 scores rounded to 5% are LAS=85% MLAS=75% BLEX=75%)
ro_rrt        LAS= 80.32% MLAS= 71.21% BLEX= 71.82% (OK: Result F1 scores rounded to 5% are LAS=80% MLAS=70% BLEX=70%)
ru_syntagrus  LAS= 83.87% MLAS= 75.78% BLEX= 77.27% (OK: Result F1 scores rounded to 5% are LAS=85% MLAS=75% BLEX=75%)
sr_set        LAS= 82.12% MLAS= 69.12% BLEX= 73.06% (OK: Result F1 scores rounded to 5% are LAS=80% MLAS=70% BLEX=75%)
sk_snk        LAS= 75.73% MLAS= 54.34% BLEX= 59.71% (OK: Result F1 scores rounded to 5% are LAS=75% MLAS=55% BLEX=60%)
sl_ssj        LAS= 77.72% MLAS= 63.96% BLEX= 68.97% (OK: Result F1 scores rounded to 5% are LAS=80% MLAS=65% BLEX=70%)
es_ancora     LAS= 85.08% MLAS= 76.81% BLEX= 77.48% (OK: Result F1 scores rounded to 5% are LAS=85% MLAS=75% BLEX=75%)
sv_lines      LAS= 76.23% MLAS= 62.16% BLEX= 67.63% (OK: Result F1 scores rounded to 5% are LAS=75% MLAS=60% BLEX=70%)
sv_talbanken  LAS= 75.39% MLAS= 66.87% BLEX= 68.25% (OK: Result F1 scores rounded to 5% are LAS=75% MLAS=65% BLEX=70%)
tr_imst       LAS= 54.83% MLAS= 44.25% BLEX= 45.81% (OK: Result F1 scores rounded to 5% are LAS=55% MLAS=45% BLEX=45%)
uk_iu         LAS= 77.94% MLAS= 59.66% BLEX= 68.07% (OK: Result F1 scores rounded to 5% are LAS=80% MLAS=60% BLEX=70%)
ur_udtb       LAS= 77.44% MLAS= 49.91% BLEX= 63.55% (OK: Result F1 scores rounded to 5% are LAS=75% MLAS=50% BLEX=65%)
ug_udt        LAS= 56.88% MLAS= 37.43% BLEX= 43.34% (OK: Result F1 scores rounded to 5% are LAS=55% MLAS=35% BLEX=45%)
vi_vtb        LAS= 43.65% MLAS= 37.39% BLEX= 39.18% (OK: Result F1 scores rounded to 5% are LAS=45% MLAS=35% BLEX=40%)

Test data

The official evaluation of the baseline system after it had been run on Tira with the test data as input:

af_afribooms  LAS= 77.88% MLAS= 64.48% BLEX= 66.60% (OK: Result F1 scores rounded to 5% are LAS=80% MLAS=65% BLEX=65%)
grc_perseus   LAS= 57.75% MLAS= 31.05% BLEX= 38.74% (OK: Result F1 scores rounded to 5% are LAS=60% MLAS=30% BLEX=40%)
grc_proiel    LAS= 67.57% MLAS= 49.51% BLEX= 55.85% (OK: Result F1 scores rounded to 5% are LAS=70% MLAS=50% BLEX=55%)
ar_padt       LAS= 66.41% MLAS= 55.01% BLEX= 57.60% (OK: Result F1 scores rounded to 5% are LAS=65% MLAS=55% BLEX=60%)
hy_armtdp     LAS= 21.79% MLAS=  6.84% BLEX= 11.94% (OK: Result F1 scores rounded to 5% are LAS=20% MLAS=5% BLEX=10%)
eu_bdt        LAS= 70.13% MLAS= 57.65% BLEX= 63.50% (OK: Result F1 scores rounded to 5% are LAS=70% MLAS=60% BLEX=65%)
br_keb        LAS= 10.25% MLAS=  0.37% BLEX=  2.10% (OK: Result F1 scores rounded to 5% are LAS=10% MLAS=0% BLEX=0%)
bg_btb        LAS= 84.91% MLAS= 75.30% BLEX= 73.78% (OK: Result F1 scores rounded to 5% are LAS=85% MLAS=75% BLEX=75%)
bxr_bdt       LAS= 12.61% MLAS=  2.09% BLEX=  4.41% (OK: Result F1 scores rounded to 5% are LAS=15% MLAS=0% BLEX=5%)
ca_ancora     LAS= 85.61% MLAS= 76.74% BLEX= 77.27% (OK: Result F1 scores rounded to 5% are LAS=85% MLAS=75% BLEX=75%)
hr_set        LAS= 78.61% MLAS= 58.72% BLEX= 70.26% (OK: Result F1 scores rounded to 5% are LAS=80% MLAS=60% BLEX=70%)
cs_cac        LAS= 83.72% MLAS= 70.89% BLEX= 77.65% (OK: Result F1 scores rounded to 5% are LAS=85% MLAS=70% BLEX=80%)
cs_fictree    LAS= 82.49% MLAS= 69.26% BLEX= 74.96% (OK: Result F1 scores rounded to 5% are LAS=80% MLAS=70% BLEX=75%)
cs_pdt        LAS= 83.94% MLAS= 74.32% BLEX= 79.39% (OK: Result F1 scores rounded to 5% are LAS=85% MLAS=75% BLEX=80%)
cs_pud        LAS= 80.08% MLAS= 66.53% BLEX= 73.79% (OK: Result F1 scores rounded to 5% are LAS=80% MLAS=65% BLEX=75%)
da_ddt        LAS= 75.43% MLAS= 65.41% BLEX= 66.04% (OK: Result F1 scores rounded to 5% are LAS=75% MLAS=65% BLEX=65%)
nl_alpino     LAS= 77.60% MLAS= 61.55% BLEX= 64.76% (OK: Result F1 scores rounded to 5% are LAS=80% MLAS=60% BLEX=65%)
nl_lassysmall LAS= 74.56% MLAS= 61.85% BLEX= 63.14% (OK: Result F1 scores rounded to 5% are LAS=75% MLAS=60% BLEX=65%)
en_ewt        LAS= 77.56% MLAS= 68.70% BLEX= 71.02% (OK: Result F1 scores rounded to 5% are LAS=80% MLAS=70% BLEX=70%)
en_gum        LAS= 74.20% MLAS= 62.66% BLEX= 62.14% (OK: Result F1 scores rounded to 5% are LAS=75% MLAS=65% BLEX=60%)
en_lines      LAS= 73.10% MLAS= 64.03% BLEX= 65.42% (OK: Result F1 scores rounded to 5% are LAS=75% MLAS=65% BLEX=65%)
en_pud        LAS= 79.56% MLAS= 67.59% BLEX= 71.14% (OK: Result F1 scores rounded to 5% are LAS=80% MLAS=70% BLEX=70%)
et_edt        LAS= 75.02% MLAS= 67.12% BLEX= 63.85% (OK: Result F1 scores rounded to 5% are LAS=75% MLAS=65% BLEX=65%)
fo_oft        LAS= 25.19% MLAS=  0.36% BLEX=  5.56% (OK: Result F1 scores rounded to 5% are LAS=25% MLAS=0% BLEX=5%)
fi_ftb        LAS= 75.64% MLAS= 65.22% BLEX= 61.76% (OK: Result F1 scores rounded to 5% are LAS=75% MLAS=65% BLEX=60%)
fi_pud        LAS= 80.15% MLAS= 73.16% BLEX= 65.46% (OK: Result F1 scores rounded to 5% are LAS=80% MLAS=75% BLEX=65%)
fi_tdt        LAS= 76.45% MLAS= 68.58% BLEX= 62.19% (OK: Result F1 scores rounded to 5% are LAS=75% MLAS=70% BLEX=60%)
fr_gsd        LAS= 81.05% MLAS= 72.16% BLEX= 74.22% (OK: Result F1 scores rounded to 5% are LAS=80% MLAS=70% BLEX=75%)
fr_sequoia    LAS= 81.12% MLAS= 71.34% BLEX= 74.41% (OK: Result F1 scores rounded to 5% are LAS=80% MLAS=70% BLEX=75%)
fr_spoken     LAS= 65.56% MLAS= 53.46% BLEX= 54.67% (OK: Result F1 scores rounded to 5% are LAS=65% MLAS=55% BLEX=55%)
gl_ctg        LAS= 76.10% MLAS= 62.11% BLEX= 65.29% (OK: Result F1 scores rounded to 5% are LAS=75% MLAS=60% BLEX=65%)
gl_treegal    LAS= 66.16% MLAS= 49.13% BLEX= 51.60% (OK: Result F1 scores rounded to 5% are LAS=65% MLAS=50% BLEX=50%)
de_gsd        LAS= 70.85% MLAS= 34.09% BLEX= 60.56% (OK: Result F1 scores rounded to 5% are LAS=70% MLAS=35% BLEX=60%)
got_proiel    LAS= 62.16% MLAS= 48.57% BLEX= 55.02% (OK: Result F1 scores rounded to 5% are LAS=60% MLAS=50% BLEX=55%)
el_gdt        LAS= 82.11% MLAS= 65.33% BLEX= 68.67% (OK: Result F1 scores rounded to 5% are LAS=80% MLAS=65% BLEX=70%)
he_htb        LAS= 57.86% MLAS= 44.09% BLEX= 46.51% (OK: Result F1 scores rounded to 5% are LAS=60% MLAS=45% BLEX=45%)
hi_hdtb       LAS= 87.15% MLAS= 69.09% BLEX= 79.93% (OK: Result F1 scores rounded to 5% are LAS=85% MLAS=70% BLEX=80%)
hu_szeged     LAS= 66.76% MLAS= 52.82% BLEX= 56.92% (OK: Result F1 scores rounded to 5% are LAS=65% MLAS=55% BLEX=55%)
zh_gsd        LAS= 57.91% MLAS= 48.49% BLEX= 52.92% (OK: Result F1 scores rounded to 5% are LAS=60% MLAS=50% BLEX=55%)
id_gsd        LAS= 74.37% MLAS= 63.42% BLEX= 62.50% (OK: Result F1 scores rounded to 5% are LAS=75% MLAS=65% BLEX=60%)
ga_idt        LAS= 62.93% MLAS= 37.66% BLEX= 42.06% (OK: Result F1 scores rounded to 5% are LAS=65% MLAS=40% BLEX=40%)
it_isdt       LAS= 86.26% MLAS= 77.06% BLEX= 77.12% (OK: Result F1 scores rounded to 5% are LAS=85% MLAS=75% BLEX=75%)
it_postwita   LAS= 66.81% MLAS= 53.64% BLEX= 53.99% (OK: Result F1 scores rounded to 5% are LAS=65% MLAS=55% BLEX=55%)
ja_gsd        LAS= 72.32% MLAS= 58.35% BLEX= 60.17% (OK: Result F1 scores rounded to 5% are LAS=70% MLAS=60% BLEX=60%)
ja_modern     LAS= 22.71% MLAS=  8.10% BLEX=  9.49% (OK: Result F1 scores rounded to 5% are LAS=25% MLAS=10% BLEX=10%)
kk_ktb        LAS= 24.21% MLAS=  7.62% BLEX=  9.79% (OK: Result F1 scores rounded to 5% are LAS=25% MLAS=10% BLEX=10%)
ko_gsd        LAS= 61.40% MLAS= 54.10% BLEX= 50.50% (OK: Result F1 scores rounded to 5% are LAS=60% MLAS=55% BLEX=50%)
ko_kaist      LAS= 70.25% MLAS= 61.49% BLEX= 57.68% (OK: Result F1 scores rounded to 5% are LAS=70% MLAS=60% BLEX=60%)
kmr_mg        LAS= 23.92% MLAS=  5.47% BLEX= 11.86% (OK: Result F1 scores rounded to 5% are LAS=25% MLAS=5% BLEX=10%)
la_ittb       LAS= 75.95% MLAS= 66.08% BLEX= 71.87% (OK: Result F1 scores rounded to 5% are LAS=75% MLAS=65% BLEX=70%)
la_perseus    LAS= 47.61% MLAS= 30.16% BLEX= 32.19% (OK: Result F1 scores rounded to 5% are LAS=50% MLAS=30% BLEX=30%)
la_proiel     LAS= 59.66% MLAS= 47.05% BLEX= 53.65% (OK: Result F1 scores rounded to 5% are LAS=60% MLAS=45% BLEX=55%)
lv_lvtb       LAS= 69.43% MLAS= 54.96% BLEX= 58.25% (OK: Result F1 scores rounded to 5% are LAS=70% MLAS=55% BLEX=60%)
pcm_nsc       LAS= 12.18% MLAS=  4.60% BLEX= 10.87% (OK: Result F1 scores rounded to 5% are LAS=10% MLAS=5% BLEX=10%)
sme_giella    LAS= 56.98% MLAS= 46.05% BLEX= 42.35% (OK: Result F1 scores rounded to 5% are LAS=55% MLAS=45% BLEX=40%)
no_bokmaal    LAS= 83.47% MLAS= 74.65% BLEX= 76.32% (OK: Result F1 scores rounded to 5% are LAS=85% MLAS=75% BLEX=75%)
no_nynorsk    LAS= 82.13% MLAS= 72.40% BLEX= 74.22% (OK: Result F1 scores rounded to 5% are LAS=80% MLAS=70% BLEX=75%)
no_nynorsklia LAS= 48.95% MLAS= 37.60% BLEX= 40.69% (OK: Result F1 scores rounded to 5% are LAS=50% MLAS=40% BLEX=40%)
fro_srcmf     LAS= 79.27% MLAS= 70.70% BLEX= 74.45% (OK: Result F1 scores rounded to 5% are LAS=80% MLAS=70% BLEX=75%)
cu_proiel     LAS= 65.46% MLAS= 53.96% BLEX= 58.39% (OK: Result F1 scores rounded to 5% are LAS=65% MLAS=55% BLEX=60%)
fa_seraji     LAS= 79.10% MLAS= 72.20% BLEX= 69.43% (OK: Result F1 scores rounded to 5% are LAS=80% MLAS=70% BLEX=70%)
pl_lfg        LAS= 87.53% MLAS= 74.54% BLEX= 78.58% (OK: Result F1 scores rounded to 5% are LAS=90% MLAS=75% BLEX=80%)
pl_sz         LAS= 81.90% MLAS= 63.84% BLEX= 71.98% (OK: Result F1 scores rounded to 5% are LAS=80% MLAS=65% BLEX=70%)
pt_bosque     LAS= 82.07% MLAS= 67.40% BLEX= 72.04% (OK: Result F1 scores rounded to 5% are LAS=80% MLAS=65% BLEX=70%)
ro_rrt        LAS= 80.27% MLAS= 71.48% BLEX= 71.87% (OK: Result F1 scores rounded to 5% are LAS=80% MLAS=70% BLEX=70%)
ru_syntagrus  LAS= 84.59% MLAS= 76.87% BLEX= 78.01% (OK: Result F1 scores rounded to 5% are LAS=85% MLAS=75% BLEX=80%)
ru_taiga      LAS= 55.51% MLAS= 36.79% BLEX= 39.79% (OK: Result F1 scores rounded to 5% are LAS=55% MLAS=35% BLEX=40%)
sr_set        LAS= 82.07% MLAS= 70.04% BLEX= 74.12% (OK: Result F1 scores rounded to 5% are LAS=80% MLAS=70% BLEX=75%)
sk_snk        LAS= 75.41% MLAS= 54.38% BLEX= 60.35% (OK: Result F1 scores rounded to 5% are LAS=75% MLAS=55% BLEX=60%)
sl_ssj        LAS= 77.33% MLAS= 63.47% BLEX= 68.93% (OK: Result F1 scores rounded to 5% are LAS=75% MLAS=65% BLEX=70%)
sl_sst        LAS= 46.95% MLAS= 34.19% BLEX= 38.73% (OK: Result F1 scores rounded to 5% are LAS=45% MLAS=35% BLEX=40%)
es_ancora     LAS= 84.43% MLAS= 76.01% BLEX= 76.43% (OK: Result F1 scores rounded to 5% are LAS=85% MLAS=75% BLEX=75%)
sv_lines      LAS= 74.06% MLAS= 58.62% BLEX= 66.39% (OK: Result F1 scores rounded to 5% are LAS=75% MLAS=60% BLEX=65%)
sv_pud        LAS= 70.63% MLAS= 43.38% BLEX= 54.47% (OK: Result F1 scores rounded to 5% are LAS=70% MLAS=45% BLEX=55%)
sv_talbanken  LAS= 77.91% MLAS= 69.22% BLEX= 70.01% (OK: Result F1 scores rounded to 5% are LAS=80% MLAS=70% BLEX=70%)
th_pud        LAS=  0.70% MLAS=  0.03% BLEX=  0.42% (OK: Result F1 scores rounded to 5% are LAS=0% MLAS=0% BLEX=0%)
tr_imst       LAS= 54.04% MLAS= 44.50% BLEX= 45.91% (OK: Result F1 scores rounded to 5% are LAS=55% MLAS=45% BLEX=45%)
uk_iu         LAS= 74.91% MLAS= 56.78% BLEX= 63.72% (OK: Result F1 scores rounded to 5% are LAS=75% MLAS=55% BLEX=65%)
hsb_ufal      LAS= 23.64% MLAS=  3.55% BLEX= 11.72% (OK: Result F1 scores rounded to 5% are LAS=25% MLAS=5% BLEX=10%)
ur_udtb       LAS= 77.29% MLAS= 50.31% BLEX= 63.74% (OK: Result F1 scores rounded to 5% are LAS=75% MLAS=50% BLEX=65%)
ug_udt        LAS= 56.26% MLAS= 36.82% BLEX= 43.53% (OK: Result F1 scores rounded to 5% are LAS=55% MLAS=35% BLEX=45%)
vi_vtb        LAS= 39.63% MLAS= 33.49% BLEX= 35.72% (OK: Result F1 scores rounded to 5% are LAS=40% MLAS=35% BLEX=35%)