IWPT 2021 Shared Task Results of combo/primary

Coarse F1 Scores

Each score pertains to the combined test set of the language, without distinguishing individual treebanks and the enhancement types they annotate. The last line shows the macro-average over languages.

LanguageTokensWordsSentencesUPOSXPOSUFeatsAllTagsLemmasUASLASCLASMLASBLEXEULASELAS
Arabic99.9797.9279.9295.0892.8692.9792.5292.3585.3881.0478.5272.8573.3278.3576.39
Bulgarian99.9399.9397.0098.7295.5597.2395.0297.2592.9889.5285.8881.9482.4787.4186.67
Czech99.9199.9194.6699.0496.6797.2095.8698.5694.8893.3092.4388.4390.9290.5789.08
Dutch99.8999.8983.8297.1695.8196.8895.1597.1493.2490.9387.1682.4383.8288.9087.07
English98.8798.9091.0695.7495.3393.5491.0395.2689.6187.2285.4276.2180.9485.2784.09
Estonian99.6799.6790.3597.4298.0696.5795.3086.0990.0087.5386.4281.9569.4285.5684.02
Finnish99.5999.5791.4598.1556.4497.3455.2789.1793.7692.2891.3388.0178.1888.7987.28
French99.6099.3995.6197.6799.3993.4792.5397.4192.2189.2985.3775.7782.6288.1087.32
Italian99.9599.5998.7698.3998.2397.9997.4398.3294.6993.2790.0687.4488.1891.1690.40
Latvian99.8199.8198.7497.3590.2494.9789.5496.5392.9190.2588.6381.0684.7886.2284.57
Lithuanian99.8199.8187.7497.2693.9095.0593.1193.7688.0384.7583.2577.2276.5081.2879.75
Polish99.4999.6598.5198.5294.5893.2190.6697.8194.7992.7591.5082.5889.1890.2287.65
Russian99.6799.6799.1598.9499.6798.0497.8298.1695.3794.2993.3491.0291.3991.7690.73
Slovak99.9799.9784.6197.8891.5895.0390.9095.6193.1991.7290.6584.3085.7388.5387.04
Swedish99.4499.4493.6497.6796.0389.1988.0392.4590.3187.8286.7772.0979.3485.2683.20
Tamil99.9291.2898.7681.0680.5183.3975.5086.4263.6256.2854.1747.6750.6253.4952.27
Ukrainian99.7899.7696.3698.2195.7695.9194.7997.8192.8590.9688.9583.5286.6887.6086.92
Average99.7299.0792.9596.7292.3994.5990.0394.7190.4687.8485.8779.6880.8385.2083.79

Qualitative F1 Scores

The system output for each language was split into parts corresponding to individual source treebanks. The scores then ignore errors in enhancement types for which the treebank lacks gold-standard annotation. The column LAvgELAS shows the qualitative ELAS averaged over treebanks of the same language; the final average in bold is averaged over languages rather than treebanks. For comparison, the last column then shows the coarse ELAS F1 for the given language.

TreebankTokensWordsSentencesUPOSXPOSUFeatsAllTagsLemmasUASLASCLASMLASBLEXEULASELASEnhancementsLAvgELAScf.
Arabic-PADT99.9797.9279.9295.0892.8692.9792.5292.3585.3881.0478.5272.8573.3278.3576.40(no xsubj)76.4076.39
Bulgarian-BTB99.9399.9397.0098.7295.5597.2395.0297.2592.9889.5285.8881.9482.4787.4186.67(no gapping)86.6786.67
Czech-FicTree99.9899.9797.7098.2791.9295.1590.5398.7996.3094.8793.7584.9591.9792.0190.7188.2789.08
Czech-CAC99.9999.9999.7699.5596.6996.9895.7998.2796.0294.6793.7289.3491.5290.5389.0788.2789.08
Czech-PDT99.9799.9793.7999.2897.4197.7696.7698.7695.0593.6392.9189.6791.5090.9189.4988.2789.08
Czech-PUD99.2099.2096.2997.2193.9993.9592.2296.6791.3487.9685.9579.0784.2186.0983.81(no codepend)88.2789.08
Dutch-Alpino99.8199.8190.5697.7496.2997.4395.7996.8694.6092.5489.4785.0985.5090.6789.0087.1187.07
Dutch-LassySmall99.9799.9778.9896.6095.3596.3594.5497.4091.9389.3885.0279.9782.2687.2185.2287.1187.07
English-EWT99.1398.7887.7095.6895.3895.4993.3896.4289.3687.1184.7979.3982.2285.2884.3483.9884.09
English-GUM98.7198.8389.7396.0495.9993.0491.2595.5789.1586.6284.1374.4979.8084.2582.8983.9884.09
English-PUD98.6999.0998.9595.5794.7791.6288.0993.6690.2787.8087.2273.5380.2186.0684.7083.9884.09
Estonian-EDT99.9499.9492.4397.9298.5096.9795.8985.6991.0588.7387.6483.4269.7087.4585.92(no xsubj)81.6284.02
Estonian-EWT98.6998.6983.0295.6096.4695.1193.1287.5486.1383.1281.7776.3368.3578.8677.32(no codepend, no xsubj)81.6284.02
Finnish-TDT99.7299.6892.2798.2898.7697.3496.7189.5193.7392.3391.5188.0878.6588.8787.3587.3487.28
Finnish-PUD99.4299.4290.2297.990.0097.330.0088.7093.7992.2191.0987.9177.5688.8487.33(no codepend, no xsubj)87.3487.28
French-Sequoia99.8999.5891.4198.9999.5898.1597.8298.6994.7093.5291.6288.5590.3091.1391.00(no gapping, relcl, case)88.7087.32
French-FQB99.4899.3196.4497.1399.3191.5190.3396.8891.1787.5382.7970.5179.4686.7086.40(no gapping, relcl, case)88.7087.32
Italian-ISDT99.9599.5998.7698.3998.2397.9997.4398.3294.6993.2790.0687.4488.1891.1690.4090.4090.40
Latvian-LVTB99.8199.8198.7497.3590.2494.9789.5496.5392.9190.2588.6381.0684.7886.2284.5784.5784.57
Lithuanian-ALKSNIS99.8199.8187.7497.2693.9095.0593.1193.7688.0384.7583.2577.2276.5081.2879.7579.7579.75
Polish-LFG99.6899.7999.7498.8093.9190.5586.7198.3995.9494.0993.1280.3791.3392.4183.96(no gapping)87.0287.65
Polish-PDB99.3699.6397.9298.6695.0293.7291.4097.9294.4592.4991.3082.9289.0989.6388.5587.0287.65
Polish-PUD99.5899.5997.7198.0594.2694.1992.1397.2094.5892.2790.7583.5087.8789.8188.5687.0287.65
Russian-SynTagRus99.6799.6799.1598.9499.6798.0497.8298.1695.3794.2993.3491.0291.3991.7690.73(no codepend)90.7390.73
Slovak-SNK99.9799.9784.6197.8891.5895.0390.9095.6193.1991.7290.6584.3085.7388.5387.0487.0487.04
Swedish-Talbanken99.9599.9598.4898.8097.7197.7296.9197.7493.3791.3890.0885.9387.1788.6086.7183.0783.20
Swedish-PUD98.8898.8887.9296.4694.2380.0778.5286.7887.0484.0283.2257.3070.9781.6679.4383.0783.20
Tamil-TTB99.9291.2898.7681.0680.5183.3975.5086.4263.6256.2854.1747.6750.6253.4752.25(no gapping, xsubj)52.2552.27
Ukrainian-IU99.7899.7696.3698.2195.7695.9194.7997.8192.8590.9688.9583.5286.6887.6086.9286.9286.92
Average99.6299.2393.1797.0991.8694.5289.1294.9591.3488.9187.0880.2582.0586.3084.7183.64