IWPT 2021 Shared Task Results of nuig/15232114

Coarse F1 Scores

Each score pertains to the combined test set of the language, without distinguishing individual treebanks and the enhancement types they annotate. The last line shows the macro-average over languages.

LanguageTokensWordsSentencesUPOSXPOSUFeatsAllTagsLemmasUASLASCLASMLASBLEXEULASELAS
Arabic
Bulgarian99.9399.9397.4998.810.0035.970.0097.4093.3790.0386.511.6383.6080.2178.45
Czech
Dutch
English98.3698.9191.3495.170.0032.770.0095.7687.0784.4681.786.2178.3167.5865.40
Estonian99.6299.6287.4496.490.0135.040.0095.5585.4182.4681.0111.6777.0357.4054.03
Finnish
French
Italian
Latvian99.8299.8299.0796.120.0035.610.0095.4588.5185.1982.806.5978.8659.0056.67
Lithuanian99.8499.8488.1193.400.0030.090.0092.6678.2573.5271.243.6966.0661.5459.13
Polish
Russian99.5999.5999.0398.2599.5936.3236.0597.4992.6791.0189.464.6387.1272.6666.33
Slovak99.9699.9586.0096.620.0022.680.0094.6190.0987.4985.444.9380.1373.4767.45
Swedish99.4399.4393.3696.050.0033.560.0092.4685.6482.1879.816.3973.0565.8263.12
Tamil
Ukrainian
Average46.8646.8943.6445.355.8615.412.1244.7941.2439.7838.712.6936.7231.6330.03

Qualitative F1 Scores

The system output for each language was split into parts corresponding to individual source treebanks. The scores then ignore errors in enhancement types for which the treebank lacks gold-standard annotation. The column LAvgELAS shows the qualitative ELAS averaged over treebanks of the same language; the final average in bold is averaged over languages rather than treebanks. For comparison, the last column then shows the coarse ELAS F1 for the given language.

TreebankTokensWordsSentencesUPOSXPOSUFeatsAllTagsLemmasUASLASCLASMLASBLEXEULASELASEnhancementsLAvgELAScf.
Arabic-PADT(no xsubj)0
Bulgarian-BTB99.9399.9397.4998.810.0035.970.0097.4093.3790.0386.511.6383.6080.2178.45(no gapping)78.4578.45
Czech-FicTree0
Czech-CAC0.00
Czech-PDT0.00
Czech-PUD(no codepend)0.00
Dutch-Alpino0
Dutch-LassySmall0.00
English-EWT97.2598.6388.4995.020.0033.140.0096.7686.9484.4881.518.5779.7366.3864.5065.3365.40
English-GUM98.4498.4488.2994.730.0032.040.0094.4785.1582.2279.152.9574.4165.9063.7865.3365.40
English-PUD99.6099.6099.7095.670.0032.880.0095.5588.7086.1384.165.7879.5770.2967.7165.3365.40
Estonian-EDT99.9599.9592.9897.320.0034.410.0096.0487.3384.5783.0911.2279.0159.4255.87(no xsubj)51.7354.03
Estonian-EWT98.3798.3767.6193.480.0337.330.0093.7778.3274.7373.1013.3969.4650.3447.59(no codepend, no xsubj)51.7354.03
Finnish-TDT0
Finnish-PUD(no codepend, no xsubj)0.00
French-Sequoia(no gapping, relcl, case)0
French-FQB(no gapping, relcl, case)0.00
Italian-ISDT0
Latvian-LVTB99.8299.8299.0796.120.0035.610.0095.4588.5185.1982.806.5978.8659.0056.6756.6756.67
Lithuanian-ALKSNIS99.8499.8488.1193.400.0030.090.0092.6678.2573.5271.243.6966.0661.5459.1359.1359.13
Polish-LFG(no gapping)0
Polish-PDB0.00
Polish-PUD0.00
Russian-SynTagRus99.5999.5999.0398.2599.5936.3236.0597.4992.6791.0189.464.6387.1272.6666.33(no codepend)66.3366.33
Slovak-SNK99.9699.9586.0096.620.0022.680.0094.6190.0987.4985.444.9380.1373.4767.4567.4567.45
Swedish-Talbanken99.9799.9798.5297.720.0034.220.0097.6489.2186.2083.927.7481.3368.5165.8763.0263.12
Swedish-PUD98.8598.8587.3194.260.0032.850.0086.9181.8277.8775.424.9464.2262.9360.1763.0263.12
Tamil-TTB(no gapping, xsubj)0
Ukrainian-IU0
Average41.0941.1437.6839.703.4413.711.2439.2735.8734.6033.652.6231.8427.2625.9829.89