Unofficial results (sent 2017-05-20 17:24 CET)

Dear shared task participants,

the task website now also includes a page with “unofficial results,” i.e. scores of runs that were not the final runs of your primary systems.

http://universaldependencies.org/conll17/results-unofficial.html

Although these scores do not constitute the main system ranking, some of them are interesting to see and compare. There are also runs that were performed after the official results were announced and the test data unblinded. If the unblinded output of your official run reveals that your system did not yield what it was supposed to (e.g. it picked wrong models for some languages), and you want to find out what the best possible score is, you can re-run the fixed system on TIRA, evaluate it as usual and ask me to update the table. You can do this even if you failed to deploy your system during the official test phase and you are not in the official results (but you have a virtual machine in TIRA).

In order not to break reproducibility of the official results, if you now run a system that is not identical to the system that produced your official submission, please register it in TIRA as new software and do not remove / replace the primary system in your VM.

Best regards,

Dan Zeman on behalf of the costocom :) (connl shared task organizing committee) http://universaldependencies.org/conll17/