UD Japanese PUD
Language: Japanese (code: ja
)
Family: Japanese
This treebank has been part of Universal Dependencies since the UD v2.1 release.
The following people have contributed to making this treebank part of UD: Mai Omura, Yusuke Miyao, Hiroshi Kanayama, Hiroshi Matsuda, Aya Wakasa, Kayo Yamashita, Masayuki Asahara, Takaaki Tanaka, Yugo Murawaki, Yuji Matsumoto, Kaoru Ito, Taishi Chika, Shinsuke Mori, Sumire Uematsu, Hans Uszkoreit, Vivien Macketanz, Aljoscha Burchardt, Kim Harris, Katrin Marheinecke, Slav Petrov, Tolga Kayadelen, Mohammed Attia, Ali Elkahky, Zhuoran Yu, Emily Pitler, Saran Lertpradit, Atsuko Shimada, Anna Trukhina, Martin Popel, Daniel Zeman.
Repository: UD_Japanese-PUD
Search this treebank on-line: PML-TQ
Download all treebanks: UD 2.14
License: CC BY-SA 3.0
Genre: news, wiki
Questions, comments? General annotation questions (either Japanese-specific or cross-linguistic) can be raised in the main UD issue tracker. You can report bugs in this treebank in the treebank-specific issue tracker on Github. If you want to collaborate, please contact [masayu-a (æt) ninjal • ac • jp]. Development of the treebank happens directly in the UD repository, so you may submit bug fixes as pull requests against the dev branch.
Annotation | Source |
---|---|
Lemmas | annotated manually in non-UD style, automatically converted to UD |
UPOS | annotated manually in non-UD style, automatically converted to UD |
XPOS | annotated manually |
Features | not available |
Relations | annotated manually in non-UD style, automatically converted to UD |
Description
This is a part of the Parallel Universal Dependencies (PUD) treebanks created for the CoNLL 2017 shared task on Multilingual Parsing from Raw Text to Universal Dependencies.
There are 1000 sentences in each language, always in the same order. (The sentence alignment is 1-1 but occasionally a sentence-level segment actually consists of two real sentences.) The sentences are taken from the news domain (sentence id starts in ‘n’) and from Wikipedia (sentence id starts with ‘w’). There are usually only a few sentences from each document, selected randomly, not necessarily adjacent. The digits on the second and third position in the sentence ids encode the original language of the sentence. The first 750 sentences are originally English (01). The remaining 250 sentences are originally German (02), French (03), Italian (04) or Spanish (05) and they were translated to other languages via English. Translation into German, French, Italian, Spanish, Arabic, Hindi, Chinese, Indonesian, Japanese, Korean, Portuguese, Russian, Thai and Turkish has been provided by DFKI and performed (except for German) by professional translators. Then the data has been annotated morphologically and syntactically by Google according to Google universal annotation guidelines; finally, it has been converted by members of the UD community to UD v2 guidelines.
Additional languages have been provided (both translation and native UD v2 annotation) by other teams: Czech by Charles University, Finnish by University of Turku and Swedish by Uppsala University.
The entire treebank is labeled as test set (and was used for testing in the shared task). If it is used for training in future research, the users should employ ten-fold cross-validation.
In May 2020, we introduced the same coversion method used in UD_Japanese GSD v2.6 for UD_Japanese PUD.
Acknowledgments
Statistics of UD Japanese PUD
POS Tags
ADJ – ADP – ADV – AUX – CCONJ – DET – INTJ – NOUN – NUM – PART – PRON – PROPN – PUNCT – SCONJ – SYM – VERB
Features
Relations
acl – advcl – advmod – amod – appos – aux – case – cc – ccomp – compound – cop – csubj – dep – det – discourse – fixed – mark – nmod – nsubj – nsubj:outer – nummod – obj – obl – punct – root
Tokenization and Word Segmentation
- This corpus contains 1000 sentences and 28788 tokens.
- This corpus contains 28353 tokens (98%) that are not followed by a space.
- This corpus does not contain words with spaces.
- This corpus contains 10 types of words that contain both letters and punctuation. Examples: B.C., G.D.P., HerFatherDidn’tLikeMeAnyway, Thought’YaKnew, Wi-Fi, Zettel’s, イェルバ・ブエナ, セスト・サン・ジョヴァンニ, プンタ・デル・エステ, プンタ・ラサ
Morphology
Tags
- This corpus uses 16 UPOS tags out of 17 possible: ADJ, ADP, ADV, AUX, CCONJ, DET, INTJ, NOUN, NUM, PART, PRON, PROPN, PUNCT, SCONJ, SYM, VERB
- This corpus does not use the following tags: X
- This corpus contains 14 word types tagged as particles (PART): い, か, がかっ, さ, たり, だけ, だらけ, っぽく, ね, ばかり, ほど, まで, 的, 風
- This corpus contains 18 lemmas tagged as pronouns (PRON): You, 何, 何れ, 何処, 何方, 何時, 其れ, 其処, 君, 彼, 彼女, 我々, 本の, 此れ, 此処, 私, 誰, 貴方
- This corpus contains 6 lemmas tagged as determiners (DET): あらゆる, 何の, 其の, 彼の, 或る, 此の
- This corpus contains 26 lemmas tagged as auxiliaries (AUX): がましい, させる, しめる, ず, せる, そう, た, たい, たり, だ, てる, です, ない, なり, べし, ます, らしい, られる, り, れる, 出来る, 易い, 様, 為る, 無い, 良い
- Out of the above, 2 lemmas occurred sometimes as AUX and sometimes as VERB: 出来る, 為る
- This corpus does not use the VerbForm feature.
Nominal Features
Degree and Polarity
- Neg
- AUX: ない, なかっ, ず, ん, なけれ, なく
- NOUN: 不, なし, 未, 非, 反, 無
- SCONJ: ず
Verbal Features
Pronouns, Determiners, Quantifiers
Other Features
Syntax
Auxiliary Verbs and Copula
- This corpus uses 2 lemmas as copulas (cop). Examples: だ, です.
- This corpus uses 23 lemmas as auxiliaries (aux). Examples: た, 為る, だ, れる, ます, ない, られる, 様, せる, ず, 出来る, たい, べし, です, させる, そう, たり, がましい, しめる, てる, なり, らしい, 易い.
Core Arguments, Oblique Arguments and Adjuncts
Here we consider only relations between verbs (parent) and nouns or pronouns (child).
- nsubj
- VERB--NOUN-ADP(が) (378)
- VERB--NOUN-ADP(は) (392)
- VERB--NOUN-ADP(も) (36)
- VERB--PRON-ADP(が) (27)
- VERB--PRON-ADP(は) (122)
- VERB--PRON-ADP(も) (4)
- obj
- VERB--NOUN-ADP(だけ)-ADP(を) (1)
- VERB--NOUN-ADP(など)-ADP(を) (2)
- VERB--NOUN-ADP(を) (739)
- VERB--NOUN-ADP(を)-ADP(も) (1)
- VERB--PRON-ADP(を) (23)
Relations Overview
- This corpus uses 1 relation subtypes: nsubj:outer
- The following 13 relation types are not used in this corpus at all: iobj, xcomp, vocative, expl, dislocated, clf, conj, flat, list, parataxis, orphan, goeswith, reparandum