UD Japanese PUDLUW
Language: Japanese (code: ja)
Family: Japanese
This treebank has been part of Universal Dependencies since the UD v2.9 release.
The following people have contributed to making this treebank part of UD: Mai Omura, Yusuke Miyao, Hiroshi Kanayama, Hiroshi Matsuda, Aya Wakasa, Kayo Yamashita, Masayuki Asahara, Takaaki Tanaka, Yugo Murawaki, Yuji Matsumoto, Kaoru Ito, Taishi Chika, Shinsuke Mori, Sumire Uematsu, Hans Uszkoreit, Vivien Macketanz, Aljoscha Burchardt, Kim Harris, Katrin Marheinecke, Slav Petrov, Tolga Kayadelen, Mohammed Attia, Ali Elkahky, Zhuoran Yu, Emily Pitler, Saran Lertpradit, Atsuko Shimada, Anna Trukhina, Martin Popel, Daniel Zeman.
Repository: UD_Japanese-PUDLUW
Search this treebank on-line: PML-TQ
Download all treebanks: UD 2.15
License: CC BY-SA 3.0
Genre: news, wiki
Questions, comments? General annotation questions (either Japanese-specific or cross-linguistic) can be raised in the main UD issue tracker. You can report bugs in this treebank in the treebank-specific issue tracker on Github. If you want to collaborate, please contact [masayu-a (æt) ninjal • ac • jp]. Development of the treebank happens directly in the UD repository, so you may submit bug fixes as pull requests against the dev branch.
| Annotation | Source |
|---|---|
| Lemmas | annotated manually in non-UD style, automatically converted to UD |
| UPOS | annotated manually in non-UD style, automatically converted to UD |
| XPOS | annotated manually |
| Features | not available |
| Relations | annotated manually in non-UD style, automatically converted to UD |
Description
This is a part of the Parallel Universal Dependencies (PUD) treebanks created for the CoNLL 2017 shared task on Multilingual Parsing from Raw Text to Universal Dependencies.
There are 1000 sentences in each language, always in the same order. (The
sentence alignment is 1-1 but occasionally a sentence-level segment actually
consists of two real sentences.) The sentences are taken from the news domain
(sentence id starts in n') and from Wikipedia (sentence id starts with w’).
There are usually only a few sentences from each document, selected randomly,
not necessarily adjacent. The digits on the second and third position in the
sentence ids encode the original language of the sentence. The first 750
sentences are originally English (01). The remaining 250 sentences are
originally German (02), French (03), Italian (04) or Spanish (05) and they
were translated to other languages via English. Translation into German,
French, Italian, Spanish, Arabic, Hindi, Chinese, Indonesian, Japanese,
Korean, Portuguese, Russian, Thai and Turkish has been provided by DFKI and
performed (except for German) by professional translators. Then the data has
been annotated morphologically and syntactically by Google according to Google
universal annotation guidelines; finally, it has been converted by members of
the UD community to UD v2 guidelines.
Additional languages have been provided (both translation and native UD v2 annotation) by other teams: Czech by Charles University, Finnish by University of Turku and Swedish by Uppsala University.
The entire treebank is labeled as test set (and was used for testing in the shared task). If it is used for training in future research, the users should employ ten-fold cross-validation.
In May 2020, we introduced the same coversion method used in UD_Japanese GSD v2.6 for UD_Japanese PUD.
In May 2021, we introduce UD_Japanese-PUDLUW the other word segmentation (Long Unit Word; LUW) version of UD_Japanese-PUD.
Acknowledgments
Statistics of UD Japanese PUDLUW
POS Tags
ADJ – ADP – ADV – AUX – CCONJ – DET – INTJ – NOUN – NUM – PART – PRON – PROPN – PUNCT – SCONJ – SYM – VERB
Features
Relations
acl – advcl – advmod – amod – appos – aux – case – cc – ccomp – compound – cop – csubj – dep – det – discourse – fixed – mark – nmod – nsubj – nsubj:outer – nummod – obj – obl – punct – root
Tokenization and Word Segmentation
- This corpus contains 1000 sentences and 22910 tokens.
- This corpus contains 22508 tokens (98%) that are not followed by a space.
- This corpus contains 18 types of words with spaces. Examples: A New Earth in North America, Athina Film, Canadian Blood Services, E Micro, KnuckIf You Buck, On a Wing and a Prayer, Over My Head, Shaky Ground, Simple and Select関税, Snap chat, Team Telekom, The Power of Now, Under sea, Wild West, You Tube, mos maiorum, new comers, 作者Günter de Bruyn
- This corpus contains 197 types of words that contain both letters and punctuation. Examples: ドナルド・トランプ, クリストファー・コロンブス, ボグド・ハーン, ジョン・バーゴイン将軍, ティナ・アンセルミ, マルコ・アントニオ, 1％以上, 3％レート, 66％以上, 70％以上, 80%分, 90％分, B.C., B.C.警察, G.D.P.成長, HerFatherDidn’tLikeMeAnyway, Thought’YaKnew, Wi-Fi, Z.Amin, Zettel’sTraum, アビバ・インベスターズ, アムネスティ・インターナショナル, アレハンドラ・ゴンザレス・アナヤ, アンデス・デ・ティポ・アンドイノ, アントニ・ガウディ, アンドリュー・マー, アンドレ・プライス, アンヘル・ピンタド, アーネスト・シャックルトン卿, イェルバ・ブエナ・ガーデンズ, イヤー・オブ・ザ・リバー, イレーヌ・ジョリオ＝キュリー, インターネット・ディスカッション, インド・オーストラリア断層線, イヴ・サンローラン, ウェディング・ベル, ウォルト・ディズニー, エディンバラ・イーブニング・クーラント, エリック・ソルヘイム氏, エル・グレコ, エル・ムンド氏, エンニオ・モリコーネ, オバマ大統領特別補佐コリ・シューマン, オート・ガロンヌ地域, オードリー・ヘプバーン主演, カサ・サンタ・マルタ, カステルフランコ・ヴェネト, カンバーランド・ストラサーン公, ガース・ジェニングス, キキ・ディー
Morphology
Tags
- This corpus uses 16 UPOS tags out of 17 possible: ADJ, ADP, ADV, AUX, CCONJ, DET, INTJ, NOUN, NUM, PART, PRON, PROPN, PUNCT, SCONJ, SYM, VERB
- This corpus does not use the following tags: X
- This corpus contains 8 word types tagged as particles (PART): い, か, たり, だけ, ね, ばかり, ほど, まで
- This corpus contains 23 lemmas tagged as pronouns (PRON): You, 何, 何れ, 何処, 何方, 何時, 其れ, 其れ等, 其処, 君達, 彼, 彼女, 彼等, 我々, 本の, 此れ, 此れ等, 此処, 私, 私達, 誰, 貴方, 貴方方
- This corpus contains 6 lemmas tagged as determiners (DET): あらゆる, 何の, 其の, 彼の, 或る, 此の
- This corpus contains 48 lemmas tagged as auxiliaries (AUX): かもしれない, ことがある, ことができる, こととなる, ことにする, ことになる, ことはない, こともある, こともない, させる, しめる, ず, せる, そう, た, たい, たらいい, だ, つつある, ていく, ている, ておる, てく, てくる, てくれる, てしまう, てはならない, てもいい, てる, である, です, でない, ではありません, ではない, でもある, ない, なければならない, なり, に過ぎない, のだ, のである, のです, のではない, べし, ます, られる, れる, 様
- This corpus does not use the VerbForm feature.
Nominal Features
Degree and Polarity
- Neg
- AUX: ない, なかっ, ず, ん, なければならない, なく, なければならなかっ, なければならなく
Verbal Features
Pronouns, Determiners, Quantifiers
Other Features
Syntax
Auxiliary Verbs and Copula
- This corpus uses 2 lemmas as copulas (cop). Examples: だ, です.
- This corpus uses 48 lemmas as auxiliaries (aux). Examples: た, だ, ている, れる, である, ます, ない, られる, 様, せる, ことができる, のだ, ず, ておる, ことになる, ではない, たい, べし, てくる, かもしれない, てしまう, なければならない, こととなる, ことはない, のです, でない, ことがある, です, てはならない, ではありません, のである, こともない, させる, そう, ていく, てくれる, に過ぎない, ことにする, こともある, しめる, たらいい, つつある, てく, てもいい, てる, でもある, なり, のではない.
Core Arguments, Oblique Arguments and Adjuncts
Here we consider only relations between verbs (parent) and nouns or pronouns (child).
- nsubj
- VERB--NOUN-ADP(が) (356)
- VERB--NOUN-ADP(は) (352)
- VERB--NOUN-ADP(も) (28)
- VERB--PRON-ADP(が) (31)
- VERB--PRON-ADP(は) (139)
- VERB--PRON-ADP(も) (4)
- obj
- VERB--NOUN-ADP(だけ)-ADP(を) (1)
- VERB--NOUN-ADP(など)-ADP(を) (1)
- VERB--NOUN-ADP(など)-ADP(を通じて) (1)
- VERB--NOUN-ADP(を) (725)
- VERB--NOUN-ADP(を)-ADP(も) (1)
- VERB--NOUN-ADP(をもって) (1)
- VERB--NOUN-ADP(を通じて) (1)
- VERB--PRON-ADP(を) (23)
Relations Overview
- This corpus uses 1 relation subtypes: nsubj:outer
- The following 13 relation types are not used in this corpus at all: iobj, xcomp, vocative, expl, dislocated, clf, conj, flat, list, parataxis, orphan, goeswith, reparandum