Statistical syntax-based machine translation approach to diacritization problem - Hai, Nguyen Minh; Tuấn, Nguyễn Minh (Journal of Computer Science and Cybernetics, 2014)
<!–[if gte mso 9]> Normal 0 false false false EN-US X-NONE X-NONE <![endif]–><span style="font-size: 11.0pt; line-height: 115%; font-family: "Calibri","sans-serif"; mso-ascii-theme-font: minor-latin; mso-fareast-font-family: "Times New Roman"; mso-fareast-theme-font: minor-fareast; mso-hansi-theme-font: minor-latin; mso-bidi-font-family: "Times New Roman"; mso-bidi-theme-font: minor-bidi; mso-ansi-language: EN-GB; mso-fareast-language: EN-GB; mso-bidi-language: AR-SA;">In this paper, the automatic diacritization of a language is modeled as a statistical syntax-based machine translation problem with the source undiacritized text and the target diacritized text of the same language. The grammatical inference technique ABL proposed in [2] is extended for learning a probabilistic synchronous context-free grammar from training corpus containing plain diacritized sentences only. The diacritization is to parse input sentences by the probabilistic CKY parsing algorithm for received grammar. This method is applied to Vietnamese with high quality result. As language independent building way, it can be applied to the other languages.<!–[if gte mso 9]> <![endif]–><!–[if gte mso 10]> <mce:style><! /* Style Definitions */ table.MsoNormalTable mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-priority:99; mso-style-parent:""; mso-padding-alt:0cm 5.4pt 0cm 5.4pt; mso-para-margin-top:0cm; mso-para-margin-right:0cm; mso-para-margin-bottom:10.0pt; mso-para-margin-left:0cm; line-height:115%; mso-pagination:widow-orphan; font-size:11.0pt; font-family:"Calibri","sans-serif"; mso-ascii-font-family:Calibri; mso-ascii-theme-font:minor-latin; mso-hansi-font-family:Calibri; mso-hansi-theme-font:minor-latin; –> <!–[endif] –>