【计算语言学概论】8 机器翻译

MachinetranslationIntroductiontoComputationalLinguistics–25April2017IntroductionTranslationoffulltextsfromthesourcelanguagetothetargetlanguageComputerAidedTranslation(CAT)Hardtask:contentshouldbethesamebutthemorphologicalandsyntacticrulesofthetargetlanguageshouldbeobservedQualityNowMTqualityiswaybelowthatofhumantranslationBUT:forsomespecific(delimited)topicsitissufficientrecipesweatherforecasttechnicaldescriptionsThemostdesiredNLPapplicationthroughouttheworld„tohelpthosewithdisabilities”Thetaskoftranslation„Ijustlookupthewords,Itranslatethesentence,andit’sdone”Let1sentencecontain15wordsonaverageLet1wordhave3meaningsonaverage1sentencecanhave315differenttranslations*permutationsduetowordorderchanges*insertionordeletionofwordsTheVauqoispyramid InterlinguaSourcelanguagesemantics TargetlanguagesemanticsSourcelanguagesyntax TargetlanguagesyntaxSourcelanguagewords Targetlanguagewords ApproachestoMTDictionarybasedMTSyntactictransformationbasedMTSemantictransformationbasedMTDictionarybasedMTElectronicdictionaryorbilinguallistsofwordsWordsorphrasesinSLarematchedtowordsorphrasesintheTLVerybasicsolutionProblematicissuesarefrequentProblemsinword-to-wordtranslation:Thewordcannotbefoundinthedictionary(SLwordontheTLtext)Toomanysensesinthedictionary(themostfrequentone(?)isselected)Problemsinwordform-to-wordformtranslation:Thedictionarycontainsonlyonewordform(e.g.SgNomfornouns)Inagglutinativelanguages,wordshavemanyforms(HUN)–theycannotbelistedMorphologicalanalysiswouldbenecessaryfortranslationProblemsinphrase-to-phrasetranslation:SyntacticparsingisneededforbothTLandSLResolutionofsyntacticambiguitiesSyntacticparsersshouldbeintegratedSyntactictransformationbasedMTSyntactictreeistransformedintoasyntactictreeNotonlytheleaves(words)aretranslatedThewholetreeistransformedTransformationofatreeOnenodeandonechildisselectedfromthetreewithsomeprobabilityThesenodesareordered(changetheorder,insertionordeletionofothernodes)3.LexicalunitsontheleavesaretranslatedExampleMarydidnotslapthegreenwitchMarianodabaunabofetadaalabrujaverdeCharacteristicsWordorderproblemsmightbesolvedAppropriatesolutionsforrelatedlanguagesComplextransformationrulesIncorrecttranslationsforstructuresthataredifferentinSlandTL(movement+direction):Labotellaentroalacuervaflotando.Thebottlefloatedintothecave.*Thebottleenteredthecavefloating.SemantictransformationSemanticinformationisusedduringMTLinguisticdifferencesamonglanguagescanbeneutralizedSimpleform:applyingsemanticfeatures(human,animate,male/female…)SLinformationistranslatedintoanintermediatelanguage(Interlingua)FrominterlinguaitistranslatedtoTLInformationistranslatednotonlysyntaxNowadays:English~interlinguaExampleJohngaveMaryabook.Give(john;book;mary)T<nowJeanadonnéunlivreàMarie.JánosadottMarinakegykönyvet.SemanticsbasedtranslationSimplerrulesMoreprecisetranslationBasedontheoreticalsemanticsLessexperienceDifficulttoconstructforsomelanguagepairsExamplebasedMT(EBMT)„whenhumanstranslate,theydonotusetransformationalrules,instead,theyfollowpreviouslyseenpatterns”Translationdatabase(parallelcorpus)TranslationunitsareidentifiedThemostsimilarexampleisselectedfromthedatabaseMethodsExamplebased:whichword/phraseismostsimilartotheoneinthedatabaseDictionarybased:weak,maybegoodforrelatedlanguages(syntactic)transferbased:mostmethodsarebasedonthis,acceptablesolutionsInterlingua:translationtointermediatelanguage–„futurework”StatisticalMTEachwordhasseveralpossibletranslationsSelectthemostprobablesequenceofwordsTranslationmodel:bag-of-wordstranslationLanguagemodel:meaningfulsentencesargmaxP(h|e)=argmaxP(h)*P(e|h)BLEU-scoreMeasuringthequalityoftranslationHumanevaluationisverycostlyForthesentencesofthetestset,therearesomehumantranslationsavailableThecoverageofword1,2,3,4gramsiscomparedtothehumantranslationsExampleAFöldnemkör,hanemellipszisalakúpályánkeringaNapkörül.ACsEegységetkorábbanazellipszisfélnagytengelyénekhosszakéntdefiniálták.TheEarthorbitsaroundtheSunonanelliptic,notcircularpath.TheAUwasdefinedashalfthelengthofthemajoraxisoftheellipse.TheEarthrevolvesaroundtheSunnotonacircularbutonanellipticorbit.TheunitAUwasearlierdefinedasthelengthofhalfofthemajoraxisoftheellipse.TheEarth’sorbitaroundtheSunisnotcircular,butelliptical.PreviouslytheAstronomicalUnit(AU)wasdefinedasthelengthofthesemi-majoraxisofthesaidellipse.TheEarthisnotcircular,butellipticalorbitaroundthesun.TheAUunitbeforethesemi-majoraxisoftheellipsewasdefinedasthelength.MTsystemsGoogleTranslate(http://translate.google.com/)Bing(http://www.bing.com/translator)HU-En:MetaMorpho(www.webforditas.hu)computer-aidedtranslation(CAT)tools:IntelligentdictionariesLexicaldatabasesTranslationmemoriesParalellcorporaFunnyexamplesbányászszív [minersucks]bulvárszíndarab [boulevardcolour][piece]gyertyamártás candlesaucehabképző [foam][derivationalsuffix]hajsütés hairbakinghalálnem deathgenderhalmajonéz [dyingmayonnaise]hóhányás [snow][vomit]hóhullás [snow][corpse]hőképzés [heat][training]hőkiütés [heat][knockout]időjóslat [time][prophecy]+lightverbconstruction fényigeszerkezetFunnyexamplesHerchildrengotothesameschoolasmine.Agyerekeibányakéntjárnakugyanabbaaziskolába.(MetaMorpho)Gyermekeimegyugyanaziskolamintbánya.(InterTran)Agyerekeiugyanabbaaziskolábajárnak,mintazenyémek.(reference)FunnyexamplesTherearenobiscuitsleft!Kekszeknincsenekbalra!(MetaMorpho)Nincskétszersültekbal!(InterTran)Nincstöbbkeksz!(reference)