Moses-support Digest, Vol 95, Issue 10

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: Moses-support Digest, Vol 95, Issue 8 (Yusup Ashrap)
2. Re: Incremental retraining (Hieu Hoang)


----------------------------------------------------------------------

Message: 1
Date: Sat, 6 Sep 2014 13:26:46 +0800
From: Yusup Ashrap <aphawk@gmail.com>
Subject: Re: [Moses-support] Moses-support Digest, Vol 95, Issue 8
To: moses-support@mit.edu
Message-ID:
<CADSMiU8ESejE+Z5DKAWCOkP-voiw+6F2Av2d3Pg=UV2dFgN4+w@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hi Tom ,
Thanks for the reply.

1.Yes I did run this with moses binary.Here is the command and result.



$ echo '? ? ? <xjunlphaha translation="?????">??</xjunlphaha>' |
/work/mosesdecoder/bin/moses -xml-input exclusive -f
/tmp/shared/moses-training/chuy_0826-03:04/model/moses.ini

I found out that chinese and arabic letters could not be shown properly ,
so I added my command to the last comment in the issue.
https://github.com/moses-smt/mosesdecoder/issues/73


\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\
FeatureFunction: UnknownWordPenalty0 start: 0 end: 0
line=WordPenalty
FeatureFunction: WordPenalty0 start: 1 end: 1
line=PhrasePenalty
FeatureFunction: PhrasePenalty0 start: 2 end: 2
line=PhraseDictionaryMemory name=TranslationModel0 table-limit=20
num-features=4
path=/tmp/shared/moses-training/chuy_0826-03:04/model/rule-table.gz
input-factor=0 output-factor=0
FeatureFunction: TranslationModel0 start: 3 end: 6
line=PhraseDictionaryMemory name=TranslationModel1 num-features=1
path=/tmp/shared/moses-training/chuy_0826-03:04/model/glue-grammar
input-factor=0 output-factor=0
FeatureFunction: TranslationModel1 start: 7 end: 7
line=SRILM name=LM0 factor=0 path=/tmp/shared/27uy.o5.lm order=5
FeatureFunction: LM0 start: 8 end: 8
Loading UnknownWordPenalty0
Loading WordPenalty0
Loading PhrasePenalty0
Loading LM0
/tmp/shared/27uy.o5.lm: line 3483: warning: non-zero probability for <unk>
in closed-vocabulary LM
Loading TranslationModel0
Start loading text phrase table. Moses format : [4.729] seconds
Reading /tmp/shared/moses-training/chuy_0826-03:04/model/rule-table.gz
----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100
****************************************************************************************************
Loading TranslationModel1
Start loading text phrase table. Moses format : [760.530] seconds
Reading /tmp/shared/moses-training/chuy_0826-03:04/model/glue-grammar
----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100
****************************************************************************************************
max-chart-span: 20
max-chart-span: 1000
IO from STDOUT/STDIN
Created input-output object : [760.594] seconds
*Unknown input type: 3*
Name:moses VmPeak:22480296 kB VmRSS:22437712 kB
RSSMax:22450000 kB user:762.409 sys:20.947 CPU:783.356
real:783.449
\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\


2.Text Does not include non-printing ltr , rtl chars.



Many Thanks.




> Message: 4
> Date: Fri, 05 Sep 2014 19:28:56 +0700
> From: Tom Hoar <tahoar@precisiontranslationtools.com>
> Subject: Re: [Moses-support] mosesserver -xml-input exclusive
> segmentation fault
> To: moses-support@mit.edu
> Message-ID: <5409AC88.1050407@precisiontranslationtools.com>
> Content-Type: text/plain; charset="utf-8"
>
> Have you run this through the standard moses binary to see if it's a
> mosesserver or shared xml-input problem?
>
> Also, you're forcing Chinese characters to Arabic/Persian script. Does
> your text include the non-printing left-to-right and right-to-left
> characters?
>
>
>
>
>
> On 09/05/2014 06:54 PM, Yusup Ashrap wrote:
> > Hi All,
> >
> > I am having problems with using mosesserver with xml-input flag.
> > mosesserver works fine without adding tagged words, but with tagged
> > words it quits with a segmentation fault.
> >
> > I added detailed issue on github.
> > https://github.com/moses-smt/mosesdecoder/issues/73
> >
> > I started server using this following command.
> > #./mosesserver -xml-input exclusive -f moses.ini
> >
> > and I invoke translation service with my python xml-rpc code.
> >
> >
> >
> > | '? ? ? <np translation="?????" >??</np>'|
> >
> > after this mosesserver quitted with a segmentation fault without any
> > detailed stacktrace.
> >
> >
> >
> >
> > Error is just one line and I could get any related info from google or
> > mailing list.
> >
> > Could you tell me what possibly went wrong or what should I need to
> > make this work?
> >
> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20140906/86407b9b/attachment-0001.htm

------------------------------

Message: 2
Date: Sat, 06 Sep 2014 09:50:33 +0100
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] Incremental retraining
To: sandipandandapat@mail.com, moses-support <moses-support@mit.edu>,
ulrich.germann@gmail.com
Message-ID: <540ACAD9.1040800@gmail.com>
Content-Type: text/plain; charset="windows-1252"

I'm not sure how many scores there are in the phrase table
PhraseDictionaryBitextSampling
It may be 4. In which case you must specify

[feature]
PhraseDictionaryBitextSampling name=PT0 num-features=4 ...

[weight]
PT0= 0.1 0.2 0.3 0.4

On 05/09/14 14:12, Sandipan Dandapat wrote:
> Hi,
> During incremental retraining I specified the following line in moses .ini
> PhraseDictionaryBitextSampling name=PT0 output-factor=0 num-features=9
> path=/home/sandipan/inc_retrain/MT_sys/EnPl/mtdata_pro/train. L1=en
> L2=pl pfwd=g pbwd=g smooth=0 sample=1000 workers=1
>
> this generates the error:
> Feature function PT0 specified 9 dense scores or weights. Actually has 0.
>
> which is solved when num-features is changed to '0'
> but generates the error below:
>
> Exception: moses/TranslationModel/UG/mmsapt.cpp:381 in virtual void
> Moses::Mmsapt::Load() threw util::Exception because
> `this->m_feature_names.size() != this->m_numScoreComponents'.
> At moses/TranslationModel/UG/mmsapt.cpp:381: number of feature values
> provided by Phrase table (7) does not match number specified in Moses
> config file (0)!
> Changing it to 7 also does not help.
>
> I have tried with
> Mmsapt name=PT0 output-factor=0 num-features=0
> base=/home/sandipan/inc_retrain/MT_sys/EnPl/mtdata_pro/train. L1=en L2=pl
>
> but does not work.
> What I need to do at this stage of retraining using moses?
>
> Thanks and regards,
> sandipan
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20140906/7f95e9f8/attachment-0001.htm

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 95, Issue 10
*********************************************

Tweet