Send Moses-support mailing list submissions to
moses-support@mit.edu
To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu
You can reach the person managing the list at
moses-support-owner@mit.edu
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."
Today's Topics:
1. Re: error during tuning step (factored training) (Tomasz Gawryl)
2. Re: How to use Moses Wrapper? (Oskar Solich)
3. Re: error during tuning step (factored training) (Hieu Hoang)
----------------------------------------------------------------------
Message: 1
Date: Mon, 12 Oct 2015 09:20:23 +0200
From: "Tomasz Gawryl" <tomasz.gawryl@skrivanek.pl>
Subject: Re: [Moses-support] error during tuning step (factored
training)
To: <moses-support@mit.edu>
Message-ID: <000d01d104be$75bb46c0$6131d440$@gawryl@skrivanek.pl>
Content-Type: text/plain; charset="us-ascii"
Hi Hieu,
Command below (I hope this is it ;).
Regards,
Tomek
moses@SKR-moses:~/working/experiments/FACTORED/steps/1$ more
TRAINING_build-ttable.1
#!/bin/bash
[...]
/home/moses/src/mosesdecoder/scripts/training/train-model.perl -mgiza
-mgiza-cpus 8 -cores 8 -parallel -sort-buffer-size 10G -sort-batch-size 253
-sort-compre
ss gzip -sort-parallel 10 -dont-zip -first-step 6 -last-step 6
-external-bin-dir /home/moses/src/mosesdecoder/tools -f en -e pl -alignment
grow-diag-final-and
-max-phrase-length 5 -parts 5 -reordering msd-bidirectional-fe
-score-options '--GoodTuring --MinScore 2:0.0001' -parallel
-input-factor-max 0 -alignment-fac
tors 0-0 -translation-factors 0-0,1 -reordering-factors 0-0 -decoding-steps
t0 -extract-file /home/moses/working/experiments/FACTORED/model/extract.1
-lexical
-file /home/moses/working/experiments/FACTORED/model/lex.1
-phrase-translation-table
/home/moses/working/experiments/FACTORED/model/phrase-table.1.0-0,1
moses@SKR-moses:~/working/experiments/FACTORED/steps/1$ more
TRAINING_binarize-config.1
#!/bin/bash
[...]
mkdir -p /home/moses/working/experiments/FACTORED/model
/home/moses/src/mosesdecoder/scripts/training/binarize-model.perl
/home/moses/working/experiments/FACTORED/model/moses.ini.1
/home/moses/working/experiments/FACTORED/model/moses.bin.ini.1 -B
inarizer "/home/moses/src/mosesdecoder/bin/CreateOnDiskPt 1 1 4 100 2"
--------------
whats the exact command you used to binarize the phrase-table?
On 11/10/2015 17:00, Tomasz Gawryl wrote:
>
> Hi,
>
> My factored training ends core dumped during tuning step.:
>
> /home/moses/working/experiments/FACTORED/lm/ACROSS=pos.lm.1: line 11:
> warning: n
>
> on-zero probability for <unk> in closed-vocabulary LM
>
> terminate called after throwing an instance of 'util::Exception'
>
> what():
> moses/TranslationModel/RuleTable/PhraseDictionaryOnDisk.cpp:100 in vi
>
> rtual void Moses::PhraseDictionaryOnDisk::InitializeForInput(const
> ttasksptr&) t
>
> hrew util::Exception because `obj->GetMisc("NumTargetFactors") !=
> m_output.size(
>
> )'.
>
> On-disk phrase table has 1 target factors.. The ini file specified 2
> target factors
>
> Aborted (core dumped)
>
> Exit code: 134
>
> My config file is similar to example config file for factored trainings.
>
> What should I do to fix it?
>
> Regards
>
> **
>
> *Tomek *
>
------------------------------
Message: 2
Date: Mon, 12 Oct 2015 08:11:37 +0000
From: Oskar Solich <Oskar.Solich@tsd-int.com>
Subject: Re: [Moses-support] How to use Moses Wrapper?
To: "'Hieu Hoang'" <hieuhoang@gmail.com>, "'moses-support@mit.edu'"
<moses-support@mit.edu>
Message-ID: <70FE68297B00D841978E41EEAB9B3BDEF3AC99@dctwo.tsd.local>
Content-Type: text/plain; charset="iso-8859-1"
Hi Hieu,
it is here in the repository:
https://github.com/moses-smt/mosesdecoder/blob/master/scripts/server/moses.py
Regards
Oskar Solich
IT and Organisation
tsd Technik-Sprachendienst GmbH
?bersetzung & Dokumentation
Mittelstra?e 12 - 14
50672 K?ln
Tel. +49-221-92 59 86-34
Fax +49-221-2 57 73 85
oskar.solich@tsd-int.com<mailto:oskar.solich@tsd-int.com>
www.tsd-int.com<blocked::file:///C:/Dokumente%20und%20Einstellungen/gabi/Anwendungsdaten/Microsoft/Signatures/www.tsd-int.com>
Handelsregister Reg. Amtsgericht K?ln - HRB 9533
Sitz der Gesellschaft: K?ln
Gesch?ftsf?hrende Gesellschafterin: Dina Freibott
The information in this e-mail and its attachments (if any) is confidential and may contain legally privileged information. If you are not the intended recipient please notify the sender immediately and do not disclose the contents to any third party. Any opinions expressed in the above e-mail are those of the author personally, and not necessarily those of tsd Technik-Sprachendienst GmbH. Although tsd believes this e-mail and any attachments are free of any virus or other malware which may affect a computer, it is the responsibility of the recipient to ensure that it is virus free, and tsd does not accept any responsibility for any loss or damage arising in any way from its use.
Von: Hieu Hoang [mailto:hieuhoang@gmail.com]
Gesendet: Freitag, 9. Oktober 2015 11:17
An: Oskar Solich; 'moses-support@mit.edu'
Betreff: Re: [Moses-support] How to use Moses Wrapper?
where did you get script/server/moses.py script from? It's not in the moses repository,
On 07/10/2015 08:40, Oskar Solich wrote:
Hi,
I want to use the wrapper moses.py at scripts/server/ but I can't find any documentation for the wrapper.
I have a CAT tool which sends and gets RPC2 requests and a moses-server and I want tokenize the sentences before translating (and detokenize after).
Regards
Oskar Solich
IT and Organisation
tsd Technik-Sprachendienst GmbH
?bersetzung & Dokumentation
Mittelstra?e 12 - 14
50672 K?ln
Tel. +49-221-92 59 86-34
Fax +49-221-2 57 73 85
oskar.solich@tsd-int.com<mailto:oskar.solich@tsd-int.com>
www.tsd-int.com<http://www.tsd-int.com>
Handelsregister Reg. Amtsgericht K?ln - HRB 9533
Sitz der Gesellschaft: K?ln
Gesch?ftsf?hrende Gesellschafterin: Dina Freibott
The information in this e-mail and its attachments (if any) is confidential and may contain legally privileged information. If you are not the intended recipient please notify the sender immediately and do not disclose the contents to any third party. Any opinions expressed in the above e-mail are those of the author personally, and not necessarily those of tsd Technik-Sprachendienst GmbH. Although tsd believes this e-mail and any attachments are free of any virus or other malware which may affect a computer, it is the responsibility of the recipient to ensure that it is virus free, and tsd does not accept any responsibility for any loss or damage arising in any way from its use.
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu<mailto:Moses-support@mit.edu>
http://mailman.mit.edu/mailman/listinfo/moses-support
--
Hieu Hoang
http://www.hoang.co.uk/hieu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20151012/ae5f4d32/attachment-0001.html
------------------------------
Message: 3
Date: Mon, 12 Oct 2015 10:19:22 +0100
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] error during tuning step (factored
training)
To: Tomasz Gawryl <tomasz.gawryl@skrivanek.pl>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAEKMkbg7eNMJ_Wv-kht95sBQKxxuY9LBEPL4BuxTfq2pZ3XqGg@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
since you have 2 factors on the target side, the binarization command
should be
CreateOnDiskPt 1 *2* 4 100 2
If you also have 2 factors on the source side, then it should be
CreateOnDiskPt *2* *2* 4 100 2
Hieu Hoang
http://www.hoang.co.uk/hieu
On 12 October 2015 at 08:20, Tomasz Gawryl <tomasz.gawryl@skrivanek.pl>
wrote:
> Hi Hieu,
>
> Command below (I hope this is it ;).
>
> Regards,
> Tomek
>
> moses@SKR-moses:~/working/experiments/FACTORED/steps/1$ more
> TRAINING_build-ttable.1
> #!/bin/bash
>
> [...]
>
> /home/moses/src/mosesdecoder/scripts/training/train-model.perl -mgiza
> -mgiza-cpus 8 -cores 8 -parallel -sort-buffer-size 10G -sort-batch-size 253
> -sort-compre
> ss gzip -sort-parallel 10 -dont-zip -first-step 6 -last-step 6
> -external-bin-dir /home/moses/src/mosesdecoder/tools -f en -e pl -alignment
> grow-diag-final-and
> -max-phrase-length 5 -parts 5 -reordering msd-bidirectional-fe
> -score-options '--GoodTuring --MinScore 2:0.0001' -parallel
> -input-factor-max 0 -alignment-fac
> tors 0-0 -translation-factors 0-0,1 -reordering-factors 0-0 -decoding-steps
> t0 -extract-file /home/moses/working/experiments/FACTORED/model/extract.1
> -lexical
> -file /home/moses/working/experiments/FACTORED/model/lex.1
> -phrase-translation-table
> /home/moses/working/experiments/FACTORED/model/phrase-table.1.0-0,1
>
> moses@SKR-moses:~/working/experiments/FACTORED/steps/1$ more
> TRAINING_binarize-config.1
> #!/bin/bash
>
> [...]
>
> mkdir -p /home/moses/working/experiments/FACTORED/model
> /home/moses/src/mosesdecoder/scripts/training/binarize-model.perl
> /home/moses/working/experiments/FACTORED/model/moses.ini.1
> /home/moses/working/experiments/FACTORED/model/moses.bin.ini.1 -B
> inarizer "/home/moses/src/mosesdecoder/bin/CreateOnDiskPt 1 1 4 100 2"
>
>
>
> --------------
> whats the exact command you used to binarize the phrase-table?
>
> On 11/10/2015 17:00, Tomasz Gawryl wrote:
> >
> > Hi,
> >
> > My factored training ends core dumped during tuning step.:
> >
> > /home/moses/working/experiments/FACTORED/lm/ACROSS=pos.lm.1: line 11:
> > warning: n
> >
> > on-zero probability for <unk> in closed-vocabulary LM
> >
> > terminate called after throwing an instance of 'util::Exception'
> >
> > what():
> > moses/TranslationModel/RuleTable/PhraseDictionaryOnDisk.cpp:100 in vi
> >
> > rtual void Moses::PhraseDictionaryOnDisk::InitializeForInput(const
> > ttasksptr&) t
> >
> > hrew util::Exception because `obj->GetMisc("NumTargetFactors") !=
> > m_output.size(
> >
> > )'.
> >
> > On-disk phrase table has 1 target factors.. The ini file specified 2
> > target factors
> >
> > Aborted (core dumped)
> >
> > Exit code: 134
> >
> > My config file is similar to example config file for factored trainings.
> >
> > What should I do to fix it?
> >
> > Regards
> >
> > **
> >
> > *Tomek *
> >
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20151012/54c688e0/attachment.html
------------------------------
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
End of Moses-support Digest, Vol 108, Issue 43
**********************************************
Subscribe to:
Post Comments (Atom)
0 Response to "Moses-support Digest, Vol 108, Issue 43"
Post a Comment