Moses-support Digest, Vol 153, Issue 11

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: ERROR: Wrong format of --lm. Expected: --lm
factor:order:filename //train-model.perl line 605. (Hieu Hoang)


----------------------------------------------------------------------

Message: 1
Date: Tue, 30 Jul 2019 08:44:49 +0100
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] ERROR: Wrong format of --lm. Expected:
--lm factor:order:filename //train-model.perl line 605.
To: "Aaron L.-F. Han" <hanlifengaaron@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAEKMkbi4pEydETQ4JQ=r+AyVAzoG2GWbR=MPXGunJwk_cw8h6w@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

On Mon, 29 Jul 2019, 19:58 Aaron Poet, <hanlifengaaron@gmail.com> wrote:

> Hi Moses-Support
>
> I firstly build the lm as below:
>
> (base) aaronhan@grove2:~/Moses$ head -20
> /home/aaronhan/Moses/lm/wmt18_5maaZhEn.arpa.en
>
> \data\
>
> ngram 1=553697
>
> ngram 2=8790963
>
> ngram 3=28838697
>
>
> \1-grams:
>
> -6.993945 <unk> 0
>
> 0 <s> -1.6929523
>
> -2.0611665 </s> 0
>
> -2.3942525 the -0.9461817
>
> -3.6470015 show -0.6162386
>
> -4.1808434 stars -0.5792577
>
> -4.1823444 X -0.53191173
>
> -5.0213814 Girls -0.33374956
>
> -2.5064495 - -0.6326866
>
> -2.5754578 a -0.9741243
>
> -5.1265607 troupe -0.3527482
>
> -2.2843323 of -0.91076595
>
> -4.663717 talented -0.3983
>
> -5.2061567 topless -0.27003714
>
> Then, I binary it as bellow:
>
> /home/aaronhan/Moses/mosesdecoder/bin/build_binary
> /home/aaronhan/Moses/lm/wmt18_5maaZhEn.arpa.en
> /home/aaronhan/Moses/lm/wmt18_5maaZhEn.blm.en
>
> (base) aaronhan@grove2:~/Moses$ ls -lh
> /home/aaronhan/Moses/lm/wmt18_5maaZhEn.blm.en
>
> -rw-r--r-- 1 aaronhan users 715M Jul 29 18:38
> /home/aaronhan/Moses/lm/wmt18_5maaZhEn.blm.en
>
> when I test it, it looks working out:
>
> (tensorflow) aaronhan@grove2:~/Moses/working$ echo "is this an English
> sentence ?" \
>
> > | /home/aaronhan/Moses/mosesdecoder/bin/query
> /home/aaronhan/Moses/lm/wmt18_5maaZhEn.blm.en
>
> is=23 2 -2.5766807 this=321 3 -1.0483611 an=108 3 -2.1032705
> English=1731 2 -2.8970845 sentence=3266 3 -2.8317716 ?=815 2
> -2.0824053 </s>=2 3 -0.035550244 Total: -13.575124 OOV: 0
>
> Perplexity including OOVs: 86.9567697150713
>
> Perplexity excluding OOVs: 86.9567697150713
>
> OOVs: 0
>
> Tokens: 7
>
> Name:query VmPeak:748300 kB VmRSS:4052 kB RSSMax:503644 kB
> user:0.008 sys:0.652 CPU:0.66 real:61.9797
>
> However, when I run the overall Moses training, it produced error as
> below: *Wrong format of --lm. Expected: --lm factor:order:filename*
>
> (tensorflow) aaronhan@grove2:~/Moses/working$ nohup nice
> /home/aaronhan/Moses/mosesdecoder/scripts/training/train-model.perl
> -root-dir train -corpus
> /home/aaronhan/NMT/THUMT/corpora/preprocessed_wmt18en5maa.true -f zh -e en
> -alignment grow-diag-final-and -reordering msd-bidirectional-fe -lm
> 0:3$HOME/Moses/lm/wmt18_5maaZhEn.blm.en:8 -external-bin-dir
> /home/aaronhan/Moses/mosesdecoder/tools/ >& training.out &
>
> [1] 23346
>
> You forgot a colon between 3 and $HOME

>
> GNU nano 2.7.4
> File: training.out
>
>
>
> nohup: ignoring input
>
> Using SCRIPTS_ROOTDIR: /home/aaronhan/Moses/mosesdecoder/scripts
>
> Using single-thread GIZA
>
> using gzip
>
> ERROR: Wrong format of --lm. Expected: --lm factor:order:filename at
> /home/aaronhan/Moses/mosesdecoder/scripts/training/train-model.perl line
> 605.
>
>
>
> Can you let me know how to solve this, please?
>
> Kind regards,
> Aaron
>
> --
>
> PhD student, Msc. (excellent), Bsc. (Maths)
>
> Google-Scholar
> <https://scholar.google.nl/citations?user=_vf3E2QAAAAJ&hl=en> ,
> Presentation <https://www.slideshare.net/AaronHanLiFeng>-slides,
> Research-Gate <https://www.researchgate.net/profile/Aaron_L-F_Han>
>
> Google-site <https://sites.google.com/view/poetgarden/home> Linkedin
> <https://www.linkedin.com/in/aaronhan/>,
>
> ADAPT <https://www.adaptcentre.ie/> ResearchCentre &
>
> School of Computing, DCU,
>
> Dublin, Ireland
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20190730/bae8fea7/attachment.html

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 153, Issue 11
**********************************************

Tweet