Moses-support Digest, Vol 99, Issue 69

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: Arabic public corpora for CASMACAT (Philipp Koehn)
2. KenLM's query utility (Tom Hoar)
3. (no subject) (Roee Aharoni)
4. Testing (Benyamin Bosari)


----------------------------------------------------------------------

Message: 1
Date: Fri, 30 Jan 2015 15:37:19 -0500
From: Philipp Koehn <phi@jhu.edu>
Subject: Re: [Moses-support] Arabic public corpora for CASMACAT
To: Mohamed Z <muhamadzeid@gmail.com>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<CAAFADDA=BvZ+95mr5gQtm8rJ1DWBCoo5gggc9Uo_ubA6v8Rdmw@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8

Hi,

you pretty much have to train the engine with the CASMACAT interface
to have everything else properly in place.

Adding "Arabic" as an option would be a very simple fix (it just has
to be added to various menus). There is very little special handling
of specific languages, for instance the tokenizer is very basic;
hopefully it works somewhat with Arabic.

I am not sure how the web-based UI handles the left-to-right order of
Arabic. It may work or not, and may depend on the browser. We have not
tested that.

You can use any corpus to train an engine by just uploading it
yourself. It has to be in XLIFF format.

I'd be keen to help testing this out, so let me know how far you get.

-phi


On Mon, Jan 26, 2015 at 7:33 PM, Mohamed Z <muhamadzeid@gmail.com> wrote:
> Hi all,
>
> I have installed CASMACAT desktop and I would like to add an English>Arabic
> engine. I see that some langauages are listed there, but mostly European
> languages. Is there a chance to add Arabic to the list? That would be really
> awesome.
>
> If this is not possible, how can I add my engine or upload it? I see an
> upload button there, but I have no clue?
>
> thanks,
> Mohamed
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>


------------------------------

Message: 2
Date: Sat, 31 Jan 2015 18:21:22 +0700
From: Tom Hoar <tahoar@precisiontranslationtools.com>
Subject: [Moses-support] KenLM's query utility
To: moses-support@mit.edu
Message-ID: <54CCBAB2.9000306@precisiontranslationtools.com>
Content-Type: text/plain; charset=utf-8; format=flowed

The KenLM `query` utility has changed in how it pipes to stdout. I'm
using Python's subprocess.Popen() and stdin.write() with
stdout.readline(). In Release 1, the output was unbuffered and piping
line-by-line worked. In the newest version (RC-3), piping hangs at
stdout.readline() as though `query` is buffering the output.

Is it possible to add a command line switch to disable output buffers,
similar to what we added to the tokenizer.perl and detokenizer.perl
scripts (-b)?


------------------------------

Message: 3
Date: Sat, 31 Jan 2015 04:07:29 -0800 (PST)
From: "Roee Aharoni" <roee.aharoni@gmail.com>
Subject: [Moses-support] (no subject)
To: moses-support@mit.edu
Message-ID: <1422706048498.c64a6756@Nodemailer>
Content-Type: text/plain; charset="utf-8"

?
Sent from Mailbox
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150131/665ca4b6/attachment-0001.htm

------------------------------

Message: 4
Date: Sat, 31 Jan 2015 15:04:29 +0000 (UTC)
From: Benyamin Bosari <b.bosari2010@yahoo.com>
Subject: [Moses-support] Testing
To: Moses-support Support <moses-support@mit.edu>
Message-ID:
<790572103.48549.1422716669094.JavaMail.yahoo@mail.yahoo.com>
Content-Type: text/plain; charset="utf-8"

Hi dear guys,
I have some problems in "Testing part" of baseline system!
After running these commands:
mkdir ~/working/binarised-model?cd ~/working?~/mosesdecoder/bin/processPhraseTableMin \?-in train/model/phrase-table.gz -nscores 4 \?-out binarised-model/phrase-table?~/mosesdecoder/bin/processLexicalTableMin \?-in train/model/reordering-table.wbe-msd-bidirectional-fe.gz \?-out binarised-model/reordering-table
The existed files in "binarised-model" folder are "phrase-table.minphr" and "reordering-table.minphr"! They are not just "phrase-table" and "reordering-table"!

Hence, after applying the changes for new "moses.ini" file in "binarised-model" folder, when I run "~/mosesdecoder/bin/moses -f ~/working/binarised-model/moses.ini", this error message appears in the terminal:
terminate called after throwing an instance of 'util::Exception'
what(): ?moses/PDTAimp.cpp:173 in void Moses::PDTAimp::Create(const std::vector<long unsigned int>&, const std::vector<long unsigned int>&, const string&, const std::vector<float>&) threw util::Exception'.bin ttable does not existAborted (core dumped)
Could you please help me to solve this problem?
Best Regards,
Benyamin?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150131/63e13159/attachment-0001.htm

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 99, Issue 69
*********************************************

Tweet