Moses-support Digest, Vol 86, Issue 43

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: EMS-Decoder-Problem (nadeem khan)
2. Re: Moses on cluster (Hieu Hoang)
3. Question: How to parse tokens with slashes in PTB style?
(Martin Velez)
4. role of lexical-weighting (Andrew)


----------------------------------------------------------------------

Message: 1
Date: Mon, 16 Dec 2013 17:05:45 -0800 (PST)
From: nadeem khan <nad_star06@yahoo.com>
Subject: Re: [Moses-support] EMS-Decoder-Problem
To: Barry Haddow <bhaddow@staffmail.ed.ac.uk>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<1387242345.65302.YahooMailNeo@web162403.mail.bf1.yahoo.com>
Content-Type: text/plain; charset="iso-8859-1"

Hi Barry;

I checked all the input steps files generated by EMS but cannot find any thing wrong there anywhere not in sgm file names nor the formatting of those ref and src files.
Cannot getting it as its just not working on my 800 above testset otherwise whole experiment done successfully when using the fr-eng toy data set.


Regards
Nadeem



On Tuesday, December 10, 2013 9:52 PM, Barry Haddow <bhaddow@staffmail.ed.ac.uk> wrote:

Hi Nadeem

It looks like something went wrong earlier in the EVALUATION section,
possibly in the input-from-sgm step. I would check all the steps in this
section for errors.

It is also not clear to me that the truecaser will work with Hindi as it
is designed for languages written in the latin script,

cheers - Barry


On 07/12/13 18:51, nadeem khan wrote:
>
>
>
> Hello Sir;
>
>? ? I am using EMS now and getting into a problem with my data of hindi
> language.
> I ran EMS on config.toy just fine there was not a single error but
> when it comes to my own data and experiment I am getting stuck with
> BLEU and BLEU-c Crashed.
> When I invistaged the problem there is only 1 single Input Segment in
> test.input.tc.1. why and how the EMS taking only 1 segment from my
> input test-src.sgm file? and when I investigated further there is a
> fatal error under EVALUATION_test_nist-bleu-c.1.STDERR of no id in
> srcset. why I am getting that???? as I am giving it the complete sgm
> frame for wrapping out the output.
>
> I am sending you my those testdata sgm file as well as the input and
> output generated by EMS for my dataset.
> Please have a look at it and Reply with your kind comments to resolve
> these? issues
> Waiting for your kind response
>
> THANK YOU
> Regards
> nadeem
>
>
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support


--
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20131216/7c1609fc/attachment-0001.htm

------------------------------

Message: 2
Date: Tue, 17 Dec 2013 01:36:59 +0000
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] Moses on cluster
To: Prasanth K <prasanthk.ms09@gmail.com>, moses-support
<moses-support@MIT.EDU>
Message-ID: <52AFAABB.7050608@gmail.com>
Content-Type: text/plain; charset="iso-8859-1"

there have been people who have used Moses on Torque
http://search.gmane.org/search.php?group=gmane.comp.nlp.moses.user&query=torque

The main things you have to parallelize in the training are:
1. giza++ - i'm not sure if mgiza can be run in a cluster.
2. extract - you have to make a cluster version of the script
scripts/generic/extract-parallel.perl
3. score - cluster version of
scripts/generic/score-parallel.perl

On 16/12/2013 07:07, Prasanth K wrote:
> Hi,
>
> I noticed a recent thread about the use of SGE cluster to run Moses. I
> now know Thomas Meyer provided a script to get Moses (decoder) running
> on a cluster using SGE. Also, that folks at Edinburgh are using a
> large multi-core machine to run Moses (from @Hieu's mail in the same
> thread).
>
> My question is as follows:
> 1. Are there others who have tried to run Moses on clusters that don't
> use SGE compatible schedulers (Torque ...).
>
> 2. I recently got access to a cluster which uses Slurm as a resource
> manager and scheduler. I am wondering if writing a similar perl script
> using Slurm is my best option, if some one hasn't already done so?
>
> 3. If I port the existing parallelizer using Torque to this Slurm, I
> would prefer that the entire pipeline of Moses is run on the cluster
> (unlike moses-parallel.pl <http://moses-parallel.pl> which
> parallelizes the decoder alone), so could you also confirm that this
> script is the best place to start with:
>
> scripts/ems/support/generic-parallelizer.perl
>
> Thanks.
>
> - Prasanth
>
> --
> "Theories have four stages of acceptance. i) this is worthless
> nonsense; ii) this is an interesting, but perverse, point of view,
> iii) this is true, but quite unimportant; iv) I always said so."
>
> --- J.B.S. Haldane
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20131217/da6e9762/attachment-0001.htm

------------------------------

Message: 3
Date: Mon, 16 Dec 2013 17:45:23 -0800
From: Martin Velez <marvelez@ucdavis.edu>
Subject: [Moses-support] Question: How to parse tokens with slashes in
PTB style?
To: moses-support@mit.edu
Message-ID:
<CAMoVDcigQZMBf+CoMSHJN8kPHGZ7yuMCG+hCeMotCGp-5b=sLw@mail.gmail.com>
Content-Type: text/plain; charset="iso-8859-1"

I would like to tokenize tokens with forward slashes in the same way PTB
does it.

For example:
Input: "Resolution 55/100"
Output: "Resolution 55 / 100" (using default options)
Output: "Resolution 55 %/% 100" (using "-penn" options)
Desired Output: "Resolution 55/100"

I skimmed through the code. I found the relevant commented code at line
400 of the tokenizer.perl script. If I commented it out, will I achieve my
goal? Or will I break something?

Saludos!
*Martin Velez*
*UC Davis*
marvelez@ucdavis.edu
http://csiflabs.cs.ucdavis.edu/~marvelez/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20131216/013f2f3b/attachment-0001.htm

------------------------------

Message: 4
Date: Tue, 17 Dec 2013 14:21:50 +0900
From: Andrew <ravenyj@hotmail.com>
Subject: [Moses-support] role of lexical-weighting
To: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID: <BLU171-W23AA4ABC20B1A6937EC970B2DB0@phx.gbl>
Content-Type: text/plain; charset="iso-2022-jp"

Hello,
I am not very sure what lexical-weighting is about.Does it take semantic resemblance into consideration? If so, how? And if not, how does it differ from usual alignment?How would using no-lexical-weighting option in training affect the alignments?
I apologize that the question is rather broad, but I I would greatly appreciate your advice.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20131217/6b9f1b0c/attachment.htm

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 86, Issue 43
*********************************************

Tweet