Moses-support Digest, Vol 115, Issue 14

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Hebrew translation tools (Jo?o Pita Costa)
2. Finding the top 5 most ambiguous words (Joe Jean)
3. Re: Finding the top 5 most ambiguous words (Matt Post)
4. Re: Finding the top 5 most ambiguous words (Barry Haddow)


----------------------------------------------------------------------

Message: 1
Date: Fri, 13 May 2016 11:33:16 +0200
From: Jo?o Pita Costa <joao.pitacosta@ijs.si>
Subject: [Moses-support] Hebrew translation tools
To: moses-support@mit.edu
Message-ID:
<CADMpCmSfSJPLgev067-tTk8qvpX63xYymfkwuzQ0uKVJfNDPhQ@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hi all,

Here at the IJS we are setting up Moses machine translation for several
languages including Hebrew and Arabic. I have been searching through Moses
support forum and through the internet for tools for pre-processing of
corpus for the Hebrew-English and English-Hebrew translations. We will also
need to do the post-processing of the outcome. One of the problems is that
we do not know the language enough to understand what we need. I would
appreciate if somebody would tell us what tools we really need and maybe
point us to trustful open code if existing. I have the same questions above
for arabic language. Thanks in advance.

Best wishes,

*Jo?o Pita Costa*
Researcher Raziskovalec



Artificial Intelligence Laboratory
Jozef Stefan Institute
Jamova 39, Ljubljana
Slovenia

Room: N406
Phone: + 386 1 477 3905
Skype: joaopitacosta
URL: http://ailab.ijs.si/people/?uid=65
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20160513/9377b2a2/attachment-0001.html

------------------------------

Message: 2
Date: Wed, 11 May 2016 01:53:04 -0500
From: Joe Jean <joe@joejean.net>
Subject: [Moses-support] Finding the top 5 most ambiguous words
To: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID: <BLU437-SMTP87260B4AEF7DC2CFF820C8AB720@phx.gbl>
Content-Type: text/plain; charset="iso-8859-1"

Hello,
How would you go about finding the top 5 most ambiguous words in a translation system just by looking at the phrase table and the lexical translation tables? Thanks.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20160511/dbb35a6e/attachment-0001.html

------------------------------

Message: 3
Date: Fri, 13 May 2016 10:10:56 -0400
From: Matt Post <post@cs.jhu.edu>
Subject: Re: [Moses-support] Finding the top 5 most ambiguous words
To: Joe Jean <joe@joejean.net>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID: <B70CB353-9523-4074-8D89-45572878F60F@cs.jhu.edu>
Content-Type: text/plain; charset="iso-8859-1"

gzip -cd model/phrase-table.gz | cut -d\| -f1 | sort | uniq -c | sort -nr | head -n5

(according to one definition of "ambiguous")

> On May 11, 2016, at 2:53 AM, Joe Jean <joe@joejean.net> wrote:
>
> Hello,
>
> How would you go about finding the top 5 most ambiguous words in a translation system just by looking at the phrase table and the lexical translation tables? Thanks.
>
> <http://www.facebook.com/joe.jean3>_______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
> http://mailman.mit.edu/mailman/listinfo/moses-support <http://mailman.mit.edu/mailman/listinfo/moses-support>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20160513/880fdfa6/attachment-0001.html

------------------------------

Message: 4
Date: Fri, 13 May 2016 15:23:40 +0100
From: Barry Haddow <bhaddow@staffmail.ed.ac.uk>
Subject: Re: [Moses-support] Finding the top 5 most ambiguous words
To: Matt Post <post@cs.jhu.edu>, Joe Jean <joe@joejean.net>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID: <5735E36C.5050702@staffmail.ed.ac.uk>
Content-Type: text/plain; charset="windows-1252"

Hi Joe

You could also look at the entropy of the distribution. I'll leave Matt
to post the one-liner for that one,

cheers - Barry

On 13/05/16 15:10, Matt Post wrote:
> gzip -cd model/phrase-table.gz | cut -d\| -f1 | sort | uniq -c | sort
> -nr | head -n5
>
> (according to one definition of "ambiguous")
>
>> On May 11, 2016, at 2:53 AM, Joe Jean <joe@joejean.net
>> <mailto:joe@joejean.net>> wrote:
>>
>> Hello,
>>
>> How would you go about finding the top 5 most ambiguous words in a
>> translation system just by looking at the phrase table and the
>> lexical translation tables? Thanks.
>>
>> <http://www.facebook.com/joe.jean3>
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20160513/f8f64cc3/attachment.html
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: not available
Url: http://mailman.mit.edu/mailman/private/moses-support/attachments/20160513/f8f64cc3/attachment.pl

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 115, Issue 14
**********************************************

Tweet