Moses-support Digest, Vol 111, Issue 4

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: MT Marathon 2010 page hacked. (Philipp Koehn)
2. Re: Factored instead of Phrase-based Model? (Shaimaa Marzouk)


----------------------------------------------------------------------

Message: 1
Date: Wed, 6 Jan 2016 12:23:34 -0500
From: Philipp Koehn <phi@jhu.edu>
Subject: Re: [Moses-support] MT Marathon 2010 page hacked.
To: liling tan <alvations@gmail.com>
Cc: moses-support <moses-support@mit.edu>, Ventsisav Zhechev
<contact@ventsislavzhechev.eu>
Message-ID:
<CAAFADDCT=y04AB=q=ap2HbUdrAGdoVffqONPwcGYoaExYSiHmg@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hi,

I did not find anything hacked about the page, but it is maintained
by Ventsislav Zhechev.

-phi

On Wed, Jan 6, 2016 at 8:07 AM, liling tan <alvations@gmail.com> wrote:

> Dear Moses / MT Marathon organizers,
>
> I'm not sure whether this is the right place to report this.
>
> I was trying to retrieve a page from MT Marathon 2010 and it seems like a
> Russian hacker hacked the page and took over it:
> http://www.mtmarathon2010.info/ (see the lower right corner).
>
> And it's using the high google pagerank index to bait people onto pages
> like: http://www.mtmarathon2010.info/web/Program_files/survey.pdf
>
> Regards,
> Liling
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20160106/72f42bd4/attachment-0001.html

------------------------------

Message: 2
Date: Wed, 6 Jan 2016 17:27:02 +0000 (UTC)
From: Shaimaa Marzouk <marzouk_s@yahoo.de>
Subject: Re: [Moses-support] Factored instead of Phrase-based Model?
To: <Moses-support@mit.edu>, Ondrej Bojar <bojar@ufal.mff.cuni.cz>
Message-ID:
<839282891.1563382.1452101222504.JavaMail.yahoo@mail.yahoo.com>
Content-Type: text/plain; charset="utf-8"

Dear Ondrej & Moses-Team,

@Ondrej: thanks a lot for your quick feedback.

The phrase "ich habe" does not appear in the phrase table. The word alignment file includes only the first 4 sentences of the training data.

I have separated the sentence (ich habe das auto verkauf) in a separate "in" file, but got the same result. I also tried another sentence (ich verkaufe das auto), also here "ich verkaufe" can not be translated. I repeated the exact sentence (ich verkaufe das auto) many times in the training data and still get the same result.
I attach the word alignment, phrase table, training data and verbose result.. and would be very grateful to receive any tip.

I would also highly appreciate, if you could let me know, where can I find information about
1.???how to prepare the training data with additional factors, before training the Factored Model?
2.???how to train the Language Model that considers the POS?

I think that sooner or later, the sentences will get complexer and I would need to work with a Factored Model.


Many Thanks
Shaimaa






--------------------------------------------
Ondrej Bojar <bojar@ufal.mff.cuni.cz> schrieb am Mi, 6.1.2016:

Betreff: Re: [Moses-support] Factored instead of Phrase-based Model?
An: "Shaimaa Marzouk" <marzouk_s@yahoo.de>, "Shaimaa Marzouk" <marzouk_s@yahoo.de>, Moses-support@mit.edu
Datum: Mittwoch, 6. Januar, 2016 08:42 Uhr

Dear Shaimaa,

Adding factors can only
increase any out-of-vocabulary issues.

Use -v (perhaps even a higher verbosity level)
in moses to see what all translation options are considered
for the problematic sentence. There could be some
unfortunate weight settings? that for some reason prefer
identity translation. (The identity translation must however
appear in the data, or the source word must not appear in
the data, otherwise Moses would not produce identity
translation at all.)

And
then go back to the phrase table and manually search for the
lines that are supposed to cover the missing words. Here you
may find the identity entries.

And then go back and check the word alignment
this (test) sentence got in the training data. There are
most likely some issues with the alignment that prevented
proper translations to be extracted.

Best, Ondrej.


On January 6, 2016 4:48:26 AM CET, Shaimaa
Marzouk <marzouk_s@yahoo.de>
wrote:
>Dear Moses-Team,
>
>I am trying to
translate two short sentences included in the same file
>from German into English using a
?Phrase-based Model?. The first
>sentence (das auto wurde verkauft) is
translated correctly, while the
>second
is partly translated:
>
>I receive as a result for ?ich habe das
auto verkauft?
>Ich|UNK|UNK|UNK
habe|UNK|UNK|UNK the car sold? [11111]?
>[total=-203.330]???core=(-200.000,
-5.000, 5.000, 0.000, 0.000, 0.000,
>0.000, 0.000, -18.660)
>
>I tried to modify the
training data in different ways, and at last
>included the exact sentence (along with its
translation) in the
>training data (see
attachment). But, I still get the same result.
>
>Do I need to use a
?Factored Translation Model? instead of the
>?Phrase-based Model? to be able to
translate this sentence? If yes, I
>find
here http://www.statmt.org/moses/?n=Moses.FactoredTutorial
>explanation of how to train Factored
Models. Could you please tell me,
>where
can I find information about
>1.???
how to prepare the training data with additional factors,
before
>training the Factored Model?
>2.??? how to train the Language Model
that considers the POS?
>
>I currently use KenLM and Giza++.
>
>Thanks a lot for your
support.
>
>Kind
regards,
>Shaimaa
>
>------------------------------------------------------------------------
>
>_______________________________________________
>Moses-support mailing list
>Moses-support@mit.edu
>http://mailman.mit.edu/mailman/listinfo/moses-support

--
Ondrej
Bojar (mailto:obo@cuni.cz / bojar@ufal.mff.cuni.cz)
http://www.cuni.cz/~obo
-------------- next part --------------
A non-text attachment was scrubbed...
Name: aligned.grow-diag-final
Type: application/octet-stream
Size: 80 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20160106/ae84cf3c/attachment.obj
-------------- next part --------------
A non-text attachment was scrubbed...
Name: phrase-table.gz
Type: application/x-gzip
Size: 425 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20160106/ae84cf3c/attachment.gz
-------------- next part --------------
A non-text attachment was scrubbed...
Name: verbose.docx
Type: application/vnd.openxmlformats-officedocument.wordprocessingml.document
Size: 12844 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20160106/ae84cf3c/attachment.bin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: car-ready2016-2.de
Type: application/octet-stream
Size: 681 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20160106/ae84cf3c/attachment-0001.obj
-------------- next part --------------
A non-text attachment was scrubbed...
Name: car-ready2016-2.en
Type: application/octet-stream
Size: 488 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20160106/ae84cf3c/attachment-0002.obj

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 111, Issue 4
*********************************************

Tweet