Moses-support Digest, Vol 114, Issue 61

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: Phrases containing brackets mistaken for malformed
nonterminals (Tom Hoar)


----------------------------------------------------------------------

Message: 1
Date: Thu, 28 Apr 2016 22:08:36 +0700
From: Tom Hoar <tahoar@precisiontranslationtools.com>
Subject: Re: [Moses-support] Phrases containing brackets mistaken for
malformed nonterminals
To: moses-support@mit.edu
Message-ID:
<b9b22281-bb08-4dd1-a7b8-766f796a0152@precisiontranslationtools.com>
Content-Type: text/plain; charset="windows-1252"

Moses isn't confused. Square brackets are reserved characters and the
user is responsible for escaping them. To review, these are the
characters users must escape (for various reasons) both for your runtime
with the Moses binary and in your source/target training corpus. Moses'
build-in tokenizer.perl has a command-line option to enable/disable
escaping these. It also has other utilities to do it separately.

&
"
'
<
>
|
[
]

Did I miss any or has anything changed?

Tom


On 4/28/2016 9:58 PM, moses-support-request@mit.edu wrote:
> Date: Tue, 26 Apr 2016 10:24:48 +0800
> From: Markus Saers<masaers@gmail.com>
> Subject: [Moses-support] Phrases containing brackets mistaken for
> malformed nonterminals
> To:"moses-support@mit.edu" <moses-support@mit.edu>
>
> Hello,
>
> I am having problems reading in a phrase table derived from a corpus
> that (I have learned now) contained bracketed expressions such as "to
> like [someone]". I appears that Moses confuses these string with
> nonterminals. I built a regular phrase-based model, so I was a bit
> confused when it contained malformed nonterminals. Is there any way to
> tell Moses that this is a regular phrase-based model, and that it
> (he?) shouldn't look for nonterminals?
>
> /Markus

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20160428/93c756c0/attachment-0001.html

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 114, Issue 61
**********************************************

Tweet