Moses-support Digest, Vol 99, Issue 47

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: Problem with syntactic baselines, rule tables are too
small !!!! (Rico Sennrich)
2. XML Maker Bug (Li Xiang)
3. Force alignment (Rajnath Patel)
4. Moses Building Errors (mohamed hasanien)
5. Re: MGIZA is slower than GIZA (Hieu Hoang)


----------------------------------------------------------------------

Message: 1
Date: Wed, 21 Jan 2015 08:54:07 +0000 (UTC)
From: Rico Sennrich <rico.sennrich@gmx.ch>
Subject: Re: [Moses-support] Problem with syntactic baselines, rule
tables are too small !!!!
To: moses-support@mit.edu
Message-ID: <loom.20150121T095050-697@post.gmane.org>
Content-Type: text/plain; charset=utf-8

hxshi <hxshi@...> writes:

>
>
> I am trying to build a syntactic baselines.?Using?FBIS data
> as?the training set.
> But the result what I got is a?too small rule-table, even can not
> translate anything.

Hi Shi Huaxing,

I don't know if these are the only problems, but your extraction parameters
are not ideal for syntactic systems:
https://www.mail-archive.com/moses-support@mit.edu/msg11254.html

also, you can use the option --glue-grammar to create a second translation
model with some simple glue rules that make your system more robust and
should prevent empty output.

best wishes,
Rico



------------------------------

Message: 2
Date: Wed, 21 Jan 2015 16:58:21 +0800
From: Li Xiang <lixiang.ict@gmail.com>
Subject: [Moses-support] XML Maker Bug
To: moses-support <moses-support@mit.edu>
Message-ID: <8E8428C3-83C2-4CEA-B13C-D44F2536FE0A@gmail.com>
Content-Type: text/plain; charset="gb2312"

Hi all,

I am trying to use the xml marker function. But I can not get the xml translation options. According to http://www.statmt.org/moses/?n=Moses.AdvancedFeatures#ntoc33 <http://www.statmt.org/moses/?n=Moses.AdvancedFeatures#ntoc33>, I use ??ne?? as xml label. And I use the following command:

echo 'das ist <ne translation="a cute place">ein kleines haus</ne>' | moses -xml-input exclusive -f moses.ini

The error is

ERROR: Placeholder must only span 1 word: das ist <ne translation="a cute place">ein kleines haus</ne>
Unable to parse XML in line: das ist <ne translation="a cute place">ein kleines haus</ne>
Exception: Unable to parse XML in line: das ist <ne translation="a cute place">ein kleines haus</ne>

I notice that ??The words have to be surrounded by tags, such as <np...> and </np>. The name of the tags can be chosen freely.?? in the tutorial.

But I replace ??ne?? with ??np??. I can get the translation.

I think it is a bug. But I can not find the source file. Does anyone tell me where source file is or explain the reason ?


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150121/9b89fe2a/attachment-0001.htm

------------------------------

Message: 3
Date: Wed, 21 Jan 2015 16:12:23 +0530
From: Rajnath Patel <patelrajnath@gmail.com>
Subject: [Moses-support] Force alignment
To: moses-support <moses-support@mit.edu>
Message-ID:
<CAE-r4ukY=967ttfX87pq-2sQzGWaqcmDxqz_-5j5TYZYxd2s4Q@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hi all,

I was trying to use force alignment script given with mgiza. Full
alignment training generates just "eng-hin.A3.final", where as script
requires 'eng-hin.t3.final', 'eng-hin.d3.final' and 'eng-hin.n3.final' etc.
Kindly suggest how to get these files in full training with mgiza.

Thank you.

--
Regards:
Raj Nath Patel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150121/7d8a59a0/attachment-0001.htm

------------------------------

Message: 4
Date: Wed, 21 Jan 2015 12:38:06 +0000 (UTC)
From: mohamed hasanien <mhmd_hasnen@yahoo.com>
Subject: [Moses-support] Moses Building Errors
To: <moses-support@mit.edu>
Message-ID:
<1191207271.4455555.1421843886853.JavaMail.yahoo@jws10624.mail.bf1.yahoo.com>

Content-Type: text/plain; charset="utf-8"

?Hi All ,
i get this Error?when i try to build Moses using this?
./bjam --with-irstlm=/home/s0565741/workspace/temp/irstlm-5.80.03 -j8 ?
i get this error;
..failed gcc.compile.c++ /home/ubuntu/mosesdecoder/lm/builder/bin/gcc-4.8/release/debug-symbols-on/link-static/threading-multi/dump_counts_main.o...gcc.compile.c++ /home/ubuntu/mosesdecoder/lm/builder/bin/gcc-4.8/release/debug-symbols-on/link-static/threading-multi/print.ovirtual memory exhausted: Cannot allocate memorymohammed hassanien Mohammed
Egyption Programmers Vice-captain
01000121556
Egyption Programmers Syndicate
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150121/be80e341/attachment-0001.htm

------------------------------

Message: 5
Date: Wed, 21 Jan 2015 14:39:43 +0000
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] MGIZA is slower than GIZA
To: xiangli@me.com, moses-support <moses-support@mit.edu>
Message-ID: <54BFBA2F.5040101@gmail.com>
Content-Type: text/plain; charset="utf-8"

it turns out that the build system for mgiza has changed from automake
to cmake a few years ago but the compile flags wasn't properly set in
the new cmake build. Rico and I have just added the flags to cmake and
the manual-compile.sh script.
https://github.com/moses-smt/mgiza/commit/403a56ebc03da4970905195cbd5b4decf58c6686
https://github.com/moses-smt/mgiza/commit/a42e5541b2e321d4cca13f86b3f12141190e7384

This makes mgiza x4 faster:

Before:
420.76user 13.63system 1:42.12elapsed 425%CPU (0avgtext+0avgdata
181616maxresident)k

After:
101.93user 9.34system 0:45.92elapsed 242%CPU (0avgtext+0avgdata
181184maxresident)k

On 19/01/15 19:52, Hieu Hoang wrote:
>
> Hi Li
>
> You're absolutely right, mgiza has gotten slower than giza++! I have
> mgiza from 2 years ago which is x2 faster on 3 cores, but now it's x2
> slower.
>
> Currently rolling back to find the offending commit. Will get back to
> you when I find it
>
> These are the timings:
> *CURRENT MGIZA**
> *1. 25722.74user 904.54system 1:26:41elapsed 511%CPU
> (0avgtext+0avgdata 1906128maxresident)k
> 2. 24095.06user 978.64system 1:20:57elapsed 516%CPU (0avgtext+0avgdata
> 1906176maxresident)k
>
> *GIZA++*
> 4902.41user 21.95system 43:54.45elapsed 186%CPU (0avgtext+0avgdata
> 1906144maxresident)k
>
>
> *OLD **MGIZA*
> 6576.71user 570.62system 24:09.90elapsed 492%CPU (0avgtext+0avgdata
> 1906144maxresident)k
>
>
> On 17/01/15 08:41, Li Xiang wrote:
>> Hi,
>>
>> GIZA:
>>> ${mosesScript}/training/train-model.perl \
>>> --external-bin-dir "${binDir}" \
>>> --root-dir "${trainDir}" \
>>> --corpus train \
>>> --f src \
>>> --e ref \
>>> --alignment grow-diag-final-and \
>>> --parallel \
>>> --first-step 1 \
>>> --last-step 3
>> MGIZA
>>
>>> ${mosesScript}/training/train-model.perl \
>>> --external-bin-dir "${binDir}" \
>>> --root-dir "${trainDir}" \
>>> --corpus train \
>>> --f src \
>>> --e ref \
>>> --alignment grow-diag-final-and \
>>> --parallel \
>>> --first-step 1 \
>>> --last-step 3 \
>>> --mgiza --mgiza-cpus 3
>>
>>
>>> ? 2015?1?17??16:39?Hieu Hoang <Hieu.Hoang@ed.ac.uk
>>> <mailto:Hieu.Hoang@ed.ac.uk>> ???
>>>
>>> ok, can u tell me what u ran for giza++ and mgiza
>>>
>>> On 17 January 2015 at 08:29, Li Xiang <xiangli@me.com
>>> <mailto:xiangli@me.com>> wrote:
>>>
>>> Hi Hieu,
>>>
>>> I give you 5K training data for evaluate the performance. And I
>>> get similar result that mgiza is slower than giza on the data.
>>>
>>>
>>>> ? 2015?1?17??00:34?Hieu Hoang <Hieu.Hoang@ed.ac.uk
>>>> <mailto:Hieu.Hoang@ed.ac.uk>> ???
>>>>
>>>> can you provide the training corpus so I can verify your results?
>>>>
>>>> On 16 January 2015 at 15:53, Li Xiang <lixiang.ict@gmail.com
>>>> <mailto:lixiang.ict@gmail.com>> wrote:
>>>>
>>>> Hi all,
>>>>
>>>> I trained the alignment model on the same data with the
>>>> same parameters using GIZA and MGIZA respectively. The
>>>> training corpus includes 200K sentences. My server has an
>>>> Intel Quad CPU i4790K which has 4 cores and each core has 2
>>>> threads. It costs 2905 seconds for GIZA. But it costs 5259
>>>> seconds for MGIZA with 3 threads. I think MGIZA is much
>>>> faster than GIZA. But I got bad result. I do not know the
>>>> reason is the compile way or others.
>>>>
>>>> Does anyone has relative experience? Thanks.
>>>>
>>>> The following is the training command for MGIZA. And the
>>>> training data is the FBIS zh-en data. But I can not public
>>>> the data because of copyright.
>>>>
>>>>
>>>> ${mosesScript}/training/train-model.perl \
>>>> --external-bin-dir "${binDir}" \
>>>> --root-dir "${trainDir}" \
>>>> --corpus train \
>>>> --f src \
>>>> --e ref \
>>>> --alignment grow-diag-final-and \
>>>> --parallel \
>>>> --first-step 1 \
>>>> --last-step 3 \
>>>> --mgiza --mgiza-cpus 3
>>>> _______________________________________________
>>>> Moses-support mailing list
>>>> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
>>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Hieu Hoang
>>>> Research Associate
>>>> University of Edinburgh
>>>> http://www.hoang.co.uk/hieu
>>>>
>>>
>>>
>>>
>>>
>>>
>>> --
>>> Hieu Hoang
>>> Research Associate
>>> University of Edinburgh
>>> http://www.hoang.co.uk/hieu
>>>
>>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150121/abc9d1cf/attachment.htm

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 99, Issue 47
*********************************************

Tweet