Home » Uncategories » Moses-support Digest, Vol 103, Issue 19

Moses-support Digest, Vol 103, Issue 19

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: Europarl monolingual corpus (Hieu Hoang)
2. Problem with qsub when running experiments
(carla.parra@hermestrans.com)
3. Re: Problem with qsub when running experiments (Barry Haddow)
4. Fwd: Transliteration model is using processPhraseTable, which
is not found in Moses version 3.0 (Ergun Bicici)


----------------------------------------------------------------------

Message: 1
Date: Wed, 6 May 2015 14:17:21 +0400
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] Europarl monolingual corpus
To: joerg <tiedeman@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAEKMkbiJ8EyGwCZ_CmKvt1RV24mtGVrddH5gezGQSPj-K=iDLw@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

ah thanks, just what i was looking for.



Hieu Hoang
Researcher
New York University, Abu Dhabi
http://www.hoang.co.uk/hieu

On 6 May 2015 at 11:18, joerg <tiedeman@gmail.com> wrote:

>
> Go to http://opus.lingfil.uu.se
> Select the language pair your interested in and click on the language ID's
> in the column "mono" (or "raw" next to it) to download the data you like to
> use. Europarl is version 7 in the list. You can also take them from here:
> http://opus.lingfil.uu.se/Europarl/mono/
>
> Best,
> J?rg
>
>
> **********************************************************************************
> J?rg Tiedemann
> http://stp.lingfil.uu.se/~joerg/
>
>
>
> On May 6, 2015, at 7:25 AM, Hieu Hoang wrote:
>
> ah thx. The tgz file only has data for a subset of the languages.
>
> It would be useful to be able to download them all, or at least know how
> to extract them from the raw data.
>
>
>
> Hieu Hoang
> Researcher
> New York University, Abu Dhabi
> http://www.hoang.co.uk/hieu
>
> On 6 May 2015 at 03:04, Ulrich Germann <ulrich.germann@gmail.com> wrote:
>
>> Extract it from commoncrawl, of course! ;-)
>>
>> ... or get it here:
>> http://www.statmt.org/wmt13/training-monolingual-europarl-v7.tgz
>>
>> - Uli
>>
>> On Mon, May 4, 2015 at 5:46 AM, Hieu Hoang <hieuhoang@gmail.com> wrote:
>>
>>> What's the easiest way get the single-language data from the Europarl
>>> corpus as described in the 1st table in:
>>> http://statmt.org/europarl/
>>>
>>> I tried downloading the xml source
>>> http://statmt.org/europarl/v7/europarl.tgz
>>> stripping the xml and running split-sentence.perl, but this takes an
>>> unfathomably long time
>>>
>>> Hieu Hoang
>>> Researcher
>>> New York University, Abu Dhabi
>>> http://www.hoang.co.uk/hieu
>>>
>>> _______________________________________________
>>> Moses-support mailing list
>>> Moses-support@mit.edu
>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>
>>>
>>
>>
>> --
>> Ulrich Germann
>> Senior Researcher
>> School of Informatics
>> University of Edinburgh
>>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150506/e3aca76a/attachment-0001.htm

------------------------------

Message: 2
Date: Wed, 06 May 2015 13:56:53 +0200
From: carla.parra@hermestrans.com
Subject: [Moses-support] Problem with qsub when running experiments
To: moses-support@mit.edu
Message-ID: <3d75bb78f534262a812b2a419f8d532b@hermestrans.com>
Content-Type: text/plain; charset=UTF-8; format=flowed

Hi everyone,

First of all, thanks for reading and hopefully giving me some useful
pointer. I am running several SMT experiments on an Ubuntu machine. It
is a multicore machine, but I have commented out the options for running
experiments on multicore machines in the config.file.

Up to last week, I was able to run experiments without problems.
However, since yesterday I get the error:

"Can't exec "qsub": No existe el archivo o el directorio at
/home/hermesta/mosesdecoder/scripts/ems/experiment.perl line 1291."

Does anyone know what could be going on? The only thing I did was
installing asiya to assess MT output. My guess is that somehow MOSES
detects that it is a multicore machine and tries to parallelize jobs.
However, I don't understand why this was not happening last week, for
instance. I have also tried to update MOSES by running "git pull" and I
also reinstalled MOSES hoping this would fix the problem. My background
is linguistics, and thus I am a bit lost now.

Thank you very much,

Carla Parra Escart?n
Marie Curie ER - EXPERT ITN
Hermes Traducciones




------------------------------

Message: 3
Date: Wed, 06 May 2015 13:10:54 +0100
From: Barry Haddow <bhaddow@staffmail.ed.ac.uk>
Subject: Re: [Moses-support] Problem with qsub when running
experiments
To: carla.parra@hermestrans.com, moses-support@mit.edu
Message-ID: <554A04CE.6060906@staffmail.ed.ac.uk>
Content-Type: text/plain; charset=utf-8; format=flowed

Hi Carla

What's your server called?

There's a hard-coded list of Edinburgh machines in ems, so I'm wondering
if it collides with one of them,

cheers - Barry

On 06/05/15 12:56, carla.parra@hermestrans.com wrote:
> Hi everyone,
>
> First of all, thanks for reading and hopefully giving me some useful
> pointer. I am running several SMT experiments on an Ubuntu machine. It
> is a multicore machine, but I have commented out the options for running
> experiments on multicore machines in the config.file.
>
> Up to last week, I was able to run experiments without problems.
> However, since yesterday I get the error:
>
> "Can't exec "qsub": No existe el archivo o el directorio at
> /home/hermesta/mosesdecoder/scripts/ems/experiment.perl line 1291."
>
> Does anyone know what could be going on? The only thing I did was
> installing asiya to assess MT output. My guess is that somehow MOSES
> detects that it is a multicore machine and tries to parallelize jobs.
> However, I don't understand why this was not happening last week, for
> instance. I have also tried to update MOSES by running "git pull" and I
> also reinstalled MOSES hoping this would fix the problem. My background
> is linguistics, and thus I am a bit lost now.
>
> Thank you very much,
>
> Carla Parra Escart?n
> Marie Curie ER - EXPERT ITN
> Hermes Traducciones
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support


--
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.



------------------------------

Message: 4
Date: Wed, 6 May 2015 13:45:44 +0100
From: Ergun Bicici <Ergun.Bicici@computing.dcu.ie>
Subject: [Moses-support] Fwd: Transliteration model is using
processPhraseTable, which is not found in Moses version 3.0
To: moses-support <moses-support@mit.edu>, Nadir Durrani
<nadir.durrani@nu.edu.pk>
Message-ID:
<CAB2pGnee2pp38fdVAJkELFsORUwXU88EtererTGkKOZVjqOWxA@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Dear Nadir,

Thank you very much for explaining transliteration. I have "yes" for both
transliteration-module and post-decoding-transliteration in the EMS
configuration file used for en-ru.

Best Regards,
Ergun

Ergun Bi?ici, CNGL, School of Computing, DCU, www.cngl.ie
http://www.computing.dcu.ie/~ebicici/


---------- Forwarded message ----------
From: Nadir Durrani <nadir.durrani@nu.edu.pk>
Date: Wed, May 6, 2015 at 11:17 AM
Subject: Re: Transliteration model is using processPhraseTable, which is
not found in Moses version 3.0
To: Ergun Bicici <Ergun.Bicici@computing.dcu.ie>


Hi Ergun,

If you are only going to do

transliteration-module = "yes"

Moses will train the transliteration system but not going to do
anything with it. You have to select whether you want to use
post-deocoding or in-decoding transliteration.

In post-decoding method, transliteration is done in the post-decoding
step i.e. the decoder has translated all the sentences and now you
just need to replace OOV words with their best transliteration given
the context. This is Method 2 as described in the following paper

http://aclweb.org/anthology//E/E14/E14-4029.pdf

you can enable it by using

post-decoding-transliteration = "yes"


Using in-decoding method (Method 3 in the paper), you do
transliteration inside the decoder on the fly. The advantage of this
over Method 2 in theory is that you can also reorder the OOV word and
make use of other features. But it does not give any clear-cut gains.

More details here:

http://www.statmt.org/moses/?n=Advanced.OOVs

Nadir

>> On Tue, May 5, 2015 at 5:33 PM, Ergun Bicici
>> <Ergun.Bicici@computing.dcu.ie> wrote:
>> >
>> > Hi Nadir,
>> >
>> > I am using Moses 3.0 and for transliteration to work, I copied
>> > scripts/Transliteration/ from latest onto Moses 3.0 path, re-ran, and
>> > obtained translation results.
>> >
>> >
>> > Best Regards,
>> > Ergun
>> >
>> > Ergun Bi?ici, CNGL, School of Computing, DCU, www.cngl.ie
>> > http://www.computing.dcu.ie/~ebicici/
>> >
>> >
>> > On Mon, May 4, 2015 at 7:32 AM, Nadir Durrani <nadir.durrani@nu.edu.pk>
>> > wrote:
>> >>
>> >> Hi Ergun,
>> >>
>> >> processPhraseTable is no longer supported by Moses. But I see that
>> >> Phil Williams has already fixed this problem in transliteration
>> >> module, by changing
>> >>
>> >> `$MOSES_SRC/scripts/training/filter-model-given-input.pl
>> >> $TRANSLIT_MODEL/evaluation/$eval_file.filtered
>> >> $TRANSLIT_MODEL/evaluation/$eval_file.moses.table.ini
>> >> $TRANSLIT_MODEL/evaluation/$eval_file -Binarizer
>> >> "$MOSES_SRC/bin/processPhraseTable"`;
>> >>
>> >> to
>> >>
>> >> `$MOSES_SRC/scripts/training/filter-model-given-input.pl
>> >> $TRANSLIT_MODEL/evaluation/$eval_file.filtered
>> >> $TRANSLIT_MODEL/evaluation/$eval_file.moses.table.ini
>> >> $TRANSLIT_MODEL/evaluation/$eval_file -Binarizer
>> >> "$MOSES_SRC/bin/CreateOnDiskPt 1 1 4 100 2"`;
>> >>
>> >> in
>> >>
>> >> path-to-moses/scripts/Transliteration/in-decoding-transliteration.pl
>> >>
>> >> Here's the commit
>> >>
>> >>
>> >>
>> >>
https://github.com/moses-smt/mosesdecoder/commit/7e54e23fe234ac48f44beeee0e473d09a5b4d5f6
>> >>
>> >> May be you pulled and in between version where the processPhraseTable
>> >> was removed but transliteration scripts were not fixed.
>> >>
>> >> Cheers,
>> >> Nadir
>> >>
>> >>
>> >> On Mon, May 4, 2015 at 7:46 AM, <moses-support-request@mit.edu>
wrote:
>> >> > Send Moses-support mailing list submissions to
>> >> > moses-support@mit.edu
>> >> >
>> >> > To subscribe or unsubscribe via the World Wide Web, visit
>> >> > http://mailman.mit.edu/mailman/listinfo/moses-support
>> >> > or, via email, send a message with subject or body 'help' to
>> >> > moses-support-request@mit.edu
>> >> >
>> >> > You can reach the person managing the list at
>> >> > moses-support-owner@mit.edu
>> >> >
>> >> > When replying, please edit your Subject line so it is more specific
>> >> > than "Re: Contents of Moses-support digest..."
>> >> >
>> >> >
>> >> > Today's Topics:
>> >> >
>> >> > 1. Re: 12-gram language model ARPA file for 16GB (liling tan)
>> >> > 2. Transliteration model is using processPhraseTable, which is
>> >> > not found in Moses version 3.0 (Ergun Bicici)
>> >> > 3. Re: Transliteration model is using processPhraseTable, which
>> >> > is not found in Moses version 3.0 (Hieu Hoang)
>> >> > 4. Europarl monolingual corpus (Hieu Hoang)
>> >> >
>> >> >
>> >> >
>> >> >
----------------------------------------------------------------------
>> >> >
>> >> > Message: 1
>> >> > Date: Sun, 3 May 2015 19:44:12 +0200
>> >> > From: liling tan <alvations@gmail.com>
>> >> > Subject: Re: [Moses-support] 12-gram language model ARPA file for
>> >> > 16GB
>> >> > To: moses-support <moses-support@mit.edu>
>> >> > Message-ID:
>> >> >
>> >> > <CAKzPaJJ7fY=9C89POact542vu32d+H3=0i_Dnaj=YfizbFA+cQ@mail.gmail.com>
>> >> > Content-Type: text/plain; charset="utf-8"
>> >> >
>> >> > Dear Moses devs/users,
>> >> >
>> >> > For now, I only know that it takes more than 250GB. I've 250GB of
>> >> > free
>> >> > space and KenLM got "poisoned" by insufficient space...
>> >> >
>> >> > Does anyone have an idea how big would a 12-gram language model ARPA
>> >> > file
>> >> > trained on 16GB of text become?
>> >> >
>> >> > STDERR:
>> >> >
>> >> > === 1/5 Counting and sorting n-grams ===
>> >> > Reading /media/2tb/wmt15/corpus.truecase/train-lm.en
>> >> >
>> >> >
>> >> >
----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100
>> >> > tcmalloc: large alloc 7846035456 bytes == 0x10f4000 @
>> >> > tcmalloc: large alloc 73229664256 bytes == 0x1d542e000 @
>> >> >
>> >> >
>> >> >
****************************************************************************************************
>> >> > Unigram tokens 3038737446 types 5924314
>> >> > === 2/5 Calculating and sorting adjusted counts ===
>> >> > Chain sizes: 1:71091768 2:804524736 3:1508483968 4:2413574144
>> >> > 5:3519795968
>> >> > 6:4827148288 7:6335632384 8:8045247488 9:9955993600 10:12067871744
>> >> > 11:14380880896 12:16895020032
>> >> > tcmalloc: large alloc 16895025152 bytes == 0x1d542e000 @
>> >> > tcmalloc: large alloc 2413576192 bytes == 0x8f2a0000 @
>> >> > tcmalloc: large alloc 3519799296 bytes == 0x5c4488000 @
>> >> > tcmalloc: large alloc 4827152384 bytes == 0x696146000 @
>> >> > tcmalloc: large alloc 6335635456 bytes == 0x7b5cce000 @
>> >> > tcmalloc: large alloc 8045248512 bytes == 0x92f6f0000 @
>> >> > tcmalloc: large alloc 9955999744 bytes == 0xb0ef7c000 @
>> >> > tcmalloc: large alloc 12067872768 bytes == 0xd60644000 @
>> >> > tcmalloc: large alloc 14380883968 bytes == 0x12f616e000 @
>> >> > Last input should have been poison.
>> >> > Last input should have been poison.util/file.cc:196 in void
>> >> > util::WriteOrThrow(int, const void*, std::size_t) threw FDException
>> >> > because
>> >> > `ret < 1'.
>> >> > No space left on device in /tmp/PC2o3z (deleted) while writing
>> >> > 5301120368
>> >> > bytes
>> >> >
>> >> > Last input should have been poison.util/file.cc:196 in void
>> >> > util::WriteOrThrow(int, const void*, std::size_t) threw FDException
>> >> > because
>> >> > `ret < 1'.
>> >> > No space left on device in /tmp/PftXeo (deleted) while writing
>> >> > 1941075872
>> >> > bytesLast input should have been poison.
>> >> >
>> >> > util/file.cc:196 in void util::WriteOrThrow(int, const void*,
>> >> > std::size_t)
>> >> > threw FDException because `ret < 1'.
>> >> > No space left on device in /tmp/CuZcPM (deleted) while writing
>> >> > 2984722272
>> >> > bytes
>> >> >
>> >> > util/file.cc:196 in void util::WriteOrThrow(int, const void*,
>> >> > std::size_t)
>> >> > threw FDException because `ret < 1'.
>> >> > No space left on device in /tmp/F2bE8A (deleted) while writing
>> >> > 389439488
>> >> > bytes
>> >> >
>> >> > Regards,
>> >> > Liling
>> >> > -------------- next part --------------
>> >> > An HTML attachment was scrubbed...
>> >> > URL:
>> >> >
>> >> >
http://mailman.mit.edu/mailman/private/moses-support/attachments/20150503/b56dc8ba/attachment-0001.htm
>> >> >
>> >> > ------------------------------
>> >> >
>> >> > Message: 2
>> >> > Date: Sun, 3 May 2015 22:42:22 +0100
>> >> > From: Ergun Bicici <Ergun.Bicici@computing.dcu.ie>
>> >> > Subject: [Moses-support] Transliteration model is using
>> >> > processPhraseTable, which is not found in Moses version 3.0
>> >> > To: moses-support <moses-support@mit.edu>
>> >> > Message-ID:
>> >> >
>> >> > <CAB2pGncpvc4roLXwLcFcXytZHKEqSZvzaX2L16Yfo=P-vq1jBA@mail.gmail.com>
>> >> > Content-Type: text/plain; charset="utf-8"
>> >> >
>> >> > binarizing...gzip -cd
>> >> >
>> >> >
>> >> >
en-ru_path/model/Transliteration.8/tuning/filtered/phrase-table.0-0.1.1.gz
>> >> > | LC_ALL=C sort -T
en-ru_path/model/Transliteration.8/tuning/filtered
>> >> > |
>> >> > moses_3.0/mosesdecoder/bin/processPhraseTable -ttable 0 0 - -nscores
>> >> > 4
>> >> > -out
>> >> >
>> >> >
en-ru_path/model/Transliteration.8/tuning/filtered/phrase-table.0-0.1.1
>> >> > sh: moses_3.0/mosesdecoder/bin/processPhraseTable: No such file or
>> >> > directory
>> >> > sort: write failed: standard output: Broken pipe
>> >> > sort: write error
>> >> >
>> >> > How can I have processPhraseTable built?
>> >> >
>> >> > Best Regards,
>> >> > Ergun
>> >> >
>> >> > Ergun Bi?ici, CNGL, School of Computing, DCU, www.cngl.ie
>> >> > http://www.computing.dcu.ie/~ebicici/
>> >> > -------------- next part --------------
>> >> > An HTML attachment was scrubbed...
>> >> > URL:
>> >> >
>> >> >
http://mailman.mit.edu/mailman/private/moses-support/attachments/20150503/dacaa1c9/attachment-0001.htm
>> >> >
>> >> > ------------------------------
>> >> >
>> >> > Message: 3
>> >> > Date: Mon, 04 May 2015 08:31:18 +0400
>> >> > From: Hieu Hoang <hieuhoang@gmail.com>
>> >> > Subject: Re: [Moses-support] Transliteration model is using
>> >> > processPhraseTable, which is not found in Moses version 3.0
>> >> > To: Ergun Bicici <Ergun.Bicici@computing.dcu.ie>,
moses-support
>> >> > <moses-support@mit.edu>
>> >> > Message-ID: <5546F616.4000007@gmail.com>
>> >> > Content-Type: text/plain; charset="windows-1252"
>> >> >
>> >> > do you know where the processPhraseTable exec is being called from?
>> >> >
>> >> > it would be helpful so we can make sure it uses something else.
>> >> >
>> >> > if you really want processPhraseTable back, uncomment 3 lines in
>> >> > misc/Jamfile
>> >> >
>> >> > +++ b/misc/Jamfile
>> >> > @@ -1,8 +1,8 @@
>> >> > -#exe processPhraseTable : GenerateTuples.cpp processPhraseTable.cpp
>> >> > ..//boost_filesystem ../moses//moses ;
>> >> > +exe processPhraseTable : GenerateTuples.cpp processPhraseTable.cpp
>> >> > ..//boost_filesystem ../moses//moses ;
>> >> >
>> >> > exe processLexicalTable : processLexicalTable.cpp
>> >> > ..//boost_filesystem
>> >> > ../moses//moses ;
>> >> >
>> >> > -#exe queryPhraseTable : queryPhraseTable.cpp ..//boost_filesystem
>> >> > ../moses//moses ;
>> >> > +exe queryPhraseTable : queryPhraseTable.cpp ..//boost_filesystem
>> >> > ../moses//moses ;
>> >> >
>> >> > exe queryLexicalTable : queryLexicalTable.cpp ..//boost_filesystem
>> >> > ../moses//moses ;
>> >> >
>> >> > @@ -46,6 +46,6 @@ $(TOP)//boost_iostreams
>> >> > $(TOP)//boost_program_options
>> >> > ;
>> >> >
>> >> > -alias programs : 1-1-Extraction TMining generateSequences
>> >> > processLexicalTable queryLexicalTable programsMin programsProbing
>> >> > merge-sorted prunePhraseTable ;
>> >> > -#processPhraseTable queryPhraseTable
>> >> > +alias programs : 1-1-Extraction TMining generateSequences
>> >> > processLexicalTable queryLexicalTable programsMin programsProbing
>> >> > merge-sorted prunePhraseTable processPhraseTable queryPhraseTable ;
>> >> >
>> >> > On 04/05/2015 01:42, Ergun Bicici wrote:
>> >> >>
>> >> >> binarizing...gzip -cd
>> >> >>
>> >> >>
>> >> >>
en-ru_path/model/Transliteration.8/tuning/filtered/phrase-table.0-0.1.1.gz
>> >> >> | LC_ALL=C sort -T
>> >> >> en-ru_path/model/Transliteration.8/tuning/filtered
>> >> >> | moses_3.0/mosesdecoder/bin/processPhraseTable -ttable 0 0 -
>> >> >> -nscores
>> >> >> 4 -out
>> >> >>
>> >> >>
en-ru_path/model/Transliteration.8/tuning/filtered/phrase-table.0-0.1.1
>> >> >> sh: moses_3.0/mosesdecoder/bin/processPhraseTable: No such file or
>> >> >> directory
>> >> >> sort: write failed: standard output: Broken pipe
>> >> >> sort: write error
>> >> >>
>> >> >> How can I have processPhraseTable built?
>> >> >>
>> >> >> Best Regards,
>> >> >> Ergun
>> >> >>
>> >> >> Ergun Bi?ici, CNGL, School of Computing, DCU, www.cngl.ie
>> >> >> <http://www.cngl.ie>
>> >> >> http://www.computing.dcu.ie/~ebicici/
>> >> >> <http://www.computing.dcu.ie/%7Eebicici/>
>> >> >>
>> >> >>
>> >> >>
>> >> >> _______________________________________________
>> >> >> Moses-support mailing list
>> >> >> Moses-support@mit.edu
>> >> >> http://mailman.mit.edu/mailman/listinfo/moses-support
>> >> >
>> >> > --
>> >> > Hieu Hoang
>> >> > Researcher
>> >> > New York University, Abu Dhabi
>> >> > http://www.hoang.co.uk/hieu
>> >> >
>> >> > -------------- next part --------------
>> >> > An HTML attachment was scrubbed...
>> >> > URL:
>> >> >
>> >> >
http://mailman.mit.edu/mailman/private/moses-support/attachments/20150504/303023d0/attachment-0001.htm
>> >> >
>> >> > ------------------------------
>> >> >
>> >> > Message: 4
>> >> > Date: Mon, 4 May 2015 08:46:15 +0400
>> >> > From: Hieu Hoang <hieuhoang@gmail.com>
>> >> > Subject: [Moses-support] Europarl monolingual corpus
>> >> > To: moses-support <moses-support@mit.edu>
>> >> > Message-ID:
>> >> >
>> >> > <CAEKMkbiO64F_m20RwNXyDOj60FHEZ_oo+BY+hzkW3TBFukPfAQ@mail.gmail.com>
>> >> > Content-Type: text/plain; charset="utf-8"
>> >> >
>> >> > What's the easiest way get the single-language data from the
Europarl
>> >> > corpus as described in the 1st table in:
>> >> > http://statmt.org/europarl/
>> >> >
>> >> > I tried downloading the xml source
>> >> > http://statmt.org/europarl/v7/europarl.tgz
>> >> > stripping the xml and running split-sentence.perl, but this takes an
>> >> > unfathomably long time
>> >> >
>> >> > Hieu Hoang
>> >> > Researcher
>> >> > New York University, Abu Dhabi
>> >> > http://www.hoang.co.uk/hieu
>> >> > -------------- next part --------------
>> >> > An HTML attachment was scrubbed...
>> >> > URL:
>> >> >
>> >> >
http://mailman.mit.edu/mailman/private/moses-support/attachments/20150504/ba5b4087/attachment.htm
>> >> >
>> >> > ------------------------------
>> >> >
>> >> > _______________________________________________
>> >> > Moses-support mailing list
>> >> > Moses-support@mit.edu
>> >> > http://mailman.mit.edu/mailman/listinfo/moses-support
>> >> >
>> >> >
>> >> > End of Moses-support Digest, Vol 103, Issue 5
>> >> > *********************************************
>> >
>> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150506/db3dfbeb/attachment.htm

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 103, Issue 19
**********************************************

0 Response to "Moses-support Digest, Vol 103, Issue 19"

Post a Comment

Subscribe to: Post Comments (Atom)