Moses-support Digest, Vol 106, Issue 31

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: Thread-safe Lattice Decoding (Hieu Hoang)
2. OSM in EMS error (Vincent Nguyen)
3. Re: OSM in EMS error (Kenneth Heafield)
4. Re: OSM in EMS error (Vincent Nguyen)
5. Re: OSM in EMS error (Hieu Hoang)


----------------------------------------------------------------------

Message: 1
Date: Sun, 16 Aug 2015 01:58:45 +0400
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] Thread-safe Lattice Decoding
To: "James H. Cross III" <james.henry.cross.iii@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAEKMkbhf5TZLDr56=+hempKF7UHQJmTo=igMBui5eWyLm8Y9Vw@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

it's now thread-safe as far as I know. There were issues with using the old
binary phrase table.

You should be fine if you use the compact phrase-table, the on-disk
phrase-table, or the memory phrase.

Hieu Hoang
Researcher
New York University, Abu Dhabi
http://www.hoang.co.uk/hieu

On 14 August 2015 at 21:16, James H. Cross III <
james.henry.cross.iii@gmail.com> wrote:

> Hi:
>
> The website documentation notes that lattice input may not work with
> multi-threaded decoding. Is there a reason to believe this is not
> likely to work? To the extent that each thread processes a single
> input example (lattice instead of sentence), it seems like the
> shared-resource issues would be no different than with sentence input.
>
> If it is indeed not supported, can you give me some idea of what might
> be necessary to extend it to this use case?
>
> Thanks!
> James
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150815/565c1441/attachment.html

------------------------------

Message: 2
Date: Sun, 16 Aug 2015 12:11:13 +0200
From: Vincent Nguyen <vnguyen@neuf.fr>
Subject: [Moses-support] OSM in EMS error
To: moses-support <moses-support@mit.edu>
Message-ID: <55D061C1.3080804@neuf.fr>
Content-Type: text/plain; charset="windows-1252"


the build-osm crashes in EMS with following error
any clue ?

23396000 23397000 23398000 23399000 23400000Converting Bilingual
Sentence Pair into Operation Corpus
Executing: /home/moses/mosesdecoder/bin/generateSequences
/home/moses/working/model/OSM.2//e /home/moses/working/model/OSM.2//f
/home/moses/working/model/OSM.2/align
/home/moses/working/model/OSM.2//Singletons >
/home/moses/working/model/OSM.2//opCorpus
Learning Operation Sequence Translation Model
Executing: /home/moses/mosesdecoder/bin/lmplz -S 40% -T
/home/moses/working/model/tmp -S 20% -T /home/moses/working/model/OSM.2
--order 5 --text /home/moses/working/model/OSM.2//opCorpus --arpa
/home/moses/working/model/OSM.2//operationLM --prune 0 0 1 2>> /dev/stderr
Binarizing
Executing: /home/moses/mosesdecoder/bin/build_binary
/home/moses/working/model/OSM.2//operationLM
/home/moses/working/model/OSM.2//operationLM.bin
util/file.cc:68 in int util::OpenReadOrThrow(const char*) threw
ErrnoException because `-1 == (ret = open(name, 00))'.
No such file or directory while opening
/home/moses/working/model/OSM.2//operationLM
ERROR

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150816/eb4b6823/attachment-0001.html

------------------------------

Message: 3
Date: Sun, 16 Aug 2015 11:23:39 +0100
From: Kenneth Heafield <moses@kheafield.com>
Subject: Re: [Moses-support] OSM in EMS error
To: moses-support@mit.edu
Message-ID: <55D064AB.7040908@kheafield.com>
Content-Type: text/plain; charset=windows-1252

If I had to guess, you ran out of disk space. Can you find the stderr
of lmplz?

Kenneth

On 08/16/2015 11:11 AM, Vincent Nguyen wrote:
>
> the build-osm crashes in EMS with following error
> any clue ?
>
> 23396000 23397000 23398000 23399000 23400000Converting Bilingual
> Sentence Pair into Operation Corpus
> Executing: /home/moses/mosesdecoder/bin/generateSequences
> /home/moses/working/model/OSM.2//e /home/moses/working/model/OSM.2//f
> /home/moses/working/model/OSM.2/align
> /home/moses/working/model/OSM.2//Singletons >
> /home/moses/working/model/OSM.2//opCorpus
> Learning Operation Sequence Translation Model
> Executing: /home/moses/mosesdecoder/bin/lmplz -S 40% -T
> /home/moses/working/model/tmp -S 20% -T /home/moses/working/model/OSM.2
> --order 5 --text /home/moses/working/model/OSM.2//opCorpus --arpa
> /home/moses/working/model/OSM.2//operationLM --prune 0 0 1 2>> /dev/stderr
> Binarizing
> Executing: /home/moses/mosesdecoder/bin/build_binary
> /home/moses/working/model/OSM.2//operationLM
> /home/moses/working/model/OSM.2//operationLM.bin
> util/file.cc:68 in int util::OpenReadOrThrow(const char*) threw
> ErrnoException because `-1 == (ret = open(name, 00))'.
> No such file or directory while opening
> /home/moses/working/model/OSM.2//operationLM
> ERROR
>
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>


------------------------------

Message: 4
Date: Sun, 16 Aug 2015 13:21:10 +0200
From: Vincent Nguyen <vnguyen@neuf.fr>
Subject: Re: [Moses-support] OSM in EMS error
To: moses-support@mit.edu
Message-ID: <55D07226.9050208@neuf.fr>
Content-Type: text/plain; charset=windows-1252; format=flowed

I doubt because I have plenty of space and I did not get the usual "out
of space" popup from Ubuntu.
what do you mean by the stderr of lmplz ? the output below is from the
training-build-osm stderr
which calls lmplz, right ?

Le 16/08/2015 12:23, Kenneth Heafield a ?crit :
> If I had to guess, you ran out of disk space. Can you find the stderr
> of lmplz?
>
> Kenneth
>
> On 08/16/2015 11:11 AM, Vincent Nguyen wrote:
>> the build-osm crashes in EMS with following error
>> any clue ?
>>
>> 23396000 23397000 23398000 23399000 23400000Converting Bilingual
>> Sentence Pair into Operation Corpus
>> Executing: /home/moses/mosesdecoder/bin/generateSequences
>> /home/moses/working/model/OSM.2//e /home/moses/working/model/OSM.2//f
>> /home/moses/working/model/OSM.2/align
>> /home/moses/working/model/OSM.2//Singletons >
>> /home/moses/working/model/OSM.2//opCorpus
>> Learning Operation Sequence Translation Model
>> Executing: /home/moses/mosesdecoder/bin/lmplz -S 40% -T
>> /home/moses/working/model/tmp -S 20% -T /home/moses/working/model/OSM.2
>> --order 5 --text /home/moses/working/model/OSM.2//opCorpus --arpa
>> /home/moses/working/model/OSM.2//operationLM --prune 0 0 1 2>> /dev/stderr
>> Binarizing
>> Executing: /home/moses/mosesdecoder/bin/build_binary
>> /home/moses/working/model/OSM.2//operationLM
>> /home/moses/working/model/OSM.2//operationLM.bin
>> util/file.cc:68 in int util::OpenReadOrThrow(const char*) threw
>> ErrnoException because `-1 == (ret = open(name, 00))'.
>> No such file or directory while opening
>> /home/moses/working/model/OSM.2//operationLM
>> ERROR
>>
>>
>>
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support


------------------------------

Message: 5
Date: Sun, 16 Aug 2015 19:50:44 +0400
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] OSM in EMS error
To: Vincent Nguyen <vnguyen@neuf.fr>, moses-support
<moses-support@mit.edu>
Message-ID: <55D0B154.1080900@gmail.com>
Content-Type: text/plain; charset="windows-1252"

there's a double declaration of -S when running lmplz. That's either a
mistake in the config file or in the script

On 16/08/2015 14:11, Vincent Nguyen wrote:
>
> the build-osm crashes in EMS with following error
> any clue ?
>
> 23396000 23397000 23398000 23399000 23400000Converting Bilingual
> Sentence Pair into Operation Corpus
> Executing: /home/moses/mosesdecoder/bin/generateSequences
> /home/moses/working/model/OSM.2//e /home/moses/working/model/OSM.2//f
> /home/moses/working/model/OSM.2/align
> /home/moses/working/model/OSM.2//Singletons >
> /home/moses/working/model/OSM.2//opCorpus
> Learning Operation Sequence Translation Model
> Executing: /home/moses/mosesdecoder/bin/lmplz -S 40% -T
> /home/moses/working/model/tmp -S 20% -T
> /home/moses/working/model/OSM.2 --order 5 --text
> /home/moses/working/model/OSM.2//opCorpus --arpa
> /home/moses/working/model/OSM.2//operationLM --prune 0 0 1 2>> /dev/stderr
> Binarizing
> Executing: /home/moses/mosesdecoder/bin/build_binary
> /home/moses/working/model/OSM.2//operationLM
> /home/moses/working/model/OSM.2//operationLM.bin
> util/file.cc:68 in int util::OpenReadOrThrow(const char*) threw
> ErrnoException because `-1 == (ret = open(name, 00))'.
> No such file or directory while opening
> /home/moses/working/model/OSM.2//operationLM
> ERROR
>
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support

--
Hieu Hoang
Researcher
New York University, Abu Dhabi
http://www.hoang.co.uk/hieu

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150816/9e55f261/attachment.html

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 106, Issue 31
**********************************************

Tweet