Moses-support Digest, Vol 103, Issue 3

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."

Today's Topics:

1. Re: Duplicated source files (Kenneth Heafield)
2. Re: Factored Model decoder fail !! (Marwa Refaie)

----------------------------------------------------------------------

Message: 1
Date: Fri, 01 May 2015 15:58:08 -0400
From: Kenneth Heafield <moses@kheafield.com>
Subject: Re: [Moses-support] Duplicated source files
To: moses-support@mit.edu
Message-ID: <5543DAD0.2090404@kheafield.com>
Content-Type: text/plain; charset=windows-1252

If this comment accurate that gzfilebuf is only used for writing?

/** wrapper around gzip input stream. Unknown parentage
* @todo replace with boost version - output stream already uses it
*/

If so I'll just extend util/fake_ofstream.hh to have gzip support.

Time to print a bunch of integers:

FakeOFStream:

real 0m3.460s
user 0m3.459s
sys 0m0.004s

std::cout

real 0m23.010s
user 0m22.895s
sys 0m0.134s

Time to print a bunch of floats:

FakeOFStream:

real 0m34.871s
user 0m34.894s
sys 0m0.006s

std::cout

real 1m56.628s
user 1m56.690s
sys 0m0.037s

The conversion is done by https://github.com/miloyip/itoa-benchmark/ and
Google double conversion.

Kenneth

On 05/01/15 14:37, Barry Haddow wrote:
> What about the util directory?
>
> On 1 May 2015 19:13:26 BST, Hieu Hoang <hieuhoang@gmail.com> wrote:
>
> i suppose everything should reference the moses lib.
>
> that's getting a bit bloated, one day we should look at splitting it up
>
> On 30/04/2015 10:24, Jeroen Vermeulen wrote:
>
> Any chance we could re-unify the gzfilebuf and InputFileStream
> modules?
> Looks like we're carrying around 4 copies of each, and they're
> starting
> to diverge.
>
> I'd be happy to make the change, if we know a good reusable
> place to put it.
>
>
> Jeroen
> ------------------------------------------------------------------------
>
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
>
> --
> Sent from my Android device with K-9 Mail. Please excuse my brevity.
>
>
> The University of Edinburgh is a charitable body, registered in
> Scotland, with registration number SC005336.
>
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>

------------------------------

Message: 2
Date: Sat, 2 May 2015 11:50:25 +0000
From: Marwa Refaie <basmallah@hotmail.com>
Subject: Re: [Moses-support] Factored Model decoder fail !!
To: Hieu Hoang <hieuhoang@gmail.com>, "moses-support@mit.edu"
<moses-support@mit.edu>
Message-ID: <DUB118-W29BFA0D1F3A83318867B43BAD40@phx.gbl>
Content-Type: text/plain; charset="windows-1256"

Thanks
But now the output is untranslated !!!!!!!!!!!!! The decoder give me the same English input no Arabic translation ....
Any help please.... what's wrong Can't figure it out
Text to be translatedArtificial|JJ intelligence|NN Artificial|JJ intelligence|NN -LRB-|-LRB- AI|NN -RRB-|-RRB- is|VBZ the|DT intelligence|NN of|IN machines|NNS and|CC the|DT branch|NN of|IN computer|NN science|NN that|WDT aims|VBZ to|TO create|VB it|PRP .|. The Moses output Artificial intelligence Artificial intelligence -LRB- AI -RRB- is the intelligence of machines and the branch of computer science that aims to create it .
Marwa N. Refaie

Date: Fri, 1 May 2015 22:59:21 +0400
From: hieuhoang@gmail.com
To: basmallah@hotmail.com; moses-support@mit.edu
Subject: Re: [Moses-support] Factored Model decoder fail !!

in the moses.ini file, change

PhraseDictionaryOnDisk ... input-factor=0

to

PhraseDictionaryOnDisk ...
input-factor=0,1

On 01/05/2015 13:33, Marwa Refaie
wrote:

I try to build a factored model with word|pos. After
fixing all erroros, I got the "segmentation fault" error, so I
compress the phrase table by

~/mosesdecoder/bin/CreateOnDiskPt
2 2 4 100 2 ~/April/small/-dir/model/phrase-table.0-0,1.gz
phrase-table.folder

Now I
have this error !!! I change the ini file but still stuck
here ... Please any help ??

On-disk
phrase table has 2 source factors.. The ini file
specified 1 source factors .... Aborted (core dumped)
or
Segmentation Fault (core dumped)
or

No
phrase dictionary with index 1 available!

as I change ini Input factor or mapping steps.

English / Arabic corpus

Its|VBD complement|NN the|NN error|NN rate,|NN
is|NN the|NN proportion|NN of|NN incorrect|NN
predictions|NN on|NN the|NN same|NN data.|NN
See|VBD Also:|NN hold|NN out|NN sample,|NN
Machine|DTNN Learning.|DTJJ

???????|NN ?????|NN
????????|DTNN ????????|VBD ???|NN ????|NN ?????|JJ
??????|VBP ?????|NN ?????|VBP ???|IN ????????|DTNN
.|PUNC
????|VB ????|NN
:|PUNC ?????|NN ?????|VBP ???|IN ????????.|DTNN

Phrase Table 0-0,1

] proposed the ||| ]|NN
?????|VBD ||| 1 0.102035 1 0.124013 ||| 0-0 1-1 2-1 ||| 1
1 1 ||| |||
] suggests analysis of ||| ]|NN
????|VBP ?????|NN ||| 1 0.00298151 1 0.00967381 ||| 0-0
1-1 2-2 3-2 ||| 1 1 1 ||| |||

Moses.ini

# input factors
[input-factors]
0
1

# mapping
steps
[mapping]
0 T 0

[distortion-limit]
6

# feature
functions
[feature]
UnknownWordPenalty
WordPenalty
PhrasePenalty
#PhraseDictionaryMemory
name=TranslationModel0 num-features=4
path=/home/marwa/Phd/April/bilmpos/small/-dir/model/phrase-table.0-0,1.gz
input-factor=0 output-factor=0,1

PhraseDictionaryOnDisk
name=TranslationModel0 num-features=4
path=/home/marwa/Phd/April/bilmpos/small/-dir/model/phrase-table.folder
input-factor=0 output-factor=0,1

Distortion
KENLM
name=LM0 factor=2
path=/home/marwa/Phd/April/bilmpos/small/pos.binary
order=5
KENLM
name=LM1 factor=0
path=/home/marwa/Phd/April/bilmpos/small/surface.binary
order=5

Marwa N. Refaie

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

--
Hieu Hoang
Researcher
New York University, Abu Dhabi
http://www.hoang.co.uk/hieu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150502/3b500729/attachment.htm

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

End of Moses-support Digest, Vol 103, Issue 3
*********************************************

Moses-support Digest, Vol 103, Issue 3

0 Response to "Moses-support Digest, Vol 103, Issue 3"