Send Moses-support mailing list submissions to
moses-support@mit.edu
To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu
You can reach the person managing the list at
moses-support-owner@mit.edu
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."
Today's Topics:
1. Re: how to cross compile moses binary on Raspberry Pi 2 model
B (Hieu Hoang)
2. Re: how to cross compile moses binary on Raspberry Pi 2 model
B (Apurva Joshi)
3. Re: ? '? in the translated text (Philipp Koehn)
4. Re: different versions of moses yielding different
translations (Vito Mandorino)
----------------------------------------------------------------------
Message: 1
Date: Wed, 25 Nov 2015 21:23:57 +0000
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] how to cross compile moses binary on
Raspberry Pi 2 model B
To: Apurva Joshi <apurvajoshi1992@gmail.com>, moses-support@mit.edu
Message-ID: <565626ED.7070308@gmail.com>
Content-Type: text/plain; charset="windows-1252"
i also have a raspberry pi. If you find out how, i would love to know!
On 25/11/2015 08:27, Apurva Joshi wrote:
> hello all,
>
> i run moses on pc which does not contain moses successfully , by copying
>
> moses.ini
> phrase table
> reordering table
> LM
> moses( binary from ~/mosesdecoder/bin/moses)
>
> i copied above files to pc which does not contain moses
>
> and then on that pc i ran
>
> moses -f moses.ini
>
> its worked...
>
> NOW
>
> same procedure i used for running on raspberry pi..
>
> on raspberry pi board , i copied all above 5 files
>
> and ran moses -f moses.ini
>
> its showing errror that "*cant run binary"
>
> -bash: /home/ubuntu/Apurva/moses: cannot execute binary file: Exec
> format error
>
>
> *
> so i need to cross-compile moses (binary) for ARM on raspberry pi board
>
> i dont know how to do cross compilation of binary for ARM
>
> plz help and reply on same id
>
> apurvajoshi1992@gmail.com <mailto:apurvajoshi1992@gmail.com>
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
--
Hieu Hoang
http://www.hoang.co.uk/hieu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20151125/836f4c55/attachment-0001.html
------------------------------
Message: 2
Date: Thu, 26 Nov 2015 10:42:55 +0530
From: Apurva Joshi <apurvajoshi1992@gmail.com>
Subject: Re: [Moses-support] how to cross compile moses binary on
Raspberry Pi 2 model B
To: Hieu Hoang <hieuhoang@gmail.com>, moses-support@mit.edu
Message-ID:
<CAMfCXbV2gzvPF8+HANrtDavDzUn+neS5r+v5Cd=+Ac_wbjWEXw@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
hi,
i want to run moses on raspberry pi which have arm processor,
normally on pc , when we install moses, it first compiled using GCC
compiler as per config file....
but to install moses on raspberry pi ,moses should get compile with arm "
*using gcc : arm : arm-linux-gnueabihf-g++ ;*
so what changes should we need to make in configuration file of moses, so
that it will get compiled with *arm-linux-gnueabihf-g++ *instead of "gcc"...
so plz tell me how to do this cross compilation ....??
my logic -
*on PC - moses installed in following way -*
./bjam --with boost=path_of_boost_1_59_0 -j5
moses uses above ./bjam for compilation ...this will compile moses..during
this ,moses uses its config file which default uses gcc
*for arm -*
i cross compiled this "boost" using following site :
*http://stackoverflow.com/questions/29029867/cross-compile-boost-1-57-0-on-ubuntu-for-arm
<http://stackoverflow.com/questions/29029867/cross-compile-boost-1-57-0-on-ubuntu-for-arm>*
now this boost lib will be compatible to arm ...
./bjam -- with-boot=path_of_cross_compiled_boost -j5
using above , moses should start compiling using arm g++ , but it still
starts to compile using gcc ...not arm-g++
so i think we need to make some changes in configuration file of moses...
so plz tell me, do u have idea about how to run moses on raspberry pi with
arm architecture ..??? plz help
On Thu, Nov 26, 2015 at 2:53 AM, Hieu Hoang <hieuhoang@gmail.com> wrote:
> i also have a raspberry pi. If you find out how, i would love to know!
>
>
> On 25/11/2015 08:27, Apurva Joshi wrote:
>
> hello all,
>
> i run moses on pc which does not contain moses successfully , by copying
>
> moses.ini
> phrase table
> reordering table
> LM
> moses( binary from ~/mosesdecoder/bin/moses)
>
> i copied above files to pc which does not contain moses
>
> and then on that pc i ran
>
> moses -f moses.ini
>
> its worked...
>
> NOW
>
> same procedure i used for running on raspberry pi..
>
> on raspberry pi board , i copied all above 5 files
>
> and ran moses -f moses.ini
>
> its showing errror that "
>
>
>
>
> *cant run binary" -bash: /home/ubuntu/Apurva/moses: cannot execute binary
> file: Exec format error *
> so i need to cross-compile moses (binary) for ARM on raspberry pi board
>
> i dont know how to do cross compilation of binary for ARM
>
> plz help and reply on same id
>
> apurvajoshi1992@gmail.com
>
>
> _______________________________________________
> Moses-support mailing listMoses-support@mit.eduhttp://mailman.mit.edu/mailman/listinfo/moses-support
>
>
> --
> Hieu Hoanghttp://www.hoang.co.uk/hieu
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20151126/4fded3e6/attachment-0001.html
------------------------------
Message: 3
Date: Thu, 26 Nov 2015 02:25:56 -0500
From: Philipp Koehn <phi@jhu.edu>
Subject: Re: [Moses-support] ? '? in the translated text
To: ??? <fountain1128@gmail.com>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<CAAFADDBzHObDE8YL+nrz+Ym2Y6LjYDjF2A6cP7qPGhff3x1opw@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Hi,
yes, this is normal.
The script tokenizer.perl creates these tokens, you have to run
detokenizer.perl on the output to revert to normal text.
-phi
On Tue, Nov 24, 2015 at 4:18 PM, ??? <fountain1128@gmail.com> wrote:
> Hi,
>
> There are ? '? in the translated text. I noticed that escaping is
> necessary for the tokenized file during corpus preparation.
> But when the translation is done, how can I change all the ? '? back
> to the normal apostrophe?
> Here attaches my translated file.
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20151126/56367e10/attachment-0001.html
------------------------------
Message: 4
Date: Thu, 26 Nov 2015 12:16:59 +0100
From: Vito Mandorino <vito.mandorino@linguacustodia.com>
Subject: Re: [Moses-support] different versions of moses yielding
different translations
To: Barry Haddow <bhaddow@inf.ed.ac.uk>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CA+8mSmHLe2pZ390gBK-TE2pZJPpRx=yUNP0MrjwXqBEDgULsoQ@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Hi Barry,
actually with OnDisk table there is virtually no difference (0.2 average
difference no matter if re-tuning has been done or not).
With compact Phrase-table however the difference is larger. The latest test
this morning yields a loss of 14 Bleu score points without re-tuning. I
don't know which could be the cause.
Sometimes there is this message on loading the phrase-tables
tcmalloc: large alloc 1149427712 bytes == 0x28a54000 @
After re-tuning however the difference in BLEU score gets smaller even with
compact phrase-table.
Best regards,
Vito
2015-11-25 21:23 GMT+01:00 Barry Haddow <bhaddow@inf.ed.ac.uk>:
> Hi Vito
>
> The 0.2 difference is after retuning? That's normal then.
>
> But a difference of 5 bleu without retuning suggests a bug. Did you say
> that this only happens with PhraseDictionaryMultiModel?
>
> cheers - Barry
>
>
> On 25/11/15 13:53, Vito Mandorino wrote:
>
> Thank you. In our tests it seems that with the OnDisk table the quality is
> basically the same between the two versions of Moses (average 0.2
> difference in score Bleu) but for the CompactPhraseTable the difference is
> larger (2 points Bleu loss in average after re-tuning with the new version
> of Moses, and more than 5 points Bleu in average without re-tuning).
> Do you think a better quality would be obtained by running a complete
> re-training of the model with the new version of Moses?
>
>
> Best regards,
> Vito
>
> 2015-11-24 16:31 GMT+01:00 Hieu Hoang <hieuhoang@gmail.com>:
>
>> There was a change in the underlying datastructure for stacks, it changed
>> from std::set (ordered) to boost::unordered_set.
>>
>> https://github.com/moses-smt/mosesdecoder/commit/6b182ee5e987a5b2823aea7eaaa7ef0457c6a30d
>> This got some speed gains
>>
>> 1 5 10 15 20 25 30 35
>> 56 real 4m57.795s real 1m19.005s real 0m51.636s real
>> 0m49.624s real 0m49.869s real 0m52.475s real 0m53.806s real
>> 0m54.957s 13/10 baseline user 4m41.255s user 5m45.086s user
>> 6m34.053s user 8m12.430s user 8m10.667s user 8m16.486s user
>> 8m10.592s user 8m13.859s
>> sys 0m16.514s sys 0m35.494s sys 0m54.513s sys 1m10.643s
>> sys 1m18.449s sys 1m21.738s sys 1m23.133s sys 1m25.048s
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> 57 real 4m41.148s real 1m16.002s real 0m50.747s real
>> 0m48.711s real 0m49.130s real 0m51.473s real 0m53.141s real
>> 0m54.513s (56) + unordered set stack user 4m23.968s user 5m30.356s
>> user 6m26.167s user 7m39.286s user 7m56.229s user 7m52.669s
>> user 7m56.978s user 7m56.216s
>> sys 0m17.231s sys 0m35.063s sys 0m54.081s sys 1m10.137s
>> sys 1m17.194s sys 1m22.912s sys 1m25.948s sys 1m26.247s
>> However, the hypotheses are now added to the stack in a different order
>> so there will be slight differences in results
>>
>>
>> On 24/11/2015 13:53, Vito Mandorino wrote:
>>
>> Hi,
>>
>> in some of our tests a recent version of Moses (pulled from github last
>> week) and an older one do not give the same translations on the same source
>> segment (with the same moses.ini).
>> Here is the 5-best list for the translation of 'test' with the last week
>> version:
>>
>> 0 ||| test ||| LexicalReordering0= -1.1969 0 0 0 0 0 Distortion0= 0 LM0=
>> -51.1788 WordPenalty0= -1 PhrasePenalty0= 1 PhraseDictionaryMultiModel0=
>> -3.03811 -2.5834 -2.08503 -1.83075 ||| -1.27754
>> 0 ||| testing ||| LexicalReordering0= 0 0 0 0 0 0 Distortion0= 0 LM0=
>> -35.1495 WordPenalty0= -1 PhrasePenalty0= 1 PhraseDictionaryMultiModel0=
>> -5.21045 -5.04877 -4.71131 -4.66382 ||| -1.70337
>> 0 ||| funds ||| LexicalReordering0= -3.1355 0 0 0 0 0 Distortion0= 0
>> LM0= -11.3753 WordPenalty0= -1 PhrasePenalty0= 1
>> PhraseDictionaryMultiModel0= -10.8209 -10.6835 -5.14555 -5.73388 |||
>> -1.77009
>> 0 ||| known as a ||| LexicalReordering0= -3.1355 0 0 0 0 0 Distortion0=
>> 0 LM0= -58.8877 WordPenalty0= -3 PhrasePenalty0= 1
>> PhraseDictionaryMultiModel0= -4.42285 -11.9339 -5.14555 -18.0392 |||
>> -1.89152
>> 0 ||| as a ||| LexicalReordering0= -3.1355 0 0 0 0 0 Distortion0= 0 LM0=
>> -35.5353 WordPenalty0= -2 PhrasePenalty0= 1 PhraseDictionaryMultiModel0=
>> -9.34698 -11.9339 -5.14555 -9.14874 ||| -1.89159
>>
>> and with the older version of Moses:
>>
>> 0 ||| funds ||| LexicalReordering0= -3.1355 0 0 0 0 0 Distortion0= 0
>> LM0= -11.3753 WordPenalty0= -1 PhrasePenalty0= 1
>> PhraseDictionaryMultiModel0= -2.52548 -2.52544 -2.45544 -2.48609 |||
>> -0.815668
>> 0 ||| as a ||| LexicalReordering0= -3.1355 0 0 0 0 0 Distortion0= 0 LM0=
>> -35.5353 WordPenalty0= -2 PhrasePenalty0= 1 PhraseDictionaryMultiModel0=
>> -2.52464 -2.52565 -2.45544 -2.5244 ||| -0.953799
>> 0 ||| as ||| LexicalReordering0= -3.1355 0 0 0 0 0 Distortion0= 0 LM0=
>> -34.1633 WordPenalty0= -1 PhrasePenalty0= 1 PhraseDictionaryMultiModel0=
>> -2.5256 -2.52565 -2.45544 -2.48609 ||| -1.07254
>> 0 ||| known as a ||| LexicalReordering0= -3.1355 0 0 0 0 0 Distortion0=
>> 0 LM0= -58.8877 WordPenalty0= -3 PhrasePenalty0= 1
>> PhraseDictionaryMultiModel0= -2.38597 -2.52565 -2.45544 -2.52573 |||
>> -1.07536
>> 0 ||| is known as a ||| LexicalReordering0= -3.1355 0 0 0 0 0
>> Distortion0= 0 LM0= -80.8518 WordPenalty0= -4 PhrasePenalty0= 1
>> PhraseDictionaryMultiModel0= -2.37158 -2.52565 -2.45544 -2.52573 |||
>> -1.18753
>>
>> This looks very strange. The only difference is in the phrase-table
>> scores. Do you have any idea of what is going on? The only possibility
>> which come to mind is maybe a different handling of the
>> PhraseDictionaryMultiModel feature.
>> The moses.ini is in attachment.
>>
>> Best regards,
>>
>> Vito
>>
>>
>>
>> --
>> *M**. Vito MANDORINO -- Chief Scientist*
>>
>>
>> [image: Description : Description : lingua_custodia_final full logo]
>>
>> *The Translation Trustee*
>>
>> *1, Place Charles de Gaulle, **78180 Montigny-le-Bretonneux*
>>
>> *Tel : +33 1 30 44 04 23 Mobile : +33 6 84 65 68 89
>> <%2B33%206%2084%2065%2068%2089>*
>>
>> *Email :* * <massinissa.ahmim@linguacustodia.com>
>> <vito.mandorino@linguacustodia.com>vito.mandorino@linguacustodia.com
>> <vito.mandorino@linguacustodia.com>*
>>
>> *Website :* * <http://www.linguacustodia.com/>www.linguacustodia.com
>> <http://www.linguacustodia.com> - www.thetranslationtrustee.com
>> <http://www.thetranslationtrustee.com/>*
>>
>>
>> _______________________________________________
>> Moses-support mailing listMoses-support@mit.eduhttp://mailman.mit.edu/mailman/listinfo/moses-support
>>
>>
>> --
>> Hieu Hoanghttp://www.hoang.co.uk/hieu
>>
>>
>
>
> --
> *M**. Vito MANDORINO -- Chief Scientist*
>
>
> [image: Description : Description : lingua_custodia_final full logo]
>
> *The Translation Trustee*
>
> *1, Place Charles de Gaulle, **78180 Montigny-le-Bretonneux*
>
> *Tel : +33 1 30 44 04 23 Mobile : +33 6 84 65 68 89
> <%2B33%206%2084%2065%2068%2089>*
>
> *Email :* *
> <massinissa.ahmim@linguacustodia.com>vito.mandorino@linguacustodia.com
> <vito.mandorino@linguacustodia.com>*
>
> *Website :* *www.linguacustodia.com <http://www.linguacustodia.com/> -
> www.thetranslationtrustee.com <http://www.thetranslationtrustee.com/>*
>
>
> This body part will be downloaded on demand.
>
>
>
> The University of Edinburgh is a charitable body, registered in
> Scotland, with registration number SC005336.
>
>
--
*M**. Vito MANDORINO -- Chief Scientist*
[image: Description : Description : lingua_custodia_final full logo]
*The Translation Trustee*
*1, Place Charles de Gaulle, **78180 Montigny-le-Bretonneux*
*Tel : +33 1 30 44 04 23 Mobile : +33 6 84 65 68 89*
*Email :* *vito.mandorino@linguacustodia.com
<massinissa.ahmim@linguacustodia.com>*
*Website :* *www.linguacustodia.com <http://www.linguacustodia.com/> -
www.thetranslationtrustee.com <http://www.thetranslationtrustee.com/>*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20151126/428663d8/attachment.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 4421 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20151126/428663d8/attachment.jpg
------------------------------
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
End of Moses-support Digest, Vol 109, Issue 65
**********************************************
Subscribe to:
Post Comments (Atom)
0 Response to "Moses-support Digest, Vol 109, Issue 65"
Post a Comment