Send Moses-support mailing list submissions to
moses-support@mit.edu
To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu
You can reach the person managing the list at
moses-support-owner@mit.edu
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."
Today's Topics:
1. Re: lattice MBR - what happened to my nbest features?
(Barry Haddow)
2. Re: Extract phrases with no internal alignment points
(Philipp Koehn)
3. Re: Multiple Translation Tables and Back-off Models in Moses
(Philipp Koehn)
4. Re: mert.out error (Philipp Koehn)
5. Re: Moses as a service under release 3.0 (Jeroen Vermeulen)
----------------------------------------------------------------------
Message: 1
Date: Wed, 06 May 2015 22:01:39 +0100
From: Barry Haddow <bhaddow@inf.ed.ac.uk>
Subject: Re: [Moses-support] lattice MBR - what happened to my nbest
features?
To: Jeremy Gwinnup <jeremy@gwinnup.org>, moses-support@mit.edu
Message-ID: <554A8133.4020107@inf.ed.ac.uk>
Content-Type: text/plain; charset=UTF-8; format=flowed
Hi Jeremy
It's been a long time since I looked at this, but I think these are the
component scores in the linearised corpus bleu used in Lattice MBR (see
Tromble et al (2008), section 5). They are used to rescore the nbest
list, and must be implemented by replacing the original feature vectors.
So it looks like lattice MBR doesn't work with nbest lists, or at least
doesn't give you what you need for tuning. Do you really want to tune
with lattice MBR? It's going to be very slow,
cheers - Barry
On 06/05/15 20:49, Jeremy Gwinnup wrote:
> Hi,
>
> I?ve been attempting to experiment with lattice MBR with various settings and I see something weird happen to my nbest lists:
>
> 0 ||| the prime ministers of india and japan meet in tokyo ||| Distortion0= ? etc etc becomes
>
> 0 ||| the prime ministers of india and japan meet in tokyo ||| map: 0 w: 12 11.19 8.06 6.63 5.39
>
> My feature weights get replaced by map and w feature with 5 weights
>
> I?m setting the following as moses.ini parameters:
> lminimum-bayes-risk
> lmbr-p
> lmbr-r
> mbr-scale
> lmbr-pruning-factor
>
> Anything extra that I need to know as far as what map and w are, or how to get my normal features back in the nbest list?
>
> Thanks!
> -Jeremy
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
--
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.
------------------------------
Message: 2
Date: Wed, 6 May 2015 20:04:15 -0400
From: Philipp Koehn <phi@jhu.edu>
Subject: Re: [Moses-support] Extract phrases with no internal
alignment points
To: Miguel Gra?a <miguelrgraca@gmail.com>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<CAAFADDAxxO+PQ_NEwA+7KFB+1fPYX0An_atB5CTLM2T9dsvzQQ@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Hi,
the definition of consistent phrase pair that is the basis of the
phrase extraction method requires that each phrase pair has at least
one alignment point.
You should change the function ExtractTask::extract in
phrase-extract/extract-main.cpp if you want to base phrase extraction
on a different principle.
-phi
On Tue, May 5, 2015 at 5:18 AM, Miguel Gra?a <miguelrgraca@gmail.com> wrote:
> Hello,
>
> I couldn't find any information about this. Is it possible to not only
> extract phrases which have at least one alignment point, but also extract
> phrases where there are no alignment points?
>
> Thanks in advance,
> Miguel
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
------------------------------
Message: 3
Date: Wed, 6 May 2015 21:22:10 -0400
From: Philipp Koehn <phi@jhu.edu>
Subject: Re: [Moses-support] Multiple Translation Tables and Back-off
Models in Moses
To: xinzyang@mail.ustc.edu.cn
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<CAAFADDBN7cbgr3wNOb+5NNtVh-zPsCFWY1W2mxTHnSpzA7Bwog@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Hi,
I added a section to the Moses documentation on how to deal with this
situation:
http://www.statmt.org/moses/?n=Advanced.Models#ntoc5
-phi
*Caveat: Multiple Translation Tables and Lexicalized Reordering* You may
specify any number of lexicalized reordering models. Each of them will
score any translation option, no matter where it comes from. If a
lexicalized reordering table does not have an entry for a translation
options, it will not assign any score to it. In other words, such a
translation option is given the probability 1 no matter how it is
reordered. This may not be the way you want to handle it.
For instance, if you have an "in-domain" translation table and an
*out-of-domain* translation table, you can also provide an *in-domain*
reordering
table and an*out-of-domain* reordering table. If a phrase pair occurs in
both translation tables, it will be scored by both reordering tables.
However, if a phrase pairs occurs only in one of the phrase tables (and
hence reordering tables), it will be only score by one of them and get a
free ride with the other. this will have the undesirable effect of
discouraging phrase pairs that occur in both tables.
To avoid this, you can add default scores to the reordering table:
LexicalReordering name=LexicalReordering0 num-features=6
type=wbe-msd-bidirectional-fe-allff [...]
default-scores=0.5,0.3,0.2,0.5,0.3,0.2
LexicalReordering name=LexicalReordering1 num-features=6
type=wbe-msd-bidirectional-fe-allff [...]
default-scores=0.5,0.3,0.2,0.5,0.3,0.2
On Wed, May 6, 2015 at 3:00 AM, <xinzyang@mail.ustc.edu.cn> wrote:
> Hi,
>
> I am studying moses to do some research abou SMT. I have read advanced
> models in moses at the website
> http://www.statmt.org/moses/?n=Advanced.Models#ntoc5 . I have some
> questions about the introduction of "Multiple Translation Tables and
> Back-off Models".
>
> From the website, I know that moses allows the use of multiple translation
> tables. But the lexicalized reordering model is not mentioned in the
> introduction of "Multiple Translation Tables and Back-off Models". So, *could
> I use two translation tables, which I denote translation table1 and
> translation table2 respectively, and one lexicalized reordering table*
> according to the introduction of "Multiple Translation Tables and Back-off
> Models"? The phrase pairs in reordering table is the same with translation
> table1, but not the same with translation table2.
>
> When I use multiple translation tables and only one reordering table, a
> phrase pair may exist in only one translation table and don't exist in the
> reordering model, how to calculate the probability of lexicalized
> reordering model?
>
> I'm sorry for bothering you. Thanks and Best Regards,
>
> Zhenxin
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150506/6355017d/attachment-0001.htm
------------------------------
Message: 4
Date: Wed, 6 May 2015 22:25:34 -0400
From: Philipp Koehn <phi@jhu.edu>
Subject: Re: [Moses-support] mert.out error
To: emna hkiri <emna.hkiri@gmail.com>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<CAAFADDDBF4f0QV4erk_gyNeKbrJbrtmR43cjTUtvZR231=RSCQ@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Hi,
what are the contents of "extract.out" and "extract.err"?
-phi
On Fri, May 1, 2015 at 12:54 PM, emna hkiri <emna.hkiri@gmail.com> wrote:
> dear friends
> I am stuck with this message in my mert.out after going through line 1539:
>
> Scoring the nbestlist.
>
> exec: /home/emna/emna/working/mert-work/extractor.sh
>
> Executing: /home/emna/emna/working/mert-work/extractor.sh > extract.out 2>
> extract.err
>
> Exit code: 1
>
> ERROR: Failed to run '/home/emna/emna/working/mert-work/extractor.sh'. at
> /home/emna/emna/mosesdecoder/scripts/training/mert-moses.pl line 1539.
>
> any help please? ? the file mert-work/extractor.sh is in the correct place
>
> thank you in advance
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150506/ee3f552e/attachment-0001.htm
------------------------------
Message: 5
Date: Thu, 07 May 2015 11:38:04 +0700
From: Jeroen Vermeulen <jtv@precisiontranslationtools.com>
Subject: Re: [Moses-support] Moses as a service under release 3.0
To: Barry Haddow <bhaddow@inf.ed.ac.uk>, kwame porter robinson
<k.porter.robinson@gmail.com>, moses-support <moses-support@mit.edu>
Message-ID:
<A0C00FC6-A7BC-493D-AB97-D0687F877C02@precisiontranslationtools.com>
Content-Type: text/plain; charset="utf-8"
On May 7, 2015 2:42:53 AM GMT+07:00, Barry Haddow <bhaddow@inf.ed.ac.uk> wrote:
>Hi Kwame
>
>The --server switch was introduced post v3, so you would have to use
>current master if you want to go down that route. However you will
>still
>hit the xmlrpc issues.
>
>> made soft links from the so.8 libs to so.7 so that mosesserver can
>> load the library (figuring that the functionality/interface was the
>same).
>Probably there's a reason for the version number change from 7 to 8,
>and
>you've just hit the incompatibility. Make sure you build and run
>against
>the same version of xmlrpc-c - we're using 1.16.33 which is probably
>old, but works.
>
>cheers - Barry
>
>
>On 06/05/15 19:59, kwame porter robinson wrote:
>> Hi,
>>
>> I am attempting to use the binary release 3 moses as a service.
>Recent
>> discussion indicates that mosesserver is depreciated and that moses
>> --server should be used instead. However, running 'moses --server
>> --server-port 1234 -f phrase-model/moses.ini' complains about illegal
>
>> switches.
>>
>> So I looked at mosseserver. It complains about libxmlrpc libraries
>> (*.so.7) not being found. So I installed the libxmlrpc-c++ package
>> (1.33.06) and made soft links from the so.8 libs to so.7 so that
>> mosesserver can load the library (figuring that the
>> functionality/interface was the same).
>>
>> However, moseserver seg faults, when called with 'mosesserver
>> --server-port 1234 -f phrase-model/moses.ini'. Does anyone know
>> where/how to get libxmlrpc libraries required by moses release 3.0 or
>
>> otherwise make moses as a service w/o having to re-build it? I'm
>using
>> Ubunut 14.04.
>>
>> If all else fails I can use moses' stdio but I was hoping to get this
>
>> working. Thank you for your time.
>>
>> - Kwame
>>
>>
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
>
>------------------------------------------------------------------------
>
>The University of Edinburgh is a charitable body, registered in
>Scotland, with registration number SC005336.
>
>
>------------------------------------------------------------------------
>
>_______________________________________________
>Moses-support mailing list
>Moses-support@mit.edu
>http://mailman.mit.edu/mailman/listinfo/moses-support
Also, C++ ABIs are by nature much more brittle than what you get with C libraries. It only takes a very small change, sometimes even just a compiler upgrade, to break binary compatibility.
Jeroen
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150507/a1a8d194/attachment.htm
------------------------------
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
End of Moses-support Digest, Vol 103, Issue 24
**********************************************
Subscribe to:
Post Comments (Atom)
0 Response to "Moses-support Digest, Vol 103, Issue 24"
Post a Comment