Moses-support Digest, Vol 110, Issue 34

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: PhraseDictionaryCompact is not registered
(Marcin Junczys-Dowmunt)
2. Re: Slides or paper walking through
SearchNormal::ProcessOneHypothesis ? (Wilker Aziz)
3. Re: Compiling Moses with -fPIC (Miriam K?shammer)


----------------------------------------------------------------------

Message: 1
Date: Fri, 18 Dec 2015 16:12:43 +0100
From: Marcin Junczys-Dowmunt <junczys@amu.edu.pl>
Subject: Re: [Moses-support] PhraseDictionaryCompact is not registered
To: moses-support@mit.edu
Message-ID: <5674226B.8060909@amu.edu.pl>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed

Hi,
I'd say you didn't install cmph or compile against it, look again at:

http://www.statmt.org/moses/?n=Advanced.RuleTables#ntoc3

On 18.12.2015 15:15, Andrew wrote:
> I'm following the baseline system page step-by-step as it says.
> I've binarized the phrase table and reordering table using
> processPhraseTableMin and processLexicalTableMin,
> edited the moses.ini as written,
> but upon executing, it gives an exception with
> "PhraseDictionaryCompact is not registered" message.
>
> I've done some googling, and tried running processLexicalTable
> (without "Min") to no good,
> and also tried editing as PhraseDictionaryBinary, PhraseDictionaryOnDisk,
> which succeeded in running the task, but gets aborted upon writing the
> input sentence.
>
> Is there be any other workaround / fix to this?
>
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support



------------------------------

Message: 2
Date: Fri, 18 Dec 2015 17:12:52 +0100
From: Wilker Aziz <will.aziz@gmail.com>
Subject: Re: [Moses-support] Slides or paper walking through
SearchNormal::ProcessOneHypothesis ?
To: Matthias Huck <mhuck@inf.ed.ac.uk>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<CA+=Luv8_iHrqMEWxHaO4UNrW+YHmmq9V3QS-AVaFZ6EJnMzA5g@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hi,

I hope it is not too late to add to this discussion.

If you are comfortable with weighted deduction, Adam Lopez's 2009 EACL
paper is very a good reference for phrase-based reordering spaces. If I
remember well the implementation in Moses does exactly what he shows with
the logic program WLd.

http://alopez.github.io/papers/eacl2009-lopez.pdf

Cheers,

Wilker

On 16 December 2015 at 00:56, Matthias Huck <mhuck@inf.ed.ac.uk> wrote:

> Hi Lane,
>
> Well, you can find excellent descriptions of phrase-based decoding
> algorithms in the literature, though possibly not all details of this
> specific implementation.
>
> I like this description:
>
> R. Zens, and H. Ney. Improvements in Dynamic Programming Beam Search for
> Phrase-based Statistical Machine Translation. In International Workshop
> on Spoken Language Translation (IWSLT), pages 195-205, Honolulu, HI,
> USA, October 2008.
>
> http://www.hltpr.rwth-aachen.de/publications/download/618/Zens-IWSLT-2008.pdf
>
> It's what's implemented in Jane, RWTH's open source statistical machine
> translation toolkit.
>
> J. Wuebker, M. Huck, S. Peitz, M. Nuhn, M. Freitag, J. Peter, S.
> Mansour, and H. Ney. Jane 2: Open Source Phrase-based and Hierarchical
> Statistical Machine Translation. In International Conference on
> Computational Linguistics (COLING), pages 483-491, Mumbai, India,
> December 2012.
>
> http://www.hltpr.rwth-aachen.de/publications/download/830/Wuebker-COLING-2012.pdf
>
> However, I believe that the distinction of coverage hypotheses and
> lexical hypotheses is a unique property of the RWTH systems.
>
> The formalization in the Zens & Ney paper is very nicely done. With hard
> distortion limits or coverage-based reordering constraints, you may need
> a few more steps in the algorithm. E.g., if you have a hard distortion
> limit, you will probably want to avoid leaving a gap and then extending
> your sequence in a way that puts your current position further away from
> the gap than your maximum jump width. Other people should know more
> about how exactly Moses' phrase-based decoder is dealing with this.
>
> I can recommend Richard Zens' PhD thesis as well.
> http://www.hltpr.rwth-aachen.de/publications/download/562/Zens--2008.pdf
>
> I also remember that the following publication from Microsoft Research
> is pretty helpful:
>
> Robert C. Moore and Chris Quirk, Faster Beam-Search Decoding for Phrasal
> Statistical Machine Translation, in Proceedings of MT Summit XI,
> European Association for Machine Translation, September 2007.
> http://research.microsoft.com/pubs/68097/mtsummit2007_beamsearch.pdf
>
> Cheers,
> Matthias
>
>
>
> On Tue, 2015-12-15 at 22:33 +0000, Hieu Hoang wrote:
> > I've been looking at this and it is surprisingly complicated. I think
> > the code is designed to predetermine if extending a hypothesis will
> > lead it down a path that won't ever be completed.
> >
> >
> > Don't know any slide that explains the reasoning, Philipp Koehn
> > explained it to me once and it seems pretty reasonable.
> >
> >
> >
> > I wouldn't mind seeing this code cleaned up a bit and abstracted and
> > formalised. I've made a start with the cleanup in my new decoder
> >
> >
> https://github.com/moses-smt/mosesdecoder/blob/perf_moses2/contrib/other-builds/moses2/Search/Search.cpp#L36
> > Search::CanExtend()
> >
> >
> > There was an Aachen paper from years ago comparing different
> > distortion limit heuristics - can't remember the authors or title.
> > Maybe someone know more
> >
> >
> >
> >
> >
> > Hieu Hoang
> > http://www.hoang.co.uk/hieu
> >
> >
> > On 15 December 2015 at 20:59, Lane Schwartz <dowobeha@gmail.com>
> > wrote:
> > Hey all,
> >
> >
> > So the SearchNormal::ProcessOneHypothesis() method in
> > SearchNormal.cpp is responsible for taking an existing
> > hypothesis, creating all legal new extension hypotheses, and
> > adding those new hypotheses to the appropriate decoder
> > stacks.
> >
> >
> > First off, the method is actually reasonably well commented,
> > so kudos to whoever did that. :)
> >
> >
> > That said, does anyone happen to have any slides that actually
> > walk through this process, specifically slides that take into
> > account the interaction with the distortion limit? That
> > interaction is where most of the complexity of this method
> > comes from. I don't know about others, but even having a
> > pretty good notion of what's going on here, the discussion of
> > "the closest thing to the left" is still a bit opaque.
> >
> >
> > Anyway, if anyone knows of a good set of slides, or even a
> > good description in a paper, of what's going on here, I'd
> > appreciate any pointers.
> >
> >
> > Thanks,
> > Lane
> >
> >
> >
> > _______________________________________________
> > Moses-support mailing list
> > Moses-support@mit.edu
> > http://mailman.mit.edu/mailman/listinfo/moses-support
> >
> >
> >
> > _______________________________________________
> > Moses-support mailing list
> > Moses-support@mit.edu
> > http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
>
> --
> The University of Edinburgh is a charitable body, registered in
> Scotland, with registration number SC005336.
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>



--
Wilker Aziz
http://wilkeraziz.github.io <http://pers-www.wlv.ac.uk/~in1676/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20151218/c64898b3/attachment-0001.html

------------------------------

Message: 3
Date: Fri, 18 Dec 2015 17:17:57 +0100
From: "Miriam K?shammer" <miriam.kaeshammer@gmx.de>
Subject: Re: [Moses-support] Compiling Moses with -fPIC
To: Moses-support@mit.edu
Message-ID:
<trinity-82548735-e4c6-451d-8ac0-8fc7e94c20bb-1450455476982@3capp-gmx-bs17>

Content-Type: text/plain; charset="us-ascii"

An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20151218/9ddcfe28/attachment.html

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 110, Issue 34
**********************************************

Tweet