Moses-support Digest, Vol 104, Issue 65

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: please help me with the code - getting word index
(Matthias Huck)
2. Exception when exiting moses (Marcin Junczys-Dowmunt)
3. Re: Exception when exiting moses (Hieu Hoang)
4. Re: Exception when exiting moses (Marcin Junczys-Dowmunt)
5. How to force not to reuse certain files in EMS (Dingyuan Wang)
6. Re: Exception when exiting moses (Hieu Hoang)


----------------------------------------------------------------------

Message: 1
Date: Sat, 20 Jun 2015 17:22:47 +0100
From: Matthias Huck <mhuck@inf.ed.ac.uk>
Subject: Re: [Moses-support] please help me with the code - getting
word index
To: amir haghighi <amir.haghighi.64@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID: <1434817367.3645.16.camel@inf.ed.ac.uk>
Content-Type: text/plain; charset="UTF-8"

Hi,

Yes, you need to calculate the absolute position by adding up the start
position of the current rule application, the relative index within the
rule, and the span width of any right-hand side non-terminals in the
current rule with a smaller source index.

As Rico noted, you'll find some similar code examples in existing
feature functions.

Cheers,
Matthias


On Sat, 2015-06-20 at 18:05 +0430, amir haghighi wrote:
> Thanks Matthias
> ChartHypothesis::GetCurrSourceRange() gets the source span that all
> terminals and non terminals in the current hypothesis cover in the
> source sentence. I'd like to know which terminals (non terminals) are
> corresponded to which source word's index in the source. Could you
> guide me how to obtain that?
>
>
> Thanks again
>
>
> On Thu, Jun 18, 2015 at 9:48 PM, Matthias Huck <mhuck@inf.ed.ac.uk>
> wrote:
> Hi,
>
> You can calculate absolute positions in the source sentence
> based on the
> words range of the current hypothesis and those of the direct
> predecessors (in case of right-hand side non-terminals).
>
> Take a look at these methods:
>
> InputPath::GetWordsRange()
> ChartHypothesis::GetCurrSourceRange()
> ChartCellLabel::GetCoverage()
>
> Cheers,
> Matthias
>
>
> On Thu, 2015-06-18 at 20:23 +0430, amir haghighi wrote:
> > Hi everybody
> >
> >
> > I wrote the following code to get an ordered list from the
> source words
> > inside a hypothesis. It gets the words in their translation
> order, but I
> > need not only the words' strings, but also the index of each
> word in the
> > original sentence.
> >
> > could you please help me how to get the index of each word
> in srcPhrase, in
> > the sentence?
> >
> >
> > void Amir::GetSourcePhrase2(const ChartHypothesis&
> cur_hypo,Phrase
> > &srcPhrase) const
> > {
> > AmirUtils utility;
> > TargetPhrase targetPh=cur_hypo.GetCurrTargetPhrase();
> > const Phrase *sourcePh=targetPh.GetRuleSource();
> > int
> targetWordsNum=cur_hypo.GetCurrTargetPhrase().GetSize();
> > std::vector <Word> source, orderedSource;
> > std::vector <int> alignmentVector;
> > std::vector <bool> isAligned;
> >
> > std::vector <std::set <size_t> > sourcePosSets;
> >
> > for(int targetP=0; targetP< targetWordsNum; targetP++ ){
> > //std::cerr<<"setting alignments for targetword:
> "<<targetP<<endl;
> >
> >
> sourcePosSets.push_back(cur_hypo.GetCurrTargetPhrase().GetAlignTerm().GetAlignmentsForTarget(targetP));
> > }
> >
> >
> > for(int ii=targetWordsNum-1; ii>=0; ii--){
> > std::set <size_t> cur_srcPosSet=sourcePosSets[ii];
> > for (std::set <size_t>::const_iterator alignmet =
> > cur_srcPosSet.begin();alignmet != cur_srcPosSet.end();
> ++alignmet) {
> > int alignmentElement=*alignmet;
> > for(int index=0; index<ii; index++ ){ //keep the
> rightmost one and
> > remove the othres
> > //remove it from the list
> > if(sourcePosSets[index].size()>0){
> > // std::cerr<<" removing "<<*alignmet<<endl;
> > //std::cerr<<" for set with size:
> > "<<sourcePosSets[index].size()<<endl;
> > sourcePosSets[index].erase(alignmentElement);
> > }
> >
> > }
> > }
> > }
> >
> > for (size_t posT = 0; posT <
> cur_hypo.GetCurrTargetPhrase().GetSize();
> > ++posT) {
> > const Word &word =
> cur_hypo.GetCurrTargetPhrase().GetWord(posT);
> > if (word.IsNonTerminal()){
> > // non-term. fill out with prev hypo
> >
> > size_t nonTermInd =
> >
> cur_hypo.GetCurrTargetPhrase().GetAlignNonTerm().GetNonTermIndexMap()[posT];
> > const ChartHypothesis *prevHypo =
> cur_hypo.GetPrevHypo(nonTermInd);
> >
> > GetSourcePhrase2(*prevHypo,srcPhrase);
> > }
> > else{
> >
> > for(std::set<size_t>::const_iterator
> > it=sourcePosSets[posT].begin();it !=
> sourcePosSets[posT].end() ; it++
> > ){
> > srcPhrase.AddWord(sourcePh->GetWord(*it));
> > }
> > }
> > }
> >
> >
> > }
>
> > _______________________________________________
> > Moses-support mailing list
> > Moses-support@mit.edu
> > http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
>
> --
> The University of Edinburgh is a charitable body, registered
> in
> Scotland, with registration number SC005336.
>
>
>



--
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.



------------------------------

Message: 2
Date: Sun, 21 Jun 2015 15:02:20 +0200
From: Marcin Junczys-Dowmunt <junczys@amu.edu.pl>
Subject: [Moses-support] Exception when exiting moses
To: moses-support <moses-support@mit.edu>
Message-ID: <5586B5DC.2010101@amu.edu.pl>
Content-Type: text/plain; charset=UTF-8; format=flowed

Hi,
is anyone else getting exceptions when moses exits with the latest
master? It seems to be happening in my reordering table and breaks MERT.
Wasn't me though :)



------------------------------

Message: 3
Date: Sun, 21 Jun 2015 17:19:28 +0400
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] Exception when exiting moses
To: Marcin Junczys-Dowmunt <junczys@amu.edu.pl>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAEKMkbh5ekJEWpgATZp9+mR1+Tb-kF7_JwNyT1W1zWAJVbPkfw@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

i'm using my latest master fork with no problems

Hieu Hoang
Researcher
New York University, Abu Dhabi
http://www.hoang.co.uk/hieu

On 21 June 2015 at 17:02, Marcin Junczys-Dowmunt <junczys@amu.edu.pl> wrote:

> Hi,
> is anyone else getting exceptions when moses exits with the latest
> master? It seems to be happening in my reordering table and breaks MERT.
> Wasn't me though :)
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150621/5ebfbfd7/attachment-0001.htm

------------------------------

Message: 4
Date: Sun, 21 Jun 2015 15:20:19 +0200
From: Marcin Junczys-Dowmunt <junczys@amu.edu.pl>
Subject: Re: [Moses-support] Exception when exiting moses
To: Hieu Hoang <hieuhoang@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID: <5586BA13.7030102@amu.edu.pl>
Content-Type: text/plain; charset=UTF-8; format=flowed

With the compact reordering table or anything else from Compact* ?

On 21.06.2015 15:19, Hieu Hoang wrote:
> i'm using my latest master fork with no problems
>
> Hieu Hoang
> Researcher
> New York University, Abu Dhabi
> http://www.hoang.co.uk/hieu
>
> On 21 June 2015 at 17:02, Marcin Junczys-Dowmunt <junczys@amu.edu.pl
> <mailto:junczys@amu.edu.pl>> wrote:
>
> Hi,
> is anyone else getting exceptions when moses exits with the latest
> master? It seems to be happening in my reordering table and breaks
> MERT.
> Wasn't me though :)
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>



------------------------------

Message: 5
Date: Sun, 21 Jun 2015 21:23:40 +0800
From: Dingyuan Wang <abcdoyle888@gmail.com>
Subject: [Moses-support] How to force not to reuse certain files in
EMS
To: moses-support <moses-support@mit.edu>
Message-ID: <5586BADC.7060505@gmail.com>
Content-Type: text/plain; charset=utf-8

Hi,

In my experiments the main variable is the corpus text generated
before running the EMS. Sometimes the pre-built LM changes outside the
EMS too.
The EMS system checkes config file changes to determine which files
can be reused in the next experiment step. However it's a waste of
time and resources to change the filenames of the corpus and the LM
every time, or rerun the entire experiment, or go through the baseline
system manually.
I can hardly read Perl, but I think there is an easy way to make this
possible. Maybe an option or timestamp check can help.

Thanks.


------------------------------

Message: 6
Date: Sun, 21 Jun 2015 17:27:09 +0400
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] Exception when exiting moses
To: Marcin Junczys-Dowmunt <junczys@amu.edu.pl>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAEKMkbj6PhhRY_ecEKXCdaF97kVO+piEVq6ro9o+Q-gueqoe6Q@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

with compact, from today's checkin

Hieu Hoang
Researcher
New York University, Abu Dhabi
http://www.hoang.co.uk/hieu

On 21 June 2015 at 17:20, Marcin Junczys-Dowmunt <junczys@amu.edu.pl> wrote:

> With the compact reordering table or anything else from Compact* ?
>
> On 21.06.2015 15:19, Hieu Hoang wrote:
>
>> i'm using my latest master fork with no problems
>>
>> Hieu Hoang
>> Researcher
>> New York University, Abu Dhabi
>> http://www.hoang.co.uk/hieu
>>
>> On 21 June 2015 at 17:02, Marcin Junczys-Dowmunt <junczys@amu.edu.pl
>> <mailto:junczys@amu.edu.pl>> wrote:
>>
>> Hi,
>> is anyone else getting exceptions when moses exits with the latest
>> master? It seems to be happening in my reordering table and breaks
>> MERT.
>> Wasn't me though :)
>>
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150621/cde68e1e/attachment.htm

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 104, Issue 65
**********************************************

Tweet