Moses-support Digest, Vol 113, Issue 38

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: Reseting data structures for each new sentence (Hieu Hoang)
2. Preparing TMX files for use in Moses (Sa?o Kuntaric)
3. Re: Compiling with ./bjam problem (Despina Mouratidi)


----------------------------------------------------------------------

Message: 1
Date: Fri, 11 Mar 2016 18:17:11 +0000
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] Reseting data structures for each new
sentence
To: Jasneet Sabharwal <jasneet.sabharwal@sfu.ca>
Cc: moses-support <moses-support@mit.edu>
Message-ID: <56E30BA7.6090903@gmail.com>
Content-Type: text/plain; charset="utf-8"

oops, i forgot. In phrase-based decoding, the 1st hypothesis (in stack
0) have states created by the method
StatefulFeatureFunction::EmptyHypothesisState()
that you have to override for every stateful ff. Put code to rest your
state info here

On 11/03/2016 09:03, Jasneet Sabharwal wrote:
> Hi Hieu,
>
> Thank you for your reply. I tried both of these approaches.
>
> In first case, I get an exception at NOT_FOUND. When I try to print
> the words range array, the minimum is always "wordsRange: [0..0]?.
>
> In second case, "cur_hypo.GetWordsBitmap().GetNumWordsCovered() == 0?
> is never true as the minimum I get is equal to 1.
>
> As, InitializeForInput() cannot be useful in my current situation, is
> there any other way to know when I start processing a new sentence?
>
> Thank you and regards,
> Jasneet
>> On Mar 9, 2016, at 3:41 PM, Hieu Hoang <hieuhoang@gmail.com
>> <mailto:hieuhoang@gmail.com>> wrote:
>>
>> i get ya. You can do
>>
>> EvaluateWhenApplied(Hypothesis cur_hypo, FFState,
>> ScoreComponenetCollection) {
>> If (cur_hypo.GetCurrSourceWordsRange() == NOT_FOUND) then {
>> LMState = CoarseLM->BeginSentenceState()
>> }
>> }
>>
>> or
>>
>> EvaluateWhenApplied(Hypothesis cur_hypo, FFState,
>> ScoreComponenetCollection) {
>> If (cur_hypo.GetWordsBitmap().GetNumWordsCovered() == 0) then {
>> LMState = CoarseLM->BeginSentenceState()
>> }
>> }
>>
>> Each FF can also override
>> InitializeForInput()
>>
>> This might not be suitable for you on this occasion, but it's worth
>> bearing in mind
>>
>>
>> On 09/03/2016 23:20, Jasneet Sabharwal wrote:
>>> Hi Hieu,
>>>
>>> Thanks for replying. For example:
>>>
>>> EvaluateWhenApplied(Hypothesis, FFState, ScoreComponenetCollection) {
>>> If (processing a new source sentence), then {
>>> LMState = CoarseLM->BeginSentenceState()
>>> }
>>> }
>>>
>>> Here, I am trying to check if I am processing a new source sentence,
>>> so I should reset my language model states that I calling from my
>>> feature function.
>>>
>>> Does this give an idea of what I am trying to do?
>>>
>>> Regards,
>>> Jasneet
>>>
>>>> On Mar 9, 2016, at 2:51 PM, Hieu Hoang <hieuhoang@gmail.com> wrote:
>>>>
>>>> can you give an example
>>>>
>>>> Hieu Hoang
>>>> http://www.hoang.co.uk/hieu
>>>>
>>>> On 9 March 2016 at 22:36, Jasneet Sabharwal
>>>> <jasneet.sabharwal@sfu.ca> wrote:
>>>>
>>>> Hi Everyone,
>>>>
>>>> I have added a feature function in Moses. Is it possible to
>>>> know through the Hypothesis parameter in ?EvaluateWhenApplied?
>>>> function that I am now processing the first state in a new
>>>> source sentence. I would like to reset certain states whenever
>>>> we start translating a new source sentence.
>>>>
>>>> Thank you for the help.
>>>>
>>>> Regards,
>>>> Jasneet
>>>> _______________________________________________
>>>> Moses-support mailing list
>>>> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
>>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>>
>>>>
>>>
>>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20160311/9afb8574/attachment-0001.html

------------------------------

Message: 2
Date: Sat, 12 Mar 2016 13:42:05 +0100
From: Sa?o Kuntaric <saso.kuntaric@gmail.com>
Subject: [Moses-support] Preparing TMX files for use in Moses
To: moses-support@mit.edu
Message-ID:
<CANsquDp+NYjVxTioCiMMzGVX94HAb6XCbXxOz=x24RzjW6UbKw@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hi all,

I have a question that is not connected directly to Moses. I am trying to
prepare the corpora for training my engine. I have exported a few of my TMs
to the TMX format and now I am trying to create two separate UTF-8 text
files. I have tried it with the extract-tmx-corpus and tmx2txt.pl tools. I
get empty text files for both (the former tool claims that the input file
can't be read). Are there any special setting I need to set when extracting
the TMX files? I am using SDL Trados Studio 2015 for exporting the files.

Has anyone come across anything like this?

--
lp,

Sa?o
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20160312/93cce894/attachment-0001.html

------------------------------

Message: 3
Date: Sat, 12 Mar 2016 15:57:43 +0200
From: Despina Mouratidi <dmouratidi@gmail.com>
Subject: Re: [Moses-support] Compiling with ./bjam problem
To: Matthias Huck <mhuck@inf.ed.ac.uk>
Cc: moses-support@mit.edu
Message-ID:
<CAJJG54=0pyxSzzKqjucdym6==NNB9HVXPq6B4GZvbMweoGxi5w@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hello again,
i installed boost_1_60_0 and ERROR again.

.......
Data::m_score_type BLEU
Data::Scorer type from Scorer: BLEU
Data::m_score_type BLEU
Data::Scorer type from Scorer: BLEU
Data::m_score_type BLEU
Data::Scorer type from Scorer: BLEU
Data::m_score_type BLEU
Data::Scorer type from Scorer: BLEU
Data::m_score_type BLEU
Data::Scorer type from Scorer: BLEU

*** No errors detected
gcc.link
mert/bin/gcc-4.8/release/link-static/threading-multi/mira_feature_vector_test
testing.unit-test
mert/bin/gcc-4.8/release/link-static/threading-multi/mira_feature_vector_test.passed
Running 1 test case...

*** No errors detected
gcc.compile.c++
mert/bin/gcc-4.8/release/link-static/threading-multi/ForestRescoreTest.o
gcc.compile.c++
mert/bin/gcc-4.8/release/link-static/threading-multi/HypergraphTest.o
gcc.link
mert/bin/gcc-4.8/release/link-static/threading-multi/forest_rescore_test
testing.unit-test
mert/bin/gcc-4.8/release/link-static/threading-multi/forest_rescore_test.passed
Running 2 test cases...

*** No errors detected
gcc.link
mert/bin/gcc-4.8/release/link-static/threading-multi/hypergraph_test
testing.unit-test
mert/bin/gcc-4.8/release/link-static/threading-multi/hypergraph_test.passed
Running 1 test case...

*** No errors detected
gcc.compile.c++
mert/bin/gcc-4.8/release/link-static/threading-multi/OptimizerFactoryTest.o
gcc.link
mert/bin/gcc-4.8/release/link-static/threading-multi/optimizer_factory_test
testing.unit-test
mert/bin/gcc-4.8/release/link-static/threading-multi/optimizer_factory_test.passed
Running 2 test cases...

*** No errors detected
gcc.compile.c++
mert/bin/gcc-4.8/release/link-static/threading-multi/ReferenceTest.o
gcc.compile.c++
mert/bin/gcc-4.8/release/link-static/threading-multi/SingletonTest.o
gcc.link mert/bin/gcc-4.8/release/link-static/threading-multi/reference_test
...on 700th target...
testing.unit-test
mert/bin/gcc-4.8/release/link-static/threading-multi/reference_test.passed
Running 5 test cases...

*** No errors detected
gcc.link mert/bin/gcc-4.8/release/link-static/threading-multi/singleton_test
testing.unit-test
mert/bin/gcc-4.8/release/link-static/threading-multi/singleton_test.passed
Running 1 test case...

*** No errors detected
gcc.compile.c++
mert/bin/gcc-4.8/release/link-static/threading-multi/NgramTest.o
gcc.link mert/bin/gcc-4.8/release/link-static/threading-multi/ngram_test
testing.unit-test
mert/bin/gcc-4.8/release/link-static/threading-multi/ngram_test.passed
Running 3 test cases...

*** No errors detected
gcc.compile.c++
mert/bin/gcc-4.8/release/link-static/threading-multi/PointTest.o
gcc.link mert/bin/gcc-4.8/release/link-static/threading-multi/point_test
testing.unit-test
mert/bin/gcc-4.8/release/link-static/threading-multi/point_test.passed
Running 1 test case...

*** No errors detected
gcc.link mert/mert
gcc.link mert/extractor
gcc.link mert/evaluator
gcc.link mert/pro
gcc.compile.c++
mert/bin/gcc-4.8/release/link-static/threading-multi/TimerTest.o
gcc.link mert/kbmira
gcc.link mert/bin/gcc-4.8/release/link-static/threading-multi/timer_test
testing.unit-test
mert/bin/gcc-4.8/release/link-static/threading-multi/timer_test.passed
Running 1 test case...

*** No errors detected
gcc.link mert/sentence-bleu
gcc.link mert/sentence-bleu-nbest
gcc.link mert/hgdecode
gcc.compile.c++
mert/bin/gcc-4.8/release/link-static/threading-multi/VocabularyTest.o
gcc.link
mert/bin/gcc-4.8/release/link-static/threading-multi/vocabulary_test
testing.unit-test
mert/bin/gcc-4.8/release/link-static/threading-multi/vocabulary_test.passed
Running 2 test cases...

*** No errors detected
gcc.compile.c++
mert/bin/gcc-4.8/release/link-static/threading-multi/UtilTest.o
gcc.link mert/bin/gcc-4.8/release/link-static/threading-multi/util_test
testing.unit-test
mert/bin/gcc-4.8/release/link-static/threading-multi/util_test.passed
Running 3 test cases...

*** No errors detected
...failed updating 1 target...
...skipped 73 targets...
...updated 721 targets...
The build failed. If you need support, run:
/usr/bin/bjam --with-boost=/home/despina/boost_1_60_0 -j4 -a
--debug-configuration -d2 |gzip >build.log.gz
then attach build.log.gz to your e-mail.
You MUST do 3 things before sending to the mailing list:
1. Subscribe to the mailing list at
http://mailman.mit.edu/mailman/listinfo/moses-support
2. Attach build.log.gz to your e-mail
3. Say what is the EXACT command you executed when you got the error
ERROR


Thank you so much
Despina

2016-03-12 15:22 GMT+02:00 Despina Mouratidi <dmouratidi@gmail.com>:

> Hello, thank you,
> i did it. It seemed to work at first, but after a while an error appeared
> again.:( I will install boost_1_60_0.
>
> Best
> Despina
>
> 2016-03-11 20:08 GMT+02:00 Matthias Huck <mhuck@inf.ed.ac.uk>:
>
>> Hi Despina,
>>
>> It seems to me that bjam doesn't use the boost build in your home
>> directory, but some other boost version installed on the system.
>>
>> Maybe you should try
>>
>> ./bjam --with-boost=/home/despina/boost_1_55_0 -j4 -a
>>
>> Cheers,
>> Matthias
>>
>>
>> On Fri, 2016-03-11 at 16:29 +0000, Hieu Hoang wrote:
>> > compiled ok for me:
>> > ./bjam --with-boost=/Users/hieu/workspace/boost/boost_1_60_0 -j4 -a
>> > Try git pull to get the latest code and add the -a to your bjam command
>> > to forced recompilation of all files
>> >
>> > On 11/03/2016 13:07, Despina Mouratidi wrote:
>> > > Hello again, i am trying to compile moses throught ./bjam as follows
>> > >
>> > > ./bjam --with-boost=~/boost_1_55_0 -j7
>> > >
>> > >
>> > > and it gives error.
>> > >
>> > > Best Despina
>> > >
>> > >
>> > > _______________________________________________
>> > > Moses-support mailing list
>> > > Moses-support@mit.edu
>> > > http://mailman.mit.edu/mailman/listinfo/moses-support
>> >
>> > _______________________________________________
>> > Moses-support mailing list
>> > Moses-support@mit.edu
>> > http://mailman.mit.edu/mailman/listinfo/moses-support
>>
>>
>>
>> --
>> The University of Edinburgh is a charitable body, registered in
>> Scotland, with registration number SC005336.
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20160312/b6f76566/attachment.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 1.png
Type: image/png
Size: 112477 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20160312/b6f76566/attachment.png

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 113, Issue 38
**********************************************

Tweet