Send Moses-support mailing list submissions to
moses-support@mit.edu
To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu
You can reach the person managing the list at
moses-support-owner@mit.edu
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."
Today's Topics:
1. training or tuning ? (Vincent Nguyen)
2. BLEU result on baseline EMS experiment (Vincent Nguyen)
3. Re: Support for XML Markup with Confusion Network Input
(Hieu Hoang)
4. Re: BLEU result on baseline EMS experiment (Hieu Hoang)
5. decoding-graph-backoff (Saumitra Yadav)
----------------------------------------------------------------------
Message: 1
Date: Tue, 21 Jul 2015 21:09:27 +0200
From: Vincent Nguyen <vnguyen@neuf.fr>
Subject: [Moses-support] training or tuning ?
To: Hieu Hoang <hieuhoang@gmail.com>, moses-support
<moses-support@mit.edu>
Message-ID: <55AE98E7.6020402@neuf.fr>
Content-Type: text/plain; charset=windows-1252; format=flowed
thanks for your replies.
I have done the full EMS with no errors. BLEU seems fine.
the EMS was dnoe with the WMT12 parallel data.
From here what should I do for real assessment.
I have real texts (human translation)
should they be used for training or for tuning ?
second question :
is there the files of this demo http://demo.statmt.org/ so that I can
put this on my server
it's much easier to test with real data .....
Le 21/07/2015 07:21, Hieu Hoang a ?crit :
>
> On 19/07/2015 23:06, Vincent Nguyen wrote:
>> I finally went through the all Baseline process with the KenLM model.
>>
>> results are mitigated, so from here what would be the best practices ?
>>
>> 1) I saw online a bunch of corpus available from the European Union
>> should this be used to train the translation system AND the langue
>> model or just one of the 2 ?
> you cab use the data for creating both the language model and the
> translation model. The only thing you have to make sure is that your
> training data is not part of the tuning or test data
>> 2) Is there a benchmark between the different model (Kenlm, Irstlm, ...)
>> ie is there a big difference in the observed results ?
>> is it worth trying several ones ?
> Try it yourself and tell us the results.
>> 3) I read an article mentioning that the results after the tuning were
>> not as good as before ...
>> does this make any sense ?
> If you report BLEU score without tuning first, you will be crucified,
> see this thread:
> https://www.mail-archive.com/moses-support@mit.edu/msg12593.html
> You MUST tune. Tuning can sometime to difficult. See this post on how to
> pick a good tuning set:
> https://www.mail-archive.com/moses-support@mit.edu/msg12594.html
>> Thanks.
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>
------------------------------
Message: 2
Date: Tue, 21 Jul 2015 21:45:56 +0200
From: Vincent Nguyen <vnguyen@neuf.fr>
Subject: [Moses-support] BLEU result on baseline EMS experiment
To: moses-support <moses-support@mit.edu>
Message-ID: <55AEA174.5080508@neuf.fr>
Content-Type: text/plain; charset=utf-8; format=flowed
here is what I got
make sense ?
MT evaluation scorer began on 2015 Jul 20 at 23:27:39
command line: /home/moses/mosesdecoder/scripts/generic/mteval-v13a.pl -c
-c -s /home/moses/working/data/dev/newstest2011-src.fr.sgm -r
/home/moses/working/data/dev/newstest2011-ref.en.sgm -t
/home/moses/working/evaluation/newstest2011.detokenized.sgm.3
Evaluation of any-to-en translation using:
src set "newstest2011" (110 docs, 3003 segs)
ref set "newstest2011" (1 refs)
tst set "newstest2011" (1 systems)
length ratio: 0.994844739625875 (74296/74681), penalty (log):
-0.00518197480348868
NIST score = 6.8964 BLEU score = 0.2268 for system "Edinburgh"
# ------------------------------------------------------------------------
Individual N-gram scoring
1-gram 2-gram 3-gram 4-gram 5-gram 6-gram 7-gram
8-gram 9-gram
------ ------ ------ ------ ------ ------ ------
------ ------
NIST: 5.2752 1.3399 0.2499 0.0273 0.0041 0.0005 0.0000
0.0000 0.0000 "Edinburgh"
BLEU: 0.5883 0.2887 0.1636 0.0972 0.0589 0.0364 0.0230
0.0146 0.0093 "Edinburgh"
# ------------------------------------------------------------------------
Cumulative N-gram scoring
1-gram 2-gram 3-gram 4-gram 5-gram 6-gram 7-gram
8-gram 9-gram
------ ------ ------ ------ ------ ------ ------
------ ------
NIST: 5.2752 6.6151 6.8650 6.8923 6.8964 6.8969 6.8970
6.8970 6.8970 "Edinburgh"
BLEU: 0.5853 0.4100 0.3013 0.2268 0.1730 0.1333 0.1037
0.0811 0.0637 "Edinburgh"
MT evaluation scorer ended on 2015 Jul 20 at 23:28:01
------------------------------
Message: 3
Date: Wed, 22 Jul 2015 16:22:39 +0400
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] Support for XML Markup with Confusion
Network Input
To: "James H. Cross III" <james.henry.cross.iii@gmail.com>,
moses-support@mit.edu
Message-ID: <55AF8B0F.3070602@gmail.com>
Content-Type: text/plain; charset=windows-1252; format=flowed
i guess lack of interest. XML markup is usually used by more
application-focused users who don't usually use complicated things like
confusion networks, and confusion networks are used mainly by
researchers who don't use xml markups
On 18/07/2015 00:35, James H. Cross III wrote:
> Hi:
>
> Is it still the case that XML markup is not supported for confusion
> network (or lattice) input? If not, are the reasons for not supporting
> this feature because implementing it imposes particular difficulties
> or simply due to lack of interest?
>
> Thanks,
> James
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
--
Hieu Hoang
Researcher
New York University, Abu Dhabi
http://www.hoang.co.uk/hieu
------------------------------
Message: 4
Date: Wed, 22 Jul 2015 16:28:14 +0400
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] BLEU result on baseline EMS experiment
To: Vincent Nguyen <vnguyen@neuf.fr>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAEKMkbhb6sQa5ke7HiE+4W_dJ89LZ9U2ON_aqJ-jiW98JCpkKA@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
it looks ok, your bleu score is 22.68 for this test set.
I don't know what you mean by real text.
Hieu Hoang
Researcher
New York University, Abu Dhabi
http://www.hoang.co.uk/hieu
On 21 July 2015 at 23:45, Vincent Nguyen <vnguyen@neuf.fr> wrote:
> here is what I got
>
> make sense ?
>
>
> MT evaluation scorer began on 2015 Jul 20 at 23:27:39
> command line: /home/moses/mosesdecoder/scripts/generic/mteval-v13a.pl -c
> -c -s /home/moses/working/data/dev/newstest2011-src.fr.sgm -r
> /home/moses/working/data/dev/newstest2011-ref.en.sgm -t
> /home/moses/working/evaluation/newstest2011.detokenized.sgm.3
> Evaluation of any-to-en translation using:
> src set "newstest2011" (110 docs, 3003 segs)
> ref set "newstest2011" (1 refs)
> tst set "newstest2011" (1 systems)
>
> length ratio: 0.994844739625875 (74296/74681), penalty (log):
> -0.00518197480348868
> NIST score = 6.8964 BLEU score = 0.2268 for system "Edinburgh"
>
> # ------------------------------------------------------------------------
>
> Individual N-gram scoring
> 1-gram 2-gram 3-gram 4-gram 5-gram 6-gram 7-gram
> 8-gram 9-gram
> ------ ------ ------ ------ ------ ------ ------
> ------ ------
> NIST: 5.2752 1.3399 0.2499 0.0273 0.0041 0.0005 0.0000
> 0.0000 0.0000 "Edinburgh"
>
> BLEU: 0.5883 0.2887 0.1636 0.0972 0.0589 0.0364 0.0230
> 0.0146 0.0093 "Edinburgh"
>
> # ------------------------------------------------------------------------
> Cumulative N-gram scoring
> 1-gram 2-gram 3-gram 4-gram 5-gram 6-gram 7-gram
> 8-gram 9-gram
> ------ ------ ------ ------ ------ ------ ------
> ------ ------
> NIST: 5.2752 6.6151 6.8650 6.8923 6.8964 6.8969 6.8970
> 6.8970 6.8970 "Edinburgh"
>
> BLEU: 0.5853 0.4100 0.3013 0.2268 0.1730 0.1333 0.1037
> 0.0811 0.0637 "Edinburgh"
> MT evaluation scorer ended on 2015 Jul 20 at 23:28:01
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150722/1b355d7a/attachment-0001.htm
------------------------------
Message: 5
Date: Wed, 22 Jul 2015 18:29:25 +0530
From: Saumitra Yadav <yadav.saumitra07@gmail.com>
Subject: [Moses-support] decoding-graph-backoff
To: moses-support@mit.edu
Message-ID:
<CAPeFGXYGFh_SVdwXKfbk600LmTyttgJ_ir-Ehy=bx4Q9nF4YpQ@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Sir/Ma'am,
I'm trying to use multiple phrase tables for translation in Moses decoder,
with preference to 1st phrase-table, but was getting a segmentation fault
while loading 1st phrase table, so just switched the positions of
phrase-tables in moses configuration file and it was working , now the
table i want to give preference is 2nd in list , can i use
[decoding-graph-backoff]
1
3
in configuration file for backoff so that moses uses 2nd table and uses 1st
table only for words it couldn't find in 2nd phrase-table?
Regards,
Saumitra Yadav
M.Tech.
Department Of Computer Science And Technology
Goa University
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150722/492d4a8e/attachment.htm
------------------------------
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
End of Moses-support Digest, Vol 105, Issue 43
**********************************************
Subscribe to:
Post Comments (Atom)
0 Response to "Moses-support Digest, Vol 105, Issue 43"
Post a Comment