Send Moses-support mailing list submissions to
moses-support@mit.edu
To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu
You can reach the person managing the list at
moses-support-owner@mit.edu
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."
Today's Topics:
1. Translator Model Parameter Clarification (Jer Yango)
2. Re: LMs for factors unused make the decoder fail (Hieu Hoang)
3. Re: Forbidden link to binaries (Per Tunedal)
4. Re: Translator Model Parameter Clarification (Hieu Hoang)
5. Re: Chinese segmentation/tokenization (Marcin Junczys-Dowmunt)
----------------------------------------------------------------------
Message: 1
Date: Sat, 21 Mar 2015 01:32:46 +0900
From: Jer Yango <yango.jer@gmail.com>
Subject: [Moses-support] Translator Model Parameter Clarification
To: moses-support@mit.edu
Message-ID:
<CAJHp5wRyeHAKH3csAFJmCAUFtz2-gfN77MqRXbWoE_vkODODOA@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Hi there,
As seen on the training references from the website, the --lm parameter
accepts three inputs: format, order, and filename.
- --lm -- language model: <factor>:<order>:<filename> (option can be
repeated)
But the baseline system uses 4 inputs:
-lm 0:3:$HOME/lm/news-commentary-v8.fr-en.blm.en:8
I would like to find out what the factor input is, also in identifying the
fourth input. Thanks!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150321/51e72ecb/attachment-0001.htm
------------------------------
Message: 2
Date: Fri, 20 Mar 2015 16:39:19 +0000
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] LMs for factors unused make the decoder
fail
To: Stanislav Ku??k <standa.kurik@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAEKMkbhoA5NuL4e-SrV8MsZ56WxvV_WXXiazOoosvZoDon_mxg@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
your output only has factor 0 and 2. So a LM over factor 1 or 3 will result
in a segfault
Hieu Hoang
Research Associate (until March 2015)
University of Edinburgh
http://www.hoang.co.uk/hieu
On 19 March 2015 at 08:01, Stanislav Ku??k <standa.kurik@gmail.com> wrote:
> Hello,
>
> when a train a model with 0,2-0,2 translation factors and I also attach
> a LMs for a different factor (1 or 3 in this case), running the decoder
> fails. Commenting these other LMs out in the INI file fixes this.
>
> It's not a critical issue, it just strikes me as odd that LMs which
> should not be used in the decoding process at all (yes, they are loaded,
> but they should not be consulted at all, if I am correct) make it fail.
>
> Regards,
> Stanislav K.
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150320/674becdb/attachment-0001.htm
------------------------------
Message: 3
Date: Fri, 20 Mar 2015 17:39:45 +0100
From: Per Tunedal <per.tunedal@operamail.com>
Subject: Re: [Moses-support] Forbidden link to binaries
To: Nikolay Bogoychev <nheart@gmail.com>
Cc: moses-support@mit.edu
Message-ID:
<1426869585.3234704.243094454.22C8656B@webmail.messagingengine.com>
Content-Type: text/plain; charset="us-ascii"
Hi Nick, Thank you! Yours, Per Tunedal
On Thu, Mar 19, 2015, at 15:46, Nikolay Bogoychev wrote:
> Hey Per,
>
> The link seems to be outdated, as it points to RELEASE-1.0. You can
> find the current ones here:
> http://www.statmt.org/moses/RELEASE-3.0/binaries/
>
> Cheers,
>
> Nick
>
> On Thu, Mar 19, 2015 at 2:22 PM, Per Tunedal
> <per.tunedal@operamail.com> wrote:
>> Hi,
>>
I just read the page http://www.statmt.org/moses/?n=Moses.Releases and
>>
tried the link to the binaries:
>>
>>
All the binary executables are made available for download for users who
>>
do not wish to compile their own version.
>>
>>
Clicking on download gets me to the page
>> http://www.statmt.org/moses/RELEASE-1.0/binaries/
>>
showing the message:
>>
>>
Forbidden
>>
>>
You don't have permission to access /moses/RELEASE-1.0/binaries/ on this
>>
server.
>>
>>
Yours,
>>
Per Tunedal
>>
_______________________________________________
>>
Moses-support mailing list
>> Moses-support@mit.edu
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150320/1249a6d9/attachment-0001.htm
------------------------------
Message: 4
Date: Fri, 20 Mar 2015 17:02:44 +0000
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] Translator Model Parameter Clarification
To: Jer Yango <yango.jer@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAEKMkbikiaUHnVzLRnk7dKoFsxNGM4XiXSzszqRG86sdhXbzVA@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
the 4th input (8) is the LM implementation you would like the decoder to
use. 0=SRILM, 1=IRSTLM, 8=KENLM.
factor (0) is the factor in the output sentence you want the LM to use. If
you don't use factors, then it's always 0
Hieu Hoang
Research Associate (until March 2015)
University of Edinburgh
http://www.hoang.co.uk/hieu
On 20 March 2015 at 16:32, Jer Yango <yango.jer@gmail.com> wrote:
> Hi there,
>
> As seen on the training references from the website, the --lm parameter
> accepts three inputs: format, order, and filename.
>
>
> - --lm -- language model: <factor>:<order>:<filename> (option can be
> repeated)
>
> But the baseline system uses 4 inputs:
>
> -lm 0:3:$HOME/lm/news-commentary-v8.fr-en.blm.en:8
>
>
> I would like to find out what the factor input is, also in identifying the
> fourth input. Thanks!
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150320/0359f1f0/attachment-0001.htm
------------------------------
Message: 5
Date: Fri, 20 Mar 2015 19:43:33 +0100
From: Marcin Junczys-Dowmunt <junczys@amu.edu.pl>
Subject: Re: [Moses-support] Chinese segmentation/tokenization
To: moses-support@mit.edu
Message-ID: <550C6A55.2010807@amu.edu.pl>
Content-Type: text/plain; charset="utf-8"
Hi all,
thank you all for the tips. I am going with Stanford then.
I am currently producing a language model from the Christian's raw
Chinese CommonCrawl data (www.statmt.org/ngrams). Once I am done I will
be happy to share back.
Best,
Marcin
W dniu 20.03.2015 o 15:43, Tom Hoar pisze:
> We also use the Stanford Segmenter most of the time, but have also
> used many. Surprisingly, LDC's manseg also gives very good results
> with SMT and it's much faster to load than Stanford's.
>
> Like Ventzi's comments, a segmenter's absolute accuracy relative to
> the human interpretation of what is a "word" is not the most important
> factor when using it as a tokenizer for SMT. It's much more important
> for the tool to give consistent co-occurrence results relative to the
> paired language tokens. In "from ZH" environments, the
> segmented/tokenized form is never seen by humans. In "to ZH"
> environments, the recaser/detokenizer method(s) can actually repair
> errors and restore the string to what it should be.
>
> @ Venzi, thanks for mentioning KeTea. We'll test & compare.
>
> Tom
>
>
>
> On 03/20/2015 08:43 PM, "????????? ????? (Ventsislav Zhechev)" wrote:
>> Hi Marcin,
>>
>> At Autodesk we?ve been successfully using KyTea since 2011. The main
>> reason we chose this specific tool is that it has readily available
>> models for both Chinese and Japanese, which simplified the
>> integration in our workflows.
>> At least for Japanese, we also evaluated Mecab in 2011, but found
>> KyTea to serve us better.
>>
>> Keep in mind, though, that we are not very interested in the quality
>> of the segmentation per se; instead we need the MT to be of
>> sufficient quality, regardless if what the segmentation tool does
>> makes sense on its own or not.
>>
>>
>> Cheers,
>>
>> Ventzi
>>
>> ???????
>> Dr. Ventsislav Zhechev
>> Computational Linguist, Certified ScrumMaster?
>> Platform Architecture and Technologies
>> Localisation Services
>>
>> MAIN +41 32 723 91 22
>> FAX +41 32 723 93 99
>>
>> http://VentsislavZhechev.eu
>>
>> Autodesk, Inc.
>> Rue de Puits-Godet 6
>> 2000 Neuch?tel, Switzerland
>> www.autodesk.com
>>
>>
>>
>>
>>> 20.03.2015 ?., ? 14:32, moses-support-request@mit.edu
>>> <mailto:moses-support-request@mit.edu> ???????(?):
>>>
>>> Date: Fri, 20 Mar 2015 13:19:02 +0100
>>> From: Marcin Junczys-Dowmunt <junczys@amu.edu.pl
>>> <mailto:junczys@amu.edu.pl>>
>>> Subject: [Moses-support] Chinese segmentation/tokenization
>>> To: Moses Support <moses-support@mit.edu>
>>> Message-ID: <e4d171cb90994cb853a9965facaebc37@amu.edu.pl>
>>> Content-Type: text/plain; charset="us-ascii"
>>>
>>>
>>>
>>> Hi,
>>>
>>> questions appear from time to time on the list concerning Chinese
>>> segmentation/tokenization. I saw Barry mention Lingpipe and other tools.
>>> Is there a favourite tool you guys prefer to use over others?
>>>
>>> Thanks,
>>>
>>> Marcin
>>
>>
>>
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150320/1c38fd85/attachment.htm
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/jpeg
Size: 14277 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20150320/1c38fd85/attachment.jpg
------------------------------
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
End of Moses-support Digest, Vol 101, Issue 61
**********************************************
Subscribe to:
Post Comments (Atom)
0 Response to "Moses-support Digest, Vol 101, Issue 61"
Post a Comment