Home » Uncategories » Moses-support Digest, Vol 100, Issue 22

Moses-support Digest, Vol 100, Issue 22

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. SDL Language Technologies Summer Intern Programme 2015
(Bill Byrne)
2. Re: Using boost for prefix/suffix checks (Jeroen Vermeulen)
3. Fails exception: 'vector::_M_range_check' when I run on moses
to decoder (=?gb2312?B?uLax8g==?=)
4. Re: Fails exception: 'vector::_M_range_check' when I run on
moses to decoder (Tom Hoar)


----------------------------------------------------------------------

Message: 1
Date: Thu, 5 Feb 2015 17:39:05 +0000
From: Bill Byrne <bbyrne@sdl.com>
Subject: [Moses-support] SDL Language Technologies Summer Intern
Programme 2015
To: "moses-support@mit.edu" <moses-support@mit.edu>
Cc: Bill Byrne <bbyrne@sdl.com>
Message-ID: <554EA300-DE56-452C-B85C-84F81B6EB154@sdl.com>
Content-Type: text/plain; charset="us-ascii"

SDL Language Technologies is looking for outstanding PhD students for summer internships with our research teams in Los Angeles (USA) and Cambridge (UK). Interns will work one-on-one with a mentor on a three-month NLP research project.

Our projects are centred around machine translation and natural language processing. The exact nature of the project will depend on the interests and background of the intern. The objective will be novel work of commercial value, and the expectation is that successful projects will lead to a paper submission. Examples of recent projects are:

- Target dependency language models in phrase-based decoding
- Source-side preordering for translation using logistic regression and depth-first branch-and-bound search
- Translating into morphologically rich languages
- Online adaptation of MT
- Predicting MT customer ratings

If interested, please send an email to mt-jobs@sdl.com by February 15, 2015, with "2015 Internship Application -- *your name*" in the subject. Please include your CV and a statement that describes your research interests and experience. U.S. citizenship / EU passport is not required, but please indicate your preference for working in Los Angeles or in Cambridge.

SDL Language Technologies (previously: Language Weaver) is the first company to bring a statistical translation system to market. Its R&D team solves large-scale NLP problems with the goal of increasing MT's commercial impact

Bill Byrne | Senior Research Scientist, Director UK R&D Office | SDL Research | (m) +44 (0)7852910371


SDL PLC confidential, all rights reserved.
If you are not the intended recipient of this mail SDL requests and requires that you delete it without acting upon or copying any of its contents, and we further request that you advise us.
SDL PLC is a public limited company registered in England and Wales. Registered number: 02675207.
Registered address: Globe House, Clivemont Road, Maidenhead, Berkshire SL6 7DY, UK.



This message has been scanned for malware by Websense. www.websense.com



------------------------------

Message: 2
Date: Fri, 06 Feb 2015 00:56:33 +0100
From: Jeroen Vermeulen <jtv@precisiontranslationtools.com>
Subject: Re: [Moses-support] Using boost for prefix/suffix checks
To: Hieu Hoang <Hieu.Hoang@ed.ac.uk>
Cc: moses-support <moses-support@mit.edu>
Message-ID: <54D40331.2060903@precisiontranslationtools.com>
Content-Type: text/plain; charset="utf-8"

On 2015-02-05 17:25, Hieu Hoang wrote:
> great, thanks. committed
>
> https://github.com/moses-smt/mosesdecoder/commit/70e8eb54ce75feb0a7d4ed00d275e56652c0a914
> I ran the regression tests before I commit, it's more extensive

That was fast - thanks! I'll make it a habit to run the regression tests.

I did find a spot that I missed:
LexicalReorderingTableCompact::CheckAndLoad().

That function's contract looks a little confused, by the way. It
handles both the case where the given file name ends in ".minlexr" and
the case where that suffix still needs to be added. But the only call
site always adds that suffix first, so as far as I can tell CheckAndLoad
ends up looking for "x.minlexr.minlexr" before it tries "x.minlexr".

I *think* the attached patch would iron out that wrinkle, but I'm having
some trouble actually exercising the code. Maybe it's not worth it just
now. :/


Jeroen
-------------- next part --------------
A non-text attachment was scrubbed...
Name: check_and_load.diff
Type: text/x-patch
Size: 1722 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20150206/b109789d/attachment-0001.bin

------------------------------

Message: 3
Date: Fri, 6 Feb 2015 14:50:33 +0800
From: =?gb2312?B?uLax8g==?= <fubinfri@163.com>
Subject: [Moses-support] Fails exception: 'vector::_M_range_check'
when I run on moses to decoder
To: <Moses-support@mit.edu>
Message-ID: <000d01d041d9$3509fca0$9f1df5e0$@163.com>
Content-Type: text/plain; charset="gb2312"

Hi everyone:

I have a problem when I use the moses to decoder. I followed
http://www.statmt.org/moses/?n=Moses.Baseline to build up baseline. In
addition, I had edited the moses.ini and changed the path of phrase-table
and reordering-table without its suffix, but it didn??t work. I had done all
the solution that mentioned on
https://www.mail-archive.com/moses-support@mit.edu/msg11543.html, but the
problem is still existed. I??m very confused.

Here is my test command and failed result:

fubin@nlp15:~/moses$ mosesdecoder/bin/moses -f work/compact/moses.ini

Defined parameters (per moses.ini or switch):

config: work/compact/moses.ini

distortion-limit: 6

feature: UnknownWordPenalty WordPenalty PhrasePenalty
PhraseDictionaryCompact name=TranslationModel0 table-limit=20 num-features=4
path=/home/fubin/moses/work/compact/phrase-table input-factor=0
output-factor=0 LexicalReordering name=LexicalReordering0 num-features=6
type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0
path=/home/fubin/moses/work/compact/reordering-table Distortion KENLM
lazyken=0 name=LM0 factor=0 path=/home/fubin/moses/lm/europarl.fr-en.blm.en
order=3

input-factors: 0

mapping: 0 T 0

threads: 8

weight: LexicalReordering0= 0.0731976 0.0443935 0.131823 0.197196 0.
0838931 0.090232 Distortion0= 0.015492 LM0= 0.0688113 WordPenalty0=
-0.0638317 PhrasePenalty0= 0.0644346 TranslationModel0= 0.0220868 0.05336 0.
0751734 0.016075 UnknownWordPenalty0= 1

line=UnknownWordPenalty

FeatureFunction: UnknownWordPenalty0 start: 0 end: 0

line=WordPenalty

FeatureFunction: WordPenalty0 start: 1 end: 1

line=PhrasePenalty

FeatureFunction: PhrasePenalty0 start: 2 end: 2

line=PhraseDictionaryCompact name=TranslationModel0 table-limit=20
num-features=4 path=/home/fubin/moses/work/compact/phrase-table
input-factor=0 output-factor=0

FeatureFunction: TranslationModel0 start: 3 end: 6

line=LexicalReordering name=LexicalReordering0 num-features=6
type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0
path=/home/fubin/moses/work/compact/reordering-table

FeatureFunction: LexicalReordering0 start: 7 end: 12

Initializing LexicalReordering..

line=Distortion

FeatureFunction: Distortion0 start: 13 end: 13

line=KENLM lazyken=0 name=LM0 factor=0
path=/home/fubin/moses/lm/europarl.fr-en.blm.en order=3

FeatureFunction: LM0 start: 14 end: 14

Loading UnknownWordPenalty0

Loading WordPenalty0

Loading PhrasePenalty0

Loading LexicalReordering0

Loading table into memory...Exception: vector::_M_range_check



Best regards

Bingo

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150206/f513e49a/attachment-0001.htm
-------------- next part --------------
A non-text attachment was scrubbed...
Name: moses.ini
Type: application/octet-stream
Size: 1210 bytes
Desc: not available
Url : http://mailman.mit.edu/mailman/private/moses-support/attachments/20150206/f513e49a/attachment-0001.obj

------------------------------

Message: 4
Date: Fri, 06 Feb 2015 14:07:17 +0700
From: Tom Hoar <tahoar@precisiontranslationtools.com>
Subject: Re: [Moses-support] Fails exception: 'vector::_M_range_check'
when I run on moses to decoder
To: moses-support@mit.edu
Message-ID: <54D46825.8000308@precisiontranslationtools.com>
Content-Type: text/plain; charset="utf-8"

Hi Bingo.

No need for bold. :)

I think I experimented with the same as you and got the _M_range_check
due to the PhraseDictionaryCompact, not LexicalReordering.

Try with basename's extension here:
PhraseDictionaryCompact ...
path=/home/fubin/moses/work/compact/phrase-table.minphr

and without basename's extension here:
LexicalReordering ...
path=/home/fubin/moses/work/compact/reordering-table

Also, what version of Boost are you using?




On 02/06/2015 01:50 PM, ?? wrote:
>
> Hi everyone:
>
> * I have a problem when I use the moses to decoder. I followed
> http://www.statmt.org/moses/?n=Moses.Baseline to build up baseline. In
> addition, I had edited the moses.ini and changed the path of
> phrase-table and reordering-table without its suffix, but it didn?t
> work. I had done all the solution that mentioned on
> https://www.mail-archive.com/moses-support@mit.edu/msg11543.html, but
> the problem is still existed. I?m very confused.*
>
> * Here is my test command and failed result:*
>
> fubin@nlp15:~/moses$ mosesdecoder/bin/moses -f work/compact/moses.ini
>
> Defined parameters (per moses.ini or switch):
>
> config: work/compact/moses.ini
>
> distortion-limit: 6
>
> feature: UnknownWordPenalty WordPenalty PhrasePenalty
> PhraseDictionaryCompact name=TranslationModel0 table-limit=20
> num-features=4 path=/home/fubin/moses/work/compact/phrase-table
> input-factor=0 output-factor=0 LexicalReordering
> name=LexicalReordering0 num-features=6
> type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0
> path=/home/fubin/moses/work/compact/reordering-table Distortion KENLM
> lazyken=0 name=LM0 factor=0
> path=/home/fubin/moses/lm/europarl.fr-en.blm.en order=3
>
> input-factors: 0
>
> mapping: 0 T 0
>
> threads: 8
>
> weight: LexicalReordering0= 0.0731976 0.0443935 0.131823
> 0.197196 0.0838931 0.090232 Distortion0= 0.015492 LM0= 0.0688113
> WordPenalty0= -0.0638317 PhrasePenalty0= 0.0644346 TranslationModel0=
> 0.0220868 0.05336 0.0751734 0.016075 UnknownWordPenalty0= 1
>
> line=UnknownWordPenalty
>
> FeatureFunction: UnknownWordPenalty0 start: 0 end: 0
>
> line=WordPenalty
>
> FeatureFunction: WordPenalty0 start: 1 end: 1
>
> line=PhrasePenalty
>
> FeatureFunction: PhrasePenalty0 start: 2 end: 2
>
> line=PhraseDictionaryCompact name=TranslationModel0 table-limit=20
> num-features=4 path=/home/fubin/moses/work/compact/phrase-table
> input-factor=0 output-factor=0
>
> FeatureFunction: TranslationModel0 start: 3 end: 6
>
> line=LexicalReordering name=LexicalReordering0 num-features=6
> type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0
> path=/home/fubin/moses/work/compact/reordering-table
>
> FeatureFunction: LexicalReordering0 start: 7 end: 12
>
> Initializing LexicalReordering..
>
> line=Distortion
>
> FeatureFunction: Distortion0 start: 13 end: 13
>
> line=KENLM lazyken=0 name=LM0 factor=0
> path=/home/fubin/moses/lm/europarl.fr-en.blm.en order=3
>
> FeatureFunction: LM0 start: 14 end: 14
>
> Loading UnknownWordPenalty0
>
> Loading WordPenalty0
>
> Loading PhrasePenalty0
>
> Loading LexicalReordering0
>
> Loading table into memory...Exception: vector::_M_range_check
>
> Best regards
>
> Bingo
>
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150206/34a0a451/attachment.htm

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 100, Issue 22
**********************************************

0 Response to "Moses-support Digest, Vol 100, Issue 22"

Post a Comment

Subscribe to: Post Comments (Atom)