Send Moses-support mailing list submissions to
moses-support@mit.edu
To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu
You can reach the person managing the list at
moses-support-owner@mit.edu
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."
Today's Topics:
1. Re: Sparse phrase table, is still supported? (Matthias Huck)
2. Re: Sparse phrase table, is still supported? (jian zhang)
3. Re: Sparse phrase table, is still supported? (Matthias Huck)
4. Re: Moses build failed (Hieu Hoang)
----------------------------------------------------------------------
Message: 1
Date: Thu, 16 Jul 2015 02:06:23 +0100
From: Matthias Huck <mhuck@inf.ed.ac.uk>
Subject: Re: [Moses-support] Sparse phrase table, is still supported?
To: jian zhang <zhangj@computing.dcu.ie>
Cc: moses-support@mit.edu
Message-ID: <1437008783.2140.34.camel@inf.ed.ac.uk>
Content-Type: text/plain; charset="UTF-8"
Hi,
Are you planning to use binary domain indicator features? I'm not sure
whether a sparse feature function for this is currently implemented. If
you're working with a small set of domains, you can employ dense
indicators instead (domain-features = "indicator" in EMS). You'll have
to re-extract the phrase table, though. Or process it with a script to
add dense indicator values to the scores field.
I believe that there might also be some bug in the extraction pipeline
when both domain-features = "sparse indicator" and score-settings =
"--GoodTuring" are active in EMS. At least it caused me trouble a couple
of weeks ago. However, I must admit that I didn't investigate it further
at that point.
Anyway, the bottom line is that I recommend re-extracting with dense
indicators.
But let me know what you find regarding a sparse implementation.
Cheers,
Matthias
On Thu, 2015-07-16 at 00:48 +0100, jian zhang wrote:
> Hi,
>
>
> Is the sparse features at phrase table, like
>
>
>
> das Haus ||| the house ||| 0.8 0.5 0.8 0.5 2.718 ||| 0-0 1-1 ||| 5000
> 5000 2500 ||| dom_europarl 1
>
>
>
> still supported? If yes, what should I set to the ini file based on
> the example above?
>
>
> Thank,
>
>
> Jian
>
>
> --
> Jian Zhang
> Centre for Next Generation Localisation (CNGL)
> Dublin City University
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
--
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.
------------------------------
Message: 2
Date: Thu, 16 Jul 2015 02:18:58 +0100
From: jian zhang <zhangj@computing.dcu.ie>
Subject: Re: [Moses-support] Sparse phrase table, is still supported?
To: Matthias Huck <mhuck@inf.ed.ac.uk>
Cc: moses-support@mit.edu
Message-ID:
<CALA=z0CVT_BzdNYv42gnuEe6ms=f7xJFOOzw_=9yHtsBF-6PCA@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Hi Matthias,
Not for domain feature.
I want to implement some sparse features, so there are two options:
1, add to phrase table, if it is supported
2, implement sparse feature functions,
I'd like to know are there any difference between these two options, for
example, tuning, compute sentence translation scores ...
Regards,
Jian
On Thu, Jul 16, 2015 at 2:06 AM, Matthias Huck <mhuck@inf.ed.ac.uk> wrote:
> Hi,
>
> Are you planning to use binary domain indicator features? I'm not sure
> whether a sparse feature function for this is currently implemented. If
> you're working with a small set of domains, you can employ dense
> indicators instead (domain-features = "indicator" in EMS). You'll have
> to re-extract the phrase table, though. Or process it with a script to
> add dense indicator values to the scores field.
>
> I believe that there might also be some bug in the extraction pipeline
> when both domain-features = "sparse indicator" and score-settings =
> "--GoodTuring" are active in EMS. At least it caused me trouble a couple
> of weeks ago. However, I must admit that I didn't investigate it further
> at that point.
>
> Anyway, the bottom line is that I recommend re-extracting with dense
> indicators.
>
> But let me know what you find regarding a sparse implementation.
>
> Cheers,
> Matthias
>
>
> On Thu, 2015-07-16 at 00:48 +0100, jian zhang wrote:
> > Hi,
> >
> >
> > Is the sparse features at phrase table, like
> >
> >
> >
> > das Haus ||| the house ||| 0.8 0.5 0.8 0.5 2.718 ||| 0-0 1-1 ||| 5000
> > 5000 2500 ||| dom_europarl 1
> >
> >
> >
> > still supported? If yes, what should I set to the ini file based on
> > the example above?
> >
> >
> > Thank,
> >
> >
> > Jian
> >
> >
> > --
> > Jian Zhang
> > Centre for Next Generation Localisation (CNGL)
> > Dublin City University
> > _______________________________________________
> > Moses-support mailing list
> > Moses-support@mit.edu
> > http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
>
> --
> The University of Edinburgh is a charitable body, registered in
> Scotland, with registration number SC005336.
>
>
--
Jian Zhang
Centre for Next Generation Localisation (CNGL)
<http://www.cngl.ie/index.html>
Dublin City University <http://www.dcu.ie/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150716/301749d1/attachment-0001.htm
------------------------------
Message: 3
Date: Thu, 16 Jul 2015 03:23:50 +0100
From: Matthias Huck <mhuck@inf.ed.ac.uk>
Subject: Re: [Moses-support] Sparse phrase table, is still supported?
To: jian zhang <zhangj@computing.dcu.ie>
Cc: moses-support@mit.edu
Message-ID: <1437013430.2140.70.camel@inf.ed.ac.uk>
Content-Type: text/plain; charset="UTF-8"
Hi Jian,
That depends on the nature of the features you're planning to
implement.
In order to produce sparse features, you need to write a feature
function anyway.
But if it's only a handful of scores and they can be calculated during
extraction time, then go for dense features and add the scores directly
to the phrase table.
If the scores cannot be precalculated, for instance because you need
non-local information that is only available during decoding, then a
feature function implementation becomes necessary.
When you write a feature function that calculates scores during decoding
time, it can produce dense scores, sparse scores, or both types. That's
up to you.
If it's plenty of scores which are fired rarely, then sparse is the
right choice. And you certainly need a sparse feature function
implementation in case you are not aware in advance of the overall
amount of feature scores it can produce.
If you need information from phrase extraction in order to calculate
scores during decoding time, then we have something denoted as "phrase
properties". Phrase properties give you a means of storing arbitrary
additional information in the phrase table. You have to extend the
extraction pipeline to retrieve and store the phrase properties you
require. The decoder can later read this information from the phrase
table, and your feature function can utilize it in some way.
A large amount of sparse feature scores can somewhat slow down decoding
and tuning. Also, you have to use MIRA or PRO for tuning, not MERT.
Cheers,
Matthias
On Thu, 2015-07-16 at 02:18 +0100, jian zhang wrote:
> Hi Matthias,
>
>
> Not for domain feature.
>
>
> I want to implement some sparse features, so there are two options:
> 1, add to phrase table, if it is supported
> 2, implement sparse feature functions,
>
>
> I'd like to know are there any difference between these two options,
> for example, tuning, compute sentence translation scores ...
>
>
> Regards,
>
>
>
> Jian
>
>
>
> On Thu, Jul 16, 2015 at 2:06 AM, Matthias Huck <mhuck@inf.ed.ac.uk>
> wrote:
> Hi,
>
> Are you planning to use binary domain indicator features? I'm
> not sure
> whether a sparse feature function for this is currently
> implemented. If
> you're working with a small set of domains, you can employ
> dense
> indicators instead (domain-features = "indicator" in EMS).
> You'll have
> to re-extract the phrase table, though. Or process it with a
> script to
> add dense indicator values to the scores field.
>
> I believe that there might also be some bug in the extraction
> pipeline
> when both domain-features = "sparse indicator" and
> score-settings =
> "--GoodTuring" are active in EMS. At least it caused me
> trouble a couple
> of weeks ago. However, I must admit that I didn't investigate
> it further
> at that point.
>
> Anyway, the bottom line is that I recommend re-extracting with
> dense
> indicators.
>
> But let me know what you find regarding a sparse
> implementation.
>
> Cheers,
> Matthias
>
>
> On Thu, 2015-07-16 at 00:48 +0100, jian zhang wrote:
> > Hi,
> >
> >
> > Is the sparse features at phrase table, like
> >
> >
> >
> > das Haus ||| the house ||| 0.8 0.5 0.8 0.5 2.718 ||| 0-0 1-1
> ||| 5000
> > 5000 2500 ||| dom_europarl 1
> >
> >
> >
> > still supported? If yes, what should I set to the ini file
> based on
> > the example above?
> >
> >
> > Thank,
> >
> >
> > Jian
> >
> >
> > --
> > Jian Zhang
> > Centre for Next Generation Localisation (CNGL)
> > Dublin City University
>
> > _______________________________________________
> > Moses-support mailing list
> > Moses-support@mit.edu
> > http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
>
> --
> The University of Edinburgh is a charitable body, registered
> in
> Scotland, with registration number SC005336.
>
>
>
>
>
> --
> Jian Zhang
> Centre for Next Generation Localisation (CNGL)
> Dublin City University
--
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.
------------------------------
Message: 4
Date: Thu, 16 Jul 2015 09:24:03 +0400
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] Moses build failed
To: Rico Sennrich <rico.sennrich@gmx.ch>, msanguin@di.unito.it
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<CAEKMkbhROwvj6PYpu+_0j51X3z0zbegyFThGL77dSV6-d_e47g@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
i just compiled on Windows/cygwin ok. Unfortunately, you do need to be
quite experienced with computer science and linux to get very far with
moses.
The company, Precision Translation Tools, have adapted Moses specifically
for Windows (without cygwin).
http://www.precisiontranslationtools.com/smt-for-windows-and-more/
Perhaps thats a little more user-friendly
Hieu Hoang
Researcher
New York University, Abu Dhabi
http://www.hoang.co.uk/hieu
On 15 July 2015 at 20:30, Rico Sennrich <rico.sennrich@gmx.ch> wrote:
> Hi Manuela,
>
> I don't know enough about cygwin to know how the installation of the
> dependencies could have gone wrong.
>
> Can someone who uses cygwin help?
>
> best wishes,
> Rico
>
> On 15.07.2015 14:56, Manuela Sanguinetti wrote:
>
> Hi Rico,
>
> first of all, thanks for your feedback.
> I first installed all the dependencies listed on the Moses page by running
> the Cygwin 'set_up-x86_64' executable file. However, checking if everything
> was actually there (using the command "which ..."), it appeared that for a
> large number of those packages this wasn't the case. So I've just tried to
> install them using the command "apt-cyg install ...", but I got the
> message "Package ... is already installed, skipping"!
> I guess that all these "missing" dependecies should then be properly
> linked, by setting some path or environment variable ... but unfortunately
> this goes far beyond my competences in computer science :( so now I'm
> totally clueless as to what to do.
>
> Thank you anyway,
> Manuela
>
>
>
> Il 15/07/2015 15:26, Rico Sennrich ha scritto:
>
> Hello Manuela,
>
> have you installed all the dependencies listed here for Cygwin
> installation? http://www.statmt.org/moses/?n=Development.GetStarted
>
> it seems like you're missing a system-wide installation of boost.
> mosesdecoder/bjam currently isn't set-up to use the boost-jam of a local
> boost installation (even if you pass a "--with-boost" argument).
>
> best wishes,
> Rico
>
>
> On 15.07.2015 12:19, Manuela Sanguinetti wrote:
>
> Hi,
>
> I'm experiencing some problems in compiling Moses on Windows using Cygwin.
> I manually installed Boost (v. 1.58), but when I run the command
>
> ~/mosesdecoder
> $ ./bjam --with-boost=~/boost_1_58_0/ -j4
>
> the build fails.
> I'm attaching the 'build.log.gz' file, as suggested in the error message,
> hoping that someone could help me figuring out what the problem is and how
> I can fix it.
>
>
> Thank you in advance for any hint you may provide,
> MS
>
>
> _______________________________________________
> Moses-support mailing listMoses-support@mit.eduhttp://mailman.mit.edu/mailman/listinfo/moses-support
>
>
>
>
> _______________________________________________
> Moses-support mailing listMoses-support@mit.eduhttp://mailman.mit.edu/mailman/listinfo/moses-support
>
>
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150716/f76e23af/attachment.htm
------------------------------
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
End of Moses-support Digest, Vol 105, Issue 31
**********************************************
Subscribe to:
Post Comments (Atom)
0 Response to "Moses-support Digest, Vol 105, Issue 31"
Post a Comment