Moses-support Digest, Vol 105, Issue 34

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: Moses build failed (Hieu Hoang)
2. Re: Sparse phrase table, is still supported? (Philipp Koehn)


----------------------------------------------------------------------

Message: 1
Date: Thu, 16 Jul 2015 22:50:02 +0400
From: Hieu Hoang <hieuhoang@gmail.com>
Subject: Re: [Moses-support] Moses build failed
To: Manuela Sanguinetti <msanguin@di.unito.it>, Rico Sennrich
<rico.sennrich@gmx.ch>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID: <55A7FCDA.7010908@gmail.com>
Content-Type: text/plain; charset="utf-8"

i install the dependency with the setup.exe gui. But I don't install
boost. Instead, I compile boost myself using the instructions found here:
http://www.statmt.org/moses/?n=Development.GetStarted


On 16/07/2015 11:58, Manuela Sanguinetti wrote:
> Thank you,
>
> I'll try with this one.
>
> Just out of curiosity ... how did you install the required dependecies
> on cygwin? Did you use the command line?
>
>
> best wishes,
> manuela
>
> Il 16/07/2015 07:24, Hieu Hoang ha scritto:
>> i just compiled on Windows/cygwin ok. Unfortunately, you do need to
>> be quite experienced with computer science and linux to get very far
>> with moses.
>>
>> The company, Precision Translation Tools, have adapted Moses
>> specifically for Windows (without cygwin).
>> http://www.precisiontranslationtools.com/smt-for-windows-and-more/
>> Perhaps thats a little more user-friendly
>>
>> Hieu Hoang
>> Researcher
>> New York University, Abu Dhabi
>> http://www.hoang.co.uk/hieu
>>
>> On 15 July 2015 at 20:30, Rico Sennrich <rico.sennrich@gmx.ch
>> <mailto:rico.sennrich@gmx.ch>> wrote:
>>
>> Hi Manuela,
>>
>> I don't know enough about cygwin to know how the installation of
>> the dependencies could have gone wrong.
>>
>> Can someone who uses cygwin help?
>>
>> best wishes,
>> Rico
>>
>> On 15.07.2015 14:56, Manuela Sanguinetti wrote:
>>> Hi Rico,
>>>
>>> first of all, thanks for your feedback.
>>> I first installed all the dependencies listed on the Moses page
>>> by running the Cygwin 'set_up-x86_64' executable file. However,
>>> checking if everything was actually there (using the command
>>> "which ..."), it appeared that for a large number of those
>>> packages this wasn't the case. So I've just tried to install
>>> them using the command "apt-cyg install ...", but I got the
>>> message "Package ... is already installed, skipping"!
>>> I guess that all these "missing" dependecies should then be
>>> properly linked, by setting some path or environment variable
>>> ... but unfortunately this goes far beyond my competences in
>>> computer science :( so now I'm totally clueless as to what to do.
>>>
>>> Thank you anyway,
>>> Manuela
>>>
>>>
>>>
>>> Il 15/07/2015 15:26, Rico Sennrich ha scritto:
>>>> Hello Manuela,
>>>>
>>>> have you installed all the dependencies listed here for Cygwin
>>>> installation? http://www.statmt.org/moses/?n=Development.GetStarted
>>>>
>>>> it seems like you're missing a system-wide installation of
>>>> boost. mosesdecoder/bjam currently isn't set-up to use the
>>>> boost-jam of a local boost installation (even if you pass a
>>>> "--with-boost" argument).
>>>>
>>>> best wishes,
>>>> Rico
>>>>
>>>>
>>>> On 15.07.2015 12:19, Manuela Sanguinetti wrote:
>>>>> Hi,
>>>>>
>>>>> I'm experiencing some problems in compiling Moses on Windows
>>>>> using Cygwin. I manually installed Boost (v. 1.58), but when I
>>>>> run the command
>>>>>
>>>>> ~/mosesdecoder
>>>>> $ ./bjam --with-boost=~/boost_1_58_0/ -j4
>>>>>
>>>>> the build fails.
>>>>> I'm attaching the 'build.log.gz' file, as suggested in the
>>>>> error message, hoping that someone could help me figuring out
>>>>> what the problem is and how I can fix it.
>>>>>
>>>>>
>>>>> Thank you in advance for any hint you may provide,
>>>>> MS
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Moses-support mailing list
>>>>> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
>>>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Moses-support mailing list
>>>> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
>>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>
>>
>>
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>
>>
>

--
Hieu Hoang
Researcher
New York University, Abu Dhabi
http://www.hoang.co.uk/hieu

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20150716/db8bcdee/attachment-0001.htm

------------------------------

Message: 2
Date: Fri, 17 Jul 2015 09:25:03 +0700
From: Philipp Koehn <phi@jhu.edu>
Subject: Re: [Moses-support] Sparse phrase table, is still supported?
To: Matthias Huck <mhuck@inf.ed.ac.uk>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>, jian zhang
<zhangj@computing.dcu.ie>
Message-ID:
<CAAFADDDxKDk63AX-8tSz8CxFSqtXs1sNiJVCm+yxpZdAOdy2ug@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8

Hi,

I have not a clear picture what phrase table implementations support
sparse features. Until recently PhraseTableBin did, but PhraseTableCompact
did not. Not sure, if things changed either way.

-phi

On Thu, Jul 16, 2015 at 9:42 PM, Matthias Huck <mhuck@inf.ed.ac.uk> wrote:
> Hi,
>
> You're right, I claimed in the previous mail that "in order to produce
> sparse features, you need to write a feature function anyway" and this
> is of course not true if you get the sparse phrase table features to
> work.
>
> When I tried those sparse domain indicators recently, they didn't work
> out of the box, and I also don't know where to find the relevant code.
> My guess is that this functionality was broken during the course of
> Moses refactoring, but it may as well still be there and waiting to be
> activated in the moses.ini. What I did was just switching to dense
> domain indicators.
>
> Maybe Hieu can help?
>
> Cheers,
> Matthias
>
>
> On Thu, 2015-07-16 at 10:03 +0100, jian zhang wrote:
>> Hi Matthias,
>>
>>
>> Thanks for the information.
>>
>>
>> I tested on moses 3.0, adding phrase table sparse feature is seems
>> working.
>>
>>
>> However, I did not add any flag into ini, like suggested "If a phrase
>> table contains sparse features, then this needs to be flagged in the
>> configuration file by adding the word sparse after the phrase table
>> file name.". Did i miss anything?
>>
>>
>> Regards,
>>
>>
>> Jian
>>
>>
>>
>>
>>
>>
>>
>> On Thu, Jul 16, 2015 at 3:23 AM, Matthias Huck <mhuck@inf.ed.ac.uk>
>> wrote:
>> Hi Jian,
>>
>> That depends on the nature of the features you're planning to
>> implement.
>>
>> In order to produce sparse features, you need to write a
>> feature
>> function anyway.
>>
>> But if it's only a handful of scores and they can be
>> calculated during
>> extraction time, then go for dense features and add the scores
>> directly
>> to the phrase table.
>>
>> If the scores cannot be precalculated, for instance because
>> you need
>> non-local information that is only available during decoding,
>> then a
>> feature function implementation becomes necessary.
>>
>> When you write a feature function that calculates scores
>> during decoding
>> time, it can produce dense scores, sparse scores, or both
>> types. That's
>> up to you.
>>
>> If it's plenty of scores which are fired rarely, then sparse
>> is the
>> right choice. And you certainly need a sparse feature function
>> implementation in case you are not aware in advance of the
>> overall
>> amount of feature scores it can produce.
>>
>> If you need information from phrase extraction in order to
>> calculate
>> scores during decoding time, then we have something denoted as
>> "phrase
>> properties". Phrase properties give you a means of storing
>> arbitrary
>> additional information in the phrase table. You have to extend
>> the
>> extraction pipeline to retrieve and store the phrase
>> properties you
>> require. The decoder can later read this information from the
>> phrase
>> table, and your feature function can utilize it in some way.
>>
>> A large amount of sparse feature scores can somewhat slow down
>> decoding
>> and tuning. Also, you have to use MIRA or PRO for tuning, not
>> MERT.
>>
>> Cheers,
>> Matthias
>>
>>
>> On Thu, 2015-07-16 at 02:18 +0100, jian zhang wrote:
>> > Hi Matthias,
>> >
>> >
>> > Not for domain feature.
>> >
>> >
>> > I want to implement some sparse features, so there are two
>> options:
>> > 1, add to phrase table, if it is supported
>> > 2, implement sparse feature functions,
>> >
>> >
>> > I'd like to know are there any difference between these two
>> options,
>> > for example, tuning, compute sentence translation scores ...
>> >
>> >
>> > Regards,
>> >
>> >
>> >
>> > Jian
>> >
>> >
>> >
>> > On Thu, Jul 16, 2015 at 2:06 AM, Matthias Huck
>> <mhuck@inf.ed.ac.uk>
>> > wrote:
>> > Hi,
>> >
>> > Are you planning to use binary domain indicator
>> features? I'm
>> > not sure
>> > whether a sparse feature function for this is
>> currently
>> > implemented. If
>> > you're working with a small set of domains, you can
>> employ
>> > dense
>> > indicators instead (domain-features = "indicator" in
>> EMS).
>> > You'll have
>> > to re-extract the phrase table, though. Or process
>> it with a
>> > script to
>> > add dense indicator values to the scores field.
>> >
>> > I believe that there might also be some bug in the
>> extraction
>> > pipeline
>> > when both domain-features = "sparse indicator" and
>> > score-settings =
>> > "--GoodTuring" are active in EMS. At least it caused
>> me
>> > trouble a couple
>> > of weeks ago. However, I must admit that I didn't
>> investigate
>> > it further
>> > at that point.
>> >
>> > Anyway, the bottom line is that I recommend
>> re-extracting with
>> > dense
>> > indicators.
>> >
>> > But let me know what you find regarding a sparse
>> > implementation.
>> >
>> > Cheers,
>> > Matthias
>> >
>> >
>> > On Thu, 2015-07-16 at 00:48 +0100, jian zhang wrote:
>> > > Hi,
>> > >
>> > >
>> > > Is the sparse features at phrase table, like
>> > >
>> > >
>> > >
>> > > das Haus ||| the house ||| 0.8 0.5 0.8 0.5 2.718
>> ||| 0-0 1-1
>> > ||| 5000
>> > > 5000 2500 ||| dom_europarl 1
>> > >
>> > >
>> > >
>> > > still supported? If yes, what should I set to the
>> ini file
>> > based on
>> > > the example above?
>> > >
>> > >
>> > > Thank,
>> > >
>> > >
>> > > Jian
>> > >
>> > >
>> > > --
>> > > Jian Zhang
>> > > Centre for Next Generation Localisation (CNGL)
>> > > Dublin City University
>> >
>> > > _______________________________________________
>> > > Moses-support mailing list
>> > > Moses-support@mit.edu
>> > >
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>> >
>> >
>> >
>> > --
>> > The University of Edinburgh is a charitable body,
>> registered
>> > in
>> > Scotland, with registration number SC005336.
>> >
>> >
>> >
>> >
>> >
>> > --
>> > Jian Zhang
>> > Centre for Next Generation Localisation (CNGL)
>> > Dublin City University
>>
>>
>>
>> --
>> The University of Edinburgh is a charitable body, registered
>> in
>> Scotland, with registration number SC005336.
>>
>>
>>
>>
>>
>>
>> --
>> Jian Zhang
>> Centre for Next Generation Localisation (CNGL)
>> Dublin City University
>
>
>
> --
> The University of Edinburgh is a charitable body, registered in
> Scotland, with registration number SC005336.
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support


------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 105, Issue 34
**********************************************

Tweet