Moses-support Digest, Vol 93, Issue 3

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: embeddings (Miles Osborne)
2. Re: Using word embeddings in Moses (Philipp Koehn)
3. representation of coverage vector (bitset operations)?
(Adam Teichert)


----------------------------------------------------------------------

Message: 1
Date: Wed, 2 Jul 2014 09:59:58 -0400
From: Miles Osborne <miles@inf.ed.ac.uk>
Subject: Re: [Moses-support] embeddings
To: hubert.soyer@googlemail.com
Cc: moses-support@mit.edu
Message-ID:
<CAPRfTYqQJ=jcfvzwm2G6FvPjFSyCo1Ni9JwDNX66mk=B2hOOYw@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

I would model them as feature functions over phrases. You might imagine
that you can exploit vector similarity to do smoothing.

Good luck

Miles
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20140702/7bc67dc3/attachment-0001.htm

------------------------------

Message: 2
Date: Wed, 2 Jul 2014 10:02:50 -0400
From: Philipp Koehn <pkoehn@inf.ed.ac.uk>
Subject: Re: [Moses-support] Using word embeddings in Moses
To: Hubert Soyer <hubert.soyer@googlemail.com>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<CAAFADDCRV6D8pBCR9wLwQDUGD0NLifM8hrYg-Fb2xU_=5j_HWg@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8

Hi,

it would be better to include a word vector obtained by word2vec or other means
as a single factor, and generate them with a generation step to avoid filling
up the phrase table with redundant information. Unfortunately, there is no
source side generation step, which may be a useful addition to the factored
model.

Of course, the question is what to do with these vectors. I assume that you have
a new feature function in mind.

-phi

On Wed, Jul 2, 2014 at 5:04 AM, Hubert Soyer
<hubert.soyer@googlemail.com> wrote:
> Hello,
>
> I have checked the mailing list archive for this question but couldn't
> find anything.
> I'd be surprised if this question has not been asked yet, if it has,
> I'd be happy if you could point me to the corresponding mails.
>
> Recently, word representations induced by neural networks have gained
> a lot of momentum.
> Particularly often cited in this context is:
> http://code.google.com/p/word2vec/
>
> Those vector word representations are vectors that carry some semantic
> meaning in them, i.e. semantically similar words have similar vectors
> (small distances to each other).
>
> I have been wondering about the best way to incorporate them in Moses.
>
> One solution would be to incorporate them as factors in a factored model:
>
> http://www.statmt.org/moses/?n=Moses.FactoredTutorial
>
> It seems to me that I would have to treat each dimension of each word
> vector as a separate factor which would lead to a lot of factors.
> Usual dimensionalities of those word vectors are 200 or more.
>
> Is treating each dimension as a factor the best way to incorporate
> those vectors or is there anything better I can do?
> I don't have to stick to factors, if there is another way.
>
> Thank you in advance!
>
> Best,
>
> Hubert
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support


------------------------------

Message: 3
Date: Wed, 2 Jul 2014 10:57:51 -0400
From: Adam Teichert <adamteichert@gmail.com>
Subject: [Moses-support] representation of coverage vector (bitset
operations)?
To: moses-support@mit.edu
Message-ID:
<CAKiBsC-wNk9oYieGSyw43Hgvkd8cGDfOvXzqhERegvFsjH928Q@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

This is probably a very naive question, but I go for it anyway? Does
anyone have any comments about why WordsBitmap doesn't use the stl or boost
bit-vector / bit-set datastructures (for example std::vector<bool>,
std::bitset or the boost dynamically sized version)?

This is just coming up because I'm interested in doing some bit-vector
operations with the coverage vector and I'd like them to look more like
constant time operations than linear time operations in the size of the
source sentence.

Any comments?

https://github.com/moses-smt/mosesdecoder/blob/73513c182dd3e12abf4a1df82a7e7808e3dfab83/moses/WordsBitmap.h

Thanks!

--Adam
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20140702/20a2b7af/attachment-0001.htm

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 93, Issue 3
********************************************

Tweet