Moses-support Digest, Vol 89, Issue 17

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: Help. First request to MosesServer very slow
(Marcos Fernandez)
2. Re: No Moses translation without model >=3 (John D. Burger)


----------------------------------------------------------------------

Message: 1
Date: Fri, 7 Mar 2014 13:31:26 +0000 (UTC)
From: Marcos Fernandez <marcos.fernandez.lopez@usc.es>
Subject: Re: [Moses-support] Help. First request to MosesServer very
slow
To: moses-support@mit.edu
Message-ID: <loom.20140307T143115-542@post.gmane.org>
Content-Type: text/plain; charset=us-ascii

Thanks a lot for all the replies!

Marcin Junczys-Dowmunt <junczys@...> writes:

>
> Hi Marcos,
> What you can do, if you want to rule out the compact data structures are
> the issue, is load the tables at start-up into memory (if you have
> enough memory):


Marcin, I have tried what you suggest, but it didn't improve things. With
tables loaded at start-up, Moses uses 1GB more of RAM, but the translation
times remains the same.

And yes, I am using only one server instance that is running all the time.


Barry Haddow <bhaddow@...> writes:

>
> Hi Marcos
>
> I think the problem is that the rules (or phrase pairs) are now cached
> on a per thread basis. This is good for command-line Moses as it uses a
> pool of threads, and having per-thread caches means that there is no
> locking on the caches, as there used to be.
>
> mosesserver, afaik, creates a new thread for each connection, so it
> can't take advantage of the cache. This is done in the xmlrpc-c library
> so we don't have much control over it. If you dig around in the xmlrpc-c
> documentation (or code!) you might find a way to control the threading
> policy.


Barry, I am not sure that that is the cause, because in that case my "20
short sentences" file would get translated much faster with command-line
Moses, however it takes 4.5 seconds to translate it. This is, 2 seconds
(measured) for loading tables in memory, and the same 2.5 seconds than with
MosesServer to translate the file. And the behaviour is also the same as
before, the first sentence in the file takes much longer than the rest.

However, what you say perhaps could be the cause of the difference in time
between using a different xmlrpc ServerProxy object for each request
(probably, in this case xmlrpc executes each request in a different thread),
or reusing one only ServerProxy for all the requests (where there would be
only one thread, so it could take advantage of the cache).

What I understand then, is that the cache stores information of the
previously translated sentences to accelerate the translation of the next
ones. But that does not eliminate the problem of the first slower request.
As you can see, I am finding that issue even in command-line Moses (not with
the first "request", but with the first sentence in a file).

I am thinking that perhaps I have no problem, and this is just the usual way
in which Moses works. Just to make sure:

1. would you say that a time of 2-3 seconds for translating (spa-eng) a
single sentence (~15-20 words) could be a normal response time (discounting
the time of loading tables)? (Intel Xeon with 32GB RAM)
2. if now you write 3 similar sentences in a file, and execute Moses
(command-line, serial) over this file, would you expect it to take a much
shorter time (perhaps the half) than the sum of the times for the 3 single
sentences?

If the answer is yes to both (specially the second), then I am probably
worrying in vain. My worries started when I read the "MTMonkey" paper:
http://ufal.mff.cuni.cz/~pecina/files/pbml-2013.pdf

Here the authors use the approach of creating a new ServerProxy instance
each time a request is sent to MosesServer (the worst case scenario for me),
and they get great results, so I thought they were not experiencing that
overhead for every request. But perhaps they just used sentences that get
translated very fast even with that overhead.

Well, in the case that this is the usual way of working for Moses, if what
Kenneth suggests is possible, that would eliminate the overhead almost
completely for MosesServer. I mean, there would still be an overhead for the
first request that each thread serves after the server is just launched, but
never more, because the caches would be filled with useful information from
that point on. I think that this would be extremely interesting for web
translating services, or to do web-page translations "on the fly".

However, I don't see a way to avoid that overhead in command-line Moses, as
it "dies" after each execution.

Marcos.



>
> I just spoke to Marcin about the problem, and we're not sure if loading
> the compact phrase table into memory would help, as you still would need
> the higher level cache (in PhraseDictionary). But you could try this anyway.
>
> cheers - Barry
>
> On 06/03/14 17:20, Marcos Fernandez wrote:
> > Hi, I am having an issue with MosesServer.
> >
> > I am using compact phrase and reordering table, and KENLM.
> >
> > The problem is this (I'll explain with an example):
> >
> > - I have one file with 20 very short sentences. I split and tokenize
> > them and send one XMLPRC request per sentence to MosesServer
> > - If I create just one XMLRPC ServerProxy instance and I use it to send
> > all the requests through it, all the sentences get translated in approx
> > 2.5 sec. The problem is that the first sentence takes almost 2 seconds
> > to get translated, while the other 19 are much faster
> > - If I create one ServerProxy instance per request, the translation time
> > rises to 30 sec (now every sentence takes almost 2 sec)
> >
> > I don't understand the reason of that delay for the first request. I
> > have followed the source of this delay to the function:
> >
> > GetTargetPhraseCollectionLEGACY(const Phrase& src)
> >
> > in the file: ...TranslationModel/PhraseDictionary.cpp
> >
> > It seems that for the first request it's needed to look for something
> > in the phrase table, while for subsequent requests it can be retrieved
> > (most of the times) from a cache.
> >
> > But, as the sentences in my file are not related one to another in any
> > way, the information on this cache can not be sentence-dependent, so why
> > wouldn't it be possible for the cache to be preloaded with the
> > information needed?
> >
> > I think that perhaps I have something misconfigured, because I have seen
> > other people using the approach of creating one ServerProxy object for
> > each XMLRPC request (which would facilitate things a lot for me), so I
> > don't think they are experiencing this overhead. Perhaps using the
> > compact formats can have something to do with it?
> >
> > Any help would be much appreciated. I paste below my moses.ini, if that
> > helps:
> >
> > Thanks :)
> >
> > ### MOSES CONFIG FILE ###
> > ###################
> >
> > # input factors
> > [input-factors]
> > 0
> >
> > # mapping steps
> > [mapping]
> > 0 T 0
> >
> > # translation tables: table type (hierarchical(0), textual (0), binary
> > (1)), source-factors, target-factors, number of scores, file
> > # OLD FORMAT is still handled for back-compatibility
> > # OLD FORMAT translation tables: source-factors, target-factors, number
> > of scores, file
> > # OLD FORMAT a binary table type (1) is assumed
> > [ttable-file]
> > 12 0 0 5 /opt/moses-compiling/modelos/es-en/phrase-model/phrase-table
> >
> > # no generation models, no generation-file section
> >
> > # language models: type(srilm/irstlm), factors, order, file
> > [lmodel-file]
> > 8 0 5
> > /opt/moses-compiling/modelos/es-en/lm/13-19-03gen_intec_head8m_sb5LM.kenlm
> >
> >
> > # limit on how many phrase translations e for each phrase f are loaded
> > # 0 = all elements loaded
> > [ttable-limit]
> > 10
> >
> > # distortion (reordering) files
> > [distortion-file]
> > 0-0 wbe-msd-bidirectional-fe-allff 6
> > /opt/moses-compiling/modelos/es-en/phrase-model/reordering-table
> >
> > # distortion (reordering) weight
> > [weight-d]
> > 0.097107
> > 0.150373
> > -0.0551767
> > -0.0307787
> > 0.114613
> > 0.214587
> > 0.0467398
> >
> > # language model weights
> > [weight-l]
> > 0.0442748
> >
> >
> > # translation model weights
> > [weight-t]
> > 0.00370888
> > 0.0425665
> > 0.0719956
> > 0.0202699
> > 0.071147
> >
> > # no generation models, no weight-generation section
> >
> > # word penalty
> > [weight-w]
> > 0.0366626
> >
> > [distortion-limit]
> > 6
> >
> > [v]
> > 0
> >
> >
>






------------------------------

Message: 2
Date: Fri, 7 Mar 2014 09:38:45 -0500
From: "John D. Burger" <john@mitre.org>
Subject: Re: [Moses-support] No Moses translation without model >=3
To: Momo Jeng <momo_jeng@outlook.com>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID: <01B5D252-818E-422E-B1DA-CC99A86C74A4@mitre.org>
Content-Type: text/plain; charset="iso-8859-1"

> Incidentally, it looks like Moses lets you do alignment purely with Model 1, by setting "-final-alignment-model 1", although like hmm, it's hard-coded to pick up the 5th iteration.

Ah, my apologies. I think this is new since the last time I looked at this (4-5 years ago). I do remember going through some minor contortions to get Giza to dump (and Moses to use) a non-standard alignment file.

- JB

On Mar 6, 2014, at 19:38 , Momo Jeng <momo_jeng@outlook.com> wrote:

> Marcin, John,
>
> Thanks! This works for me with the --hmm-align option.
>
> One note: it only works if the number of hmm iterations is set >= 5. Looking at the train-model.perl script, it has:
>
> $___GIZA_EXTENSION = 'Ahmm.5' if $___FINAL_ALIGNMENT_MODEL eq 'hmm';
>
> This seems to mean that it's specifically looking for the output of the 5th hmm iteration. If I set the number of hmm iterations to 4, Moses fails, saying it can't find an alignment file. If I set the number of hmm iterations to 10, GIZA does 10 iterations, but then picks the 5th hmm iteration (hmm dump frequency being set to 5 by default). Not a problem for me, since I can always manually change the extension in the perl script if I want to pick up a different iteration, but this seems like a bug.
>
> Incidentally, it looks like Moses lets you do alignment purely with Model 1, by setting "-final-alignment-model 1", although like hmm, it's hard-coded to pick up the 5th iteration.
>
> Cheers,
>
> Momo
>
> > Date: Thu, 6 Mar 2014 21:46:05 +0000
> > From: junczys@amu.edu.pl
> > To: moses-support@mit.edu
> > Subject: Re: [Moses-support] No Moses translation without model >=3
> >
> > You can use the HMM model with the --hmm-align option for
> > train-model.perl. The training scripts fails because you only changed
> > the options of Giza without informing the script that it should not
> > expect any *.A3 files. The --hmm-align option does this for you.
> >
> > Also there is no reason why you could not produce alignment files using
> > only model 1, you would just need to adjust the training script
> > accordingly. The uniform distribution is assumed for the first initial
> > step, parameters change to something more meaningful after several
> > iterations of the EM algorithm. However, most probably pure Model 1
> > alignment won't be very good, as it incorporates no
> > word-position-related information, the HMM model is much better usually.
> > Some people claim, that you can abandon Model 3 and 4 altogether and
> > stop after the HMM model.
> > Best,
> > Marcin
> >
> > W dniu 06.03.2014 21:29, John D. Burger pisze:
> > > On Mar 6, 2014, at 16:00 , Momo Jeng <momo_jeng@outlook.com> wrote:
> > >
> > >> I'm having a problem getting results from Moses, although I think it's really a problem with GIZA++; please let me know if there's a better place for GIZA questions.
> > >>
> > >> When I run Moses instructing GIZA++ to only do model1 and hmm iterations ( "--giza-option model1iterations=3,hmmiterations=3,model3iterations=0,model4iterations=0" ), Moses fails, because GIZA++ doesn't produce an alignment file (the .A3.final file).
> > >>
> > >> Based on a quick look at the GIZA++ code, this failure is explicit in the code. In main.cpp, m3.viterbi(...) is only called at line 652 if at least one iteration is set for model 3, 4, 5, or 6, and m3.viterbi(...) is where the code for writing the alignment file is called. So I'm wondering if this a bug, or by design. Is there some reason that I shouldn't create alignments without using model 3 or higher?
> > > You can't produce actual alignments with Model 1 - it assumes a uniform distribution from each word over all the words in the other half of the sentence pair. It's really only useful for initializing the word translation probabilities for one of the more sophisticated models.
> > >
> > > - JB
> > > _______________________________________________
> > > Moses-support mailing list
> > > Moses-support@mit.edu
> > > http://mailman.mit.edu/mailman/listinfo/moses-support
> >
> > _______________________________________________
> > Moses-support mailing list
> > Moses-support@mit.edu
> > http://mailman.mit.edu/mailman/listinfo/moses-support
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20140307/6c163993/attachment-0001.htm

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 89, Issue 17
*********************************************

Tweet