Moses-support Digest, Vol 87, Issue 36

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: When using moses, cpu is are not stable. (Wang Pidong)
2. Re: Factorized corpus in EMS (Philipp Koehn)
3. 2nd CFP: Computer Speech and Language Special Issue on Hybrid
Machine Translation (Marta Ruiz)


----------------------------------------------------------------------

Message: 1
Date: Wed, 15 Jan 2014 05:52:17 -0800
From: Wang Pidong <wangpd@comp.nus.edu.sg>
Subject: Re: [Moses-support] When using moses, cpu is are not stable.
To: Barry Haddow <bhaddow@staffmail.ed.ac.uk>
Cc: moses-support@mit.edu
Message-ID:
<CAKYvXDGHCBrtmqL66CoP-rfo6oS8okwSAKnwObU16GAwdbft+w@mail.gmail.com>
Content-Type: text/plain; charset="euc-kr"

Thanks for your head ups, Barry!
Yes, in moses (not moseserver), there is a thread pool containing fixed
number of threads to avoid the overhead of thread creations.

Then I would think the model loading may be the main reason for the
unstable CPU usage, if Hansung is using moses, not mosesserver.

Best wishes!
Pidong


On 15 January 2014 02:53, Barry Haddow <bhaddow@staffmail.ed.ac.uk> wrote:

> Hi Wang
>
> That's only true for mosserver. For regular moses and moses_chart, a fixed
> size thread pool is used.
>
> Cheers - Barry
>
> Wang Pidong <wangpd@comp.nus.edu.sg> wrote:
>
> >The unstable state can be caused by the fact that Moses create a new
> thread
> >to translate each new sentence, so when translating a large file, Moses
> >frequently creates threads and then OS has to reschedule frequently also.
> >
> >Even in single-thread Moses, given each input sentence, it also needs to
> >load phrase table, language model, etc. which may also be blocked due to
> IO
> >operations, if you use binarized models.
> >
> >Just for your reference.
> >
> >Best wishes!
> >Pidong
> >
> >
> >
> >On 14 January 2014 23:24, Hansung Cho <gyuber@gmail.com> wrote:
> >
> >> hello.
> >>
> >> To change the option "-threads 64" then monitoring at server CPUs
> status.
> >> but, result is the same..
> >> Try the other way?
> >>
> >> Attach the CPUs status picture
> >>
> >> thank you
> >> Hansung Cho
> >>
> >> [image: ?? ??? 1]
> >>
> >>
> >> 2014/1/15 Tom Hoar <tahoar@precisiontranslationtools.com>
> >>
> >>> To change the threads, run with '-threads x' added to your command
> line:
> >>>
> >>> moses -f /path/to/moses.ini -threads 20
> >>>
> >>>
> >>>
> >>>
> >>> On 01/14/2014 11:41 PM, Philipp Koehn wrote:
> >>>
> >>> Hi,
> >>>
> >>> you can try specifying more threads than the number of CPUs.
> >>> I have no idea if this speeds things up or causes blockage in
> >>> your setup. Just try it out and see what happens.
> >>>
> >>> -phi
> >>>
> >>>
> >>> On Tue, Jan 14, 2014 at 4:31 AM, Hansung Cho <gyuber@gmail.com> wrote:
> >>>
> >>>> My server configuration are multiple CPUs 16 core and running thread
> 16.
> >>>>
> >>>> How do I adjust the moses running thread?
> >>>>
> >>>> I need your help...
> >>>>
> >>>> Thank you
> >>>>
> >>>>
> >>>> 2014/1/14 Tom Hoar <tahoar@precisiontranslationtools.com>
> >>>>
> >>>>> With a system as complex as Moses, why would you think the CPU usage
> >>>>> would be steady? Each sentence task goes through many stages with
> different
> >>>>> CPU demands which probably map to the variation in each cycle.
> >>>>>
> >>>>> You can also look first at your threading configuration. If your
> system
> >>>>> has multiple CPUs (cores) and Moses is running multi-threaded, I
> suspect
> >>>>> the cycles' frequencies are related to the activities on each thread.
> >>>>>
> >>>>>
> >>>>>
> >>>>> On 01/14/2014 01:09 PM, Hansung Cho wrote:
> >>>>>
> >>>>> hello..
> >>>>>
> >>>>> I'm using mosesdecoding
> >>>>>
> >>>>> When monitoring a physical server, Server cpu is not very stable.
> >>>>> However, the state is constantly moving cpu
> >>>>>
> >>>>> Look at the picture below
> >>>>> [image: ?? ??? 1]
> >>>>>
> >>>>> Is it because the moses system cache??
> >>>>> Should any part option of the adjustment?
> >>>>>
> >>>>> My moses commit a1584c608f8bbd83921896bac9eac97d9effc0c8
> >>>>> Options related to the cache are "clean-lm-cache" ( default ==1 )
> >>>>>
> >>>>> Will be waiting for your input.
> >>>>>
> >>>>> Thank you
> >>>>> Hansung cho
> >>>>>
> >>>>>
> >>>>> _______________________________________________
> >>>>> Moses-support mailing listMoses-support@mit.eduhttp://
> mailman.mit.edu/mailman/listinfo/moses-support
> >>>>>
> >>>>>
> >>>>>
> >>>>> _______________________________________________
> >>>>> Moses-support mailing list
> >>>>> Moses-support@mit.edu
> >>>>> http://mailman.mit.edu/mailman/listinfo/moses-support
> >>>>>
> >>>>>
> >>>>
> >>>> _______________________________________________
> >>>> Moses-support mailing list
> >>>> Moses-support@mit.edu
> >>>> http://mailman.mit.edu/mailman/listinfo/moses-support
> >>>>
> >>>>
> >>>
> >>>
> >>> _______________________________________________
> >>> Moses-support mailing listMoses-support@mit.eduhttp://
> mailman.mit.edu/mailman/listinfo/moses-support
> >>>
> >>>
> >>>
> >>> _______________________________________________
> >>> Moses-support mailing list
> >>> Moses-support@mit.edu
> >>> http://mailman.mit.edu/mailman/listinfo/moses-support
> >>>
> >>>
> >>
> >> _______________________________________________
> >> Moses-support mailing list
> >> Moses-support@mit.edu
> >> http://mailman.mit.edu/mailman/listinfo/moses-support
> >>
> >> --
> >> Wang Pidong
> >>
> >> Department of Computer Science
> >> School of Computing
> >> National University of Singapore
> >>
> >>
> >
> >_______________________________________________
> >Moses-support mailing list
> >Moses-support@mit.edu
> >http://mailman.mit.edu/mailman/listinfo/moses-support
>
> --
> The University of Edinburgh is a charitable body, registered in
> Scotland, with registration number SC005336.
>
>


--
Wang Pidong

Department of Computer Science
School of Computing
National University of Singapore
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20140115/439c40b5/attachment-0001.htm

------------------------------

Message: 2
Date: Wed, 15 Jan 2014 09:13:02 -0500
From: Philipp Koehn <pkoehn@inf.ed.ac.uk>
Subject: Re: [Moses-support] Factorized corpus in EMS
To: burak ayd?n <baydinx@gmail.com>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>
Message-ID:
<CAAFADDA86EpvgH7yhWQGbLdWTFVxc=4wGYjBXjX8f9w2a6gFEw@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8

Hi,

you are on the right track.

EMS is typically configured with pointers to raw data and scripts
for tokenization, factorization, truecasing, etc. But you can also
always specify fully prepared data.

So, for instance, instead of specifying raw data for the parallel corpus

raw-stem = $toy-data/my-corpus

you can specify factorized data

factorized-stem = $toy-data/my-corpus.factored

The setting names for the different data sets (parallel corpus,
LM training corpus, tune and test set) are in the example configuration
files or can be found in experiment.meta (which is the official
authority).

-phi

On Wed, Jan 15, 2014 at 8:51 AM, burak ayd?n <baydinx@gmail.com> wrote:
> Hi all,
>
> In moses web page, there are instructions for factored training. What I want
> to to experiment is to give already factorized corpus. What will be the new
> factor definition and new Lm definition in the EMS config?
>
> When I try to replicate the original config.factor experiment, I observe
> that there are files created under corpus/ directory :
>
> factored.<expnum>.en
> factored.<expnum>.en.pos
>
> So in order to give factorized corpus as input, should I also prepare one or
> more of the files above and write it in the EMS config as well?
>
> Regards
> Burak
>
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>



------------------------------

Message: 3
Date: Wed, 15 Jan 2014 17:05:58 +0100
From: Marta Ruiz <martaruizcostajussa@gmail.com>
Subject: [Moses-support] 2nd CFP: Computer Speech and Language Special
Issue on Hybrid Machine Translation
To: Marta Ruiz <martaruizcostajussa@gmail.com>
Message-ID:
<CABEBqHJ0kBFviTNVuV3THiram2bmsrosWSDKoPTgmZd4i_WFvw@mail.gmail.com>
Content-Type: text/plain; charset="iso-8859-1"

Hybrid Machine Translation focuses on combining the best properties of
different Machine Translation (MT) paradigms. Nowadays, it is very popular
to include linguistic features in Statistical Machine Translation (SMT)
systems or to modify the standard Rule-based Machine Translation (RBMT)
architecture to include statistical knowledge. Other ways of hybridization
include the system combination techniques which may combine a rich variety
of translation paradigms.

This special issue in the prestigious Computer Speech and Language journal
will cover different approaches from Hybrid Machine Translation and take
advantage of the latest and leading research to discuss the progress in the
field.

This special issue invites contributions related to (but not limited):

1.

theoretical and experimental design of hybrid MT architectures
2.

experimental results with hybrid MT systems guided by corpus-based or
rule-based systems
3.

introduction of linguistics in corpus-based approaches
4.

rule-based systems extended or built with statistical information
5.

induction of lexical or grammatical transfer rules from corpora
6.

description of open source tools and language resources for hybrid MT
7.

description of computationally efficient algorithms for hybrid MT
8.

applications of hybrid MT systems
9.

hybrid methods applied to spoken language translation (SLT)
10.

hybrid evaluation methods
11.

system combination of different MT and SLT paradigms.

*Submission procedure*
Prospective authors should follow the regular guidelines of the Computer
Speech and Language Journal for electronic submission
(*http://ees.elsevier.com/csl
<http://ees.elsevier.com/csl>*). During submission authors must select "SI:
Hybrid Machine Translation" as Article Type.

*Important dates: *
1st March 2014: Deadline for Submissions
1st June 2014: Notification of Resubmission
1st July 2014: Deadline for Resubmission
1st August 2014: Final Decision


*Guest Editors:*
Dr. Marta R. Costa-juss?, Institute for Infocomm Research, Singapore,
*martaruizcostajussa@gmail.com
<martaruizcostajussa@gmail.com>*
Prof. Jos? A. R. Fonollosa, Universitat Polit?cnica de Catalunya,
Barcelona, *jose.fonollosa@upc.edu <jose.fonollosa@upc.edu>*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20140115/468623f1/attachment.htm

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 87, Issue 36
*********************************************

Tweet