Moses-support Digest, Vol 86, Issue 18

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: merging two translation models (Hieu Hoang)
2. CFP: The 37th Annual ACM SIGIR 2014 conference - Gold Coast
Australia - July 6-11 (Richi Nayak)
3. Re: Fwd: Fw: Warning: Too many arguments while IRSTLM
language model Training (renubalyan)


----------------------------------------------------------------------

Message: 1
Date: Fri, 6 Dec 2013 10:57:28 +0000
From: Hieu Hoang <Hieu.Hoang@ed.ac.uk>
Subject: Re: [Moses-support] merging two translation models
To: ??????? ????????? <verbalab@yandex.ru>
Cc: "moses-support@mit.edu" <moses-support@mit.edu>, Rico Sennrich
<rico.sennrich@gmx.ch>
Message-ID:
<CAEKMkbg4StwtEhgvfFatEuVvniU2sFpa0KRsYyd1csWROyZT7Q@mail.gmail.com>
Content-Type: text/plain; charset="koi8-r"

it's probably doable. The main thing you have to do is to implement
ScoreComponentCollection::UnregisterScoreProducer(this);
which is the opposite of
ScoreComponentCollection::RegisterScoreProducer(this);

there are other things you would need to do to get the decoding algorithms
to use the new phrase table. We can thing about those later.

however, i don't understand why you'll want to do this. Phrase-tables can
be binarized so they don't use any RAM, but they just memory map them.

If they use lots of RAM, then it's likely to take a long time to load and
unload so you do want to waste time creating/destroying them too often.

if you have problems implementing it, please let me know & i'll see if i
can help


On 4 December 2013 18:00, ??????? ????????? <verbalab@yandex.ru> wrote:

> Many thanx to your replies!
>
> One more question concerning these scripts: are there capability to detach
> translation models to free RAM and attached new TMs while the decoder is
> running? If not maybe you can provide a roadmap for me to contribute such
> functionality?
>
> Kind regards!
>
> 04.12.2013, 18:39, "Rico Sennrich" <rico.sennrich@gmx.ch>:
> > ??????? ????????? <verbalab@...> writes:
> >
> >> Hello, everyone!
> >>
> >> I have the following question: I have trained a huge translation model
> >
> > trained on 1m sentences of news
> >
> >> texts. Also I have several minor translation models trained on texts of
> >
> > different domains. Are there any
> >
> >> tools in Moses that can enable merging translation models? Is it
> possible
> >
> > to merge models when the decoder
> >
> >> is running?
> >>
> >> Thanks!
> >>
> >> Kind regards, Alex Kalinin.
> >
> > Hi Alex,
> >
> > there are several scripts that allow you to merge multiple translation
> > models: http://www.statmt.org/moses/?n=Moses.AdvancedFeatures#ntoc52
> >
> > if you're interested in merging models at decoding time, you can do a
> > log-linear combination of models (
> > http://www.statmt.org/moses/?n=Moses.AdvancedFeatures#ntoc18 ), or a
> > linear-interpolation / count-based merge (
> > http://www.statmt.org/moses/?n=Moses.AdvancedFeatures#ntoc55 ).
> >
> > All methods allow you to weight the models in order to prioritize
> in-domain
> > models. Check the documentation/literature if you're interested in more
> details.
> >
> > best wishes,
> > Rico
> >
> > _______________________________________________
> > Moses-support mailing list
> > Moses-support@mit.edu
> > http://mailman.mit.edu/mailman/listinfo/moses-support
>
> --
> ? ?????????, ????????? ???????
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>



--
Hieu Hoang
Research Associate
University of Edinburgh
http://www.hoang.co.uk/hieu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20131206/f2685256/attachment-0001.htm

------------------------------

Message: 2
Date: Fri, 6 Dec 2013 16:53:15 +1000
From: Richi Nayak <r.nayak@qut.edu.au>
Subject: [Moses-support] CFP: The 37th Annual ACM SIGIR 2014
conference - Gold Coast Australia - July 6-11
To: Richi Nayak <r.nayak@qut.edu.au>, "acl@aclweb.org"
<acl@aclweb.org>, "aepia@aepia.org" <aepia@aepia.org>,
"ah@listserver.tue.nl" <ah@listserver.tue.nl>,
"bionlp@lists.ccs.neu.edu" <bionlp@lists.ccs.neu.edu>,
"CHI-ANNOUNCEMENTS@acm.org" <CHI-ANNOUNCEMENTS@acm.org>,
"clef@dei.unipd.it" <clef@dei.unipd.it>, "complit@linguistlist.org"
<complit@linguistlist.org>, "corpora@uib.no" <corpora@uib.no>,
"dbitaly@list.dia.uniroma3.it" <dbitaly@list.dia.uniroma3.it>,
"diglib@infoserv.inist.fr" <diglib@infoserv.inist.fr>,
"elsnet-list@elsnet.org" <elsnet-list@elsnet.org>,
"elsnet-list@mailman.let.uu.nl" <elsnet-list@vz07-list.im.hum.uu.nl>,
"imageclef@lists.shef.ac.uk" <imageclef@lists.shef.ac.uk>,
"info@dariah.eu" <info@dariah.eu>, "IRList@lists.shef.ac.uk"
<IRList@lists.shef.ac.uk>, "jesse@listserv.utk.edu"
<jesse@listserv.utk.edu>, "liresearch@computing.dcu.ie"
<liresearch@computing.dcu.ie>, "listmaster@loria.fr"
<listmaster@loria.fr>, "ln@cines.fr" <ln@cines.fr>,
"lr_egroup@mail.iiit.ac.in" <lr_egroup@mail.iiit.ac.in>,
"maillist@afnlp.org" <maillist@afnlp.org>, "members@sigsem.org"
<members@sigsem.org>, "moses-support@mit.edu" <moses-support@mit.edu>,
"mt-list@eamt.org" <mt-list@eamt.org>, "news@multilingual.com"
<news@multilingual.com>, "nlpcall@watarts.uwaterloo.ca"
<nlpcall@watarts.uwaterloo.ca>, "nodali@helsinki.fi"
<nodali@helsinki.fi>, "portal@aclweb.org" <portal@aclweb.org>,
"Pourinfos@risc.cnrs.fr" <Pourinfos@risc.cnrs.fr>,
"publ@isca-speech.org" <publ@isca-speech.org>,
"researchers@pascal-network.org" <researchers@pascal-network.org>,
"sentproc@lists.qc.cuny.edu" <sentproc@lists.qc.cuny.edu>,
"sigann@cs.vassar.edu" <sigann@cs.vassar.edu>,
"SIGHIT-MEMBERS@LISTSERV.ACM.ORG" <SIGHIT-MEMBERS@LISTSERV.ACM.ORG>,
"sigsem@aclweb.org" <sigsem@aclweb.org>, "trec-blog@nist.gov"
<trec-blog@nist.gov>, "www-rdf-logic@w3.org" <www-rdf-logic@w3.org>,
"carol@iei.pi.cnr.it" <carol@iei.pi.cnr.it>
Message-ID:
<B33DE6A8054C2E4799AAA0562A80CD0F0C34604CF0@QUTEXMBX02.qut.edu.au>
Content-Type: text/plain; charset="Windows-1252"

CALL FOR PAPERS, Submission Due: JAN 27, 2014



ACM SIGIR 2014: THE 37th ANNUAL CONFERENCE



6-11 July, 2014, Gold Coast, Australia



Conference website: http://sigir.org/sigir2014/



SIGIR is the major international forum for the presentation of new research results and for the demonstration of new systems and techniques in the broad field of information retrieval (IR). The Conference and Program Chairs invite all those working in areas related to IR to submit original papers for review. SIGIR 2014 welcomes contributions related to any aspect of IR theory and foundation, techniques, and applications. Relevant topics include, but are not limited to:



TOPICS

? Document Representation and Content Analysis (e.g., text representation, document structure, linguistic analysis, NLP for IR, cross- and multi-lingual IR, information extraction, sentiment analysis, clustering, classification, topic models, facets, text streams)

? Queries and Query Analysis (e.g., query intent, query suggestion and prediction, query representation and reformulation, query log analysis, conversational search and dialogue, spoken queries, summarization, question answering)

? Retrieval Models and Ranking (e.g., IR theory, language models, probabilistic retrieval models, learning to rank, combining searches, diversity and aggregated search)

? Search Engine Architectures and Scalability (e.g., indexing, compression, distributed IR, P2P IR, mobile IR, cloud IR)

? Users and Interactive IR (e.g., user studies, user and task models, interaction analysis, session analysis, exploratory search, personalized search, social and collaborative search, search interface, whole session support)

? Filtering and Recommending (e.g., content-based filtering, collaborative filtering, recommender systems)

? Evaluation (e.g., test collections, experimental design, effectiveness measures, session-based evaluation, simulation)

? Web IR and Social Media Search (e.g., link analysis, click models/behavioral modeling, social tagging, social network analysis, blog and microblog search, forum search, community-based QA, adversarial IR and spam, vertical and local search)

? IR and Structured Data (e.g., XML search, ranking in databases, desktop search, entity search)

? Multimedia IR (e.g., image search, video search, speech/audio search, music search)

? Other Applications (e.g., digital libraries, enterprise search, genomics IR, legal IR, patent search, text reuse, new retrieval problems)



CONTRIBUTION TYPES

? Full papers (10 pages), Short papers (4 pages), Demos (3 pages), Tutorials, Workshops



INSTRUCTION

Requirements for paper format and appropriate content are described in the content guidelines<http://sigir.org/sigir2014/PaperContentGuidelines.php>. The requirements will be strictly enforced. Papers which do not conform to the requirements may be rejected without review, so please be sure to read this page carefully.

SIGIR 2014 solicits proposals for tutorials of either half-day (3 hours plus breaks) or full day (6 hours plus breaks) on all topics of information retrieval and its applications. Each tutorial should cover a single topic in detail. Submissions should include a cover sheet and an extended abstract.

Proposals for workshops to be held at ACM SIGIR 2014 are also solicited. Workshops will usually last for one day and will be held on Friday 11th July 2014.

IMPORTANT DATES

? 20 January 2014: Abstracts for full research papers due

? 27 January 2014: Full research papers due

? 3 February 2014: Workshop proposals due

? 17 February 2014: Short paper, demonstration, and tutorial submission deadline

? 18 April 2014: Paper, short paper, tutorial, and demonstration acceptance notifications

? 11 May 2014: Camera ready copy due (note the short timeline due to early conference date)

? 16 May 2014: Early bird registration deadline



ORGANIZERS

? General Chairs: Shlomo Geva, Andrew Trotman

? PC Chairs: Peter Bruza, Charles L. A. Clarke, Kalervo J?rvelin

@Richi Nayak ? Publicity Chair

Dr Richi Nayak, Associate Professor
Higher Degree Research Director, School of Electrical Engineering and Computer Science
Science and Engineering Faculty| Queensland University of Technology |Brisbane, QLD 4001
Office: S1206 | Ph: 313 81976 | Fax: 313 89390 | Email: r.nayak@qut.edu.au<mailto:resources.scitech@qut.edu.au>
Webpage: http://applieddatamining.info/





------------------------------

Message: 3
Date: Fri, 6 Dec 2013 22:09:12 +0530 (IST)
From: renubalyan <renubalyan@cdac.in>
Subject: Re: [Moses-support] Fwd: Fw: Warning: Too many arguments
while IRSTLM language model Training
To: bhaddow@staffmail.ed.ac.uk
Cc: moses-support@mit.edu
Message-ID:
<733136324.17867.1386347952581.JavaMail.open-xchange@webmail.cdac.in>
Content-Type: text/plain; charset="utf-8"

Hi,

Yes, the command with just '--text' option without a 'yes' works fine. Also the
next command of converting from arpa to binary file also works fine.


Thanks a lot.

Renu



On December 6, 2013 at 9:52 PM Renu Kumar <renu17775@gmail.com> wrote:

>
>
> ---------- Forwarded message ----------
> From: Renu Balyan <renubalyan@cdac.in <mailto:renubalyan@cdac.in> >
> Date: Fri, Dec 6, 2013 at 9:50 AM
> Subject: Fw: [Moses-support] Warning: Too many arguments while IRSTLM
> language model Training
> To: renu kumar < renu17775@gmail.com <mailto:renu17775@gmail.com> >
>
>
>
> ----- Original Message -----
> From: Barry Haddow <mailto:bhaddow@staffmail.ed.ac.uk>
> To: renubalyan <mailto:renubalyan@cdac.in> ; moses-support@mit.edu
> <mailto:moses-support@mit.edu>
> Sent: Friday, December 06, 2013 2:49 AM
> Subject: Re: [Moses-support] Warning: Too many arguments while IRSTLM
> language model Training
>
> Hi
>
> It looks like you are following the Moses baseline instructions (
> http://www.statmt.org/moses/?n=Moses.Baseline
> <http://www.statmt.org/moses/?n=Moses.Baseline> ). It's not explained, but
> step 5 should convert the IRSTLM iARPA file produced by step 4 to a (standard)
> ARPA file. The following step will then binarise it with KenLM.
>
> The command you ran is
>
> /home/renu/Desktop/irstlm/bin/compile-lm --text yes
> news-commentary-v8.fr-en.lm.en.gz news-commentary-v8.fr-en.arpa.en
>
> I notice that someone added a "yes" to this command in the documentation
> recently (November 13th). Does it work if you don't include "yes"?
>
> IRSTLM folks - can you clarify? Does the '--text' parameter require a 'yes'
> argument? The usage for the command suggests it does, but it used to work
> without,
>
> cheers - Barry
>
> On 04/12/13 15:58, renubalyan wrote:
>
> > > Hi,
> >
> > I am building the baseline system based on Moses manual instructions.
> >
> > I have installed Moses, GIZA++ and IRSTLM as mentioned in the manual.
> > The corpus preparation (tokenization, ...cleaning) steps also goes
> > well.
> >
> > However when I move to Language Model Training: I have some problems
> >
> > I am following these steps:
> >
> > 1. mkdir ~/lm
> >
> > 2. cd ~/lm
> >
> > 3. /home/renu/Desktop/irstlm/bin/add-start-end.sh <
> > /home/renu/Desktop/corpus/news-commentary-v8.fr-en.true.en>
> > news-commentary-v8.fr-en.sb.en
> >
> > 4. export IRSTLM=/home/renu/Desktop/irstlm;
> > /home/renu/Desktop/irstlm/bin/build-lm.sh -i news-commentary-v8.fr-en.sb.en
> > -t ./tmp -p -s improved-kneser-ney -o news-commentary-v8.fr-en.lm.en
> >
> > 5. /home/renu/Desktop/irstlm/bin/compile-lm --text yes
> > news-commentary-v8.fr-en.lm.en.gz news-commentary-v8.fr-en.arpa.en
> >
> > Steps 1-4 work well but step 5 gives me -------(Warning:Too many
> > parameters)
> >
> >
> > I have searched the web for any possible solution but could not find
> > any.
> >
> > I am not able to move ahead, kindly help.
> >
> > Thanks
> > Renu
> >
> >
> > -------------------------------------------------------------------------------------------------------------------------------
> > This e-mail is for the sole use of the intended recipient(s) and may
> > contain confidential and privileged information. If you are not the
> > intended recipient, please contact the sender by reply e-mail and
> > destroy
> > all copies and the original message. Any unauthorized review, use,
> > disclosure, dissemination, forwarding, printing or copying of this
> > email
> > is strictly prohibited and appropriate legal action will be taken.
> >
> > -------------------------------------------------------------------------------------------------------------------------------
> >
> >
> > _______________________________________________
> > Moses-support mailing list
> > Moses-support@mit.edu <mailto:Moses-support@mit.edu>
> > http://mailman.mit.edu/mailman/listinfo/moses-support
> > <http://mailman.mit.edu/mailman/listinfo/moses-support>
> >
> > >
>
> -------------------------------------------------------------------------------------------------------------------------------
> This e-mail is for the sole use of the intended recipient(s) and may
> contain confidential and privileged information. If you are not the
> intended recipient, please contact the sender by reply e-mail and destroy
> all copies and the original message. Any unauthorized review, use,
> disclosure, dissemination, forwarding, printing or copying of this email
> is strictly prohibited and appropriate legal action will be taken.
>
> -------------------------------------------------------------------------------------------------------------------------------
>

-------------------------------------------------------------------------------------------------------------------------------

This e-mail is for the sole use of the intended recipient(s) and may
contain confidential and privileged information. If you are not the
intended recipient, please contact the sender by reply e-mail and destroy
all copies and the original message. Any unauthorized review, use,
disclosure, dissemination, forwarding, printing or copying of this email
is strictly prohibited and appropriate legal action will be taken.
-------------------------------------------------------------------------------------------------------------------------------

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20131206/60123272/attachment.htm

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 86, Issue 18
*********************************************

Tweet