Send Moses-support mailing list submissions to
moses-support@mit.edu
To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu
You can reach the person managing the list at
moses-support-owner@mit.edu
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."
Today's Topics:
1. CFP: LREC Workshop on Automatic and Manual Metrics for
Operational Translation Evaluation (Lucia Specia)
2. Re: No bilingual concordancer in EMS? (Olalekan Houdan ABOU BAKAR)
3. Re: No bilingual concordancer in EMS? (Hieu Hoang)
4. filter parallel corpus (Saeed Farzi)
----------------------------------------------------------------------
Message: 1
Date: Thu, 16 Jan 2014 10:21:20 +0000
From: Lucia Specia <lspecia@gmail.com>
Subject: [Moses-support] CFP: LREC Workshop on Automatic and Manual
Metrics for Operational Translation Evaluation
To: mt-list@eamt.org, moses-support@mit.edu,
wmt-tasks@googlegroups.com
Message-ID:
<CAALeUxwG+mA2sYb-a2xMFOyhAvwXggeZEMFqc=DMO3ixmD8WNA@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
CALL FOR PAPERS
Automatic and Manual Metrics for Operational Translation Evaluation
Workshop at Language Resources and Evaluation Conference (LREC) 2014
(http://mte2014.github.io/)
While a significant body of work has been done by the machine translation
(MT) research community towards the development and meta-evaluation of
automatic metrics to assess overall MT quality, less attention has been
dedicated to more operational evaluation metrics aimed at testing whether
translations are adequate within a specific context: purpose, end-user,
task, etc., and why the MT system fails in some cases. Both of these can
benefit from some form of manual analysis. Most work in this area is
limited to productivity tests (e.g. contrasting time for human translation
and MT post-editing). A few initiatives consider more detailed metrics for
the problem, which can also be used to understand and diagnose errors in MT
systems. These include the Multidimensional Quality Metrics (MQM) recently
proposed by the EU F7 project QTLaunchPad, the TAUS Dynamic Quality
Framework, and past projects such as FEMTI, EAGLES and ISLE. Some of these
metrics are also applicable to human translation evaluation. A number of
task-based metrics have also been proposed for applications such as topic
ID / triage and reading comprehension.
The purpose of this workshop is to bring together representatives from
academia, industry and government institutions to discuss and assess
metrics for manual quality evaluation and compare them through correlation
analysis with well-established metrics for automatic evaluation such as
BLEU, METEOR and others, as well as reference-less metrics for quality
prediction.
The workshop will benefit from datasets already collected and manually
annotated for translation errors by the QTLaunchPad project as part of a
shared task on error annotation and automatic quality translation.
Submissions: We will accept two types of submissions:
1. Abstract (of up to one page)
2. One-page abstract plus full paper (6-10 pages)
Both abstracts and full papers will address any of the topics included in
this CFP (see below), but full papers have the advantage of presenting the
authors? work and ideas at a greater level of detail. Both abstract
submissions and abstract + paper submissions must be received by the
submission deadline below and will be reviewed by experts in the field.
Short slots for oral presentation will be given to all accepted
submissions, regardless of their format (abstract only or abstract + full
paper).
Topics: The workshop welcomes submissions on the topics of
- task-based translation evaluation metrics,
? specifically, metrics for machine (and/or human) translation quality
evaluation and quality estimation, be these metrics automatic,
semi-automatic or manual (rubric, error annotation, etc.),
- error analysis of machine (and human) translations (automated and manual),
? for example studies exploiting whether manually annotated translations
can contribute to the automatic detection of specific translation errors
and whether this can be used to automatically correct translations
- correlation between translation evaluation metrics, error analysis, and
task-suitability of translations.
The format of the workshop will be a half-day of short presentations on the
above topics, followed by a half-day of hands-on collaborative work with MT
metrics that show promise for the prediction of task suitability of MT
output. The afternoon hands-on work will follow from the morning?s
presentations. Thus, all submissions, both abstracts and abstracts + papers
should address at least the following points:
? definition of the metric(s) being proposed, along with an indication of
whether the metric is manual or automated,
? method of computation of the metric(s), if not already well-known,
? discussion of the applicability of the metric(s) to determining task
suitability of MT output, and
? indication of human (annotation) effort necessary to produce the
metric(s).
Submissions must be made via the START Conference Manager at
https://www.softconf.com/lrec2014/MTE/. Email submissions will not be
reviewed.
Share your LRs: When making a submission from the START page, authors will
be asked to provide essential information about resources (in a broad
sense, i.e. also technologies, standards, evaluation kits, etc.) that have
been used for the work described in the paper or are a new result of your
research. Moreover, ELRA encourages all LREC authors to share the described
LRs (data, tools, services, etc.), to enable their reuse, replicability of
experiments, including evaluation ones, etc.
Important Dates:
Submission of Abstract or Abstract plus Paper: February 7, 2014
[NOTE: Author(s) intending to submit a full paper must submit the full
paper along with the abstract in order for it to be considered for
inclusion in the workshop and publication in the workshop proceedings.]
Notification to authors: March 10, 2014
Camera-ready versions of accepted Abstract or Abstract plus Paper due to
organizing committee: March 28, 2014
Workshop Date: May 26, 2014
Organizing Committee: Keith J. Miller (MITRE), Lucia Specia (University of
Sheffield), Kim Harris (GALA and text & form), Stacey Bailey (MITRE)
--
Lucia
www.dcs.shef.ac.uk/~lucia/
--
Lucia
www.dcs.shef.ac.uk/~lucia/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20140116/484814c2/attachment-0001.htm
------------------------------
Message: 2
Date: Thu, 16 Jan 2014 06:15:18 -0500
From: Olalekan Houdan ABOU BAKAR <houdan@jolome.com>
Subject: Re: [Moses-support] No bilingual concordancer in EMS?
To: moses-support@mit.edu
Message-ID:
<CALcA+R09wvTphdg+xQEKkzPmKU2JaMtgJ771a=Neoc7X-RVG9g@mail.gmail.com>
Content-Type: text/plain; charset="iso-8859-1"
Ok, turns out biconcor is distributed with moses.. It is
/moses_src/biconcor
I was looking for it under ems as the EMS page says. Think about making
the changes on that page.
Thanks!
--
Ola
2014/1/16 Olalekan Houdan ABOU BAKAR <houdan@jolome.com>
> Trying to run the EMS script against the toy config file..
>
> I'd like the bilingual concordancer feature by enabling it the config file
> as follows:
>
> biconcor = $moses-script-dir/ems/biconcor/biconcor
>
> but moses isn't distributed with one.
>
> Which concordance tool did you find compatible with moses and the EMS web
> interface?
>
>
> Cheers
>
> --
>
> Ola
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20140116/e61a300e/attachment-0001.htm
------------------------------
Message: 3
Date: Thu, 16 Jan 2014 11:23:01 +0000
From: Hieu Hoang <Hieu.Hoang@ed.ac.uk>
Subject: Re: [Moses-support] No bilingual concordancer in EMS?
To: Olalekan Houdan ABOU BAKAR <houdan@jolome.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAEKMkbjtb=ri6Yiwh9SXUNqf4LQOssF3RpG02X23ZghK29fCNA@mail.gmail.com>
Content-Type: text/plain; charset="iso-8859-1"
thanks. I've changed the page
On 16 January 2014 11:15, Olalekan Houdan ABOU BAKAR <houdan@jolome.com>wrote:
> Ok, turns out biconcor is distributed with moses.. It is
>
> /moses_src/biconcor
>
> I was looking for it under ems as the EMS page says. Think about making
> the changes on that page.
>
> Thanks!
>
> --
>
> Ola
>
>
>
> 2014/1/16 Olalekan Houdan ABOU BAKAR <houdan@jolome.com>
>
>> Trying to run the EMS script against the toy config file..
>>
>> I'd like the bilingual concordancer feature by enabling it the config
>> file as follows:
>>
>> biconcor = $moses-script-dir/ems/biconcor/biconcor
>>
>> but moses isn't distributed with one.
>>
>> Which concordance tool did you find compatible with moses and the EMS web
>> interface?
>>
>>
>> Cheers
>>
>> --
>>
>> Ola
>>
>
>
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
--
Hieu Hoang
Research Associate
University of Edinburgh
http://www.hoang.co.uk/hieu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20140116/b1b89581/attachment-0001.htm
------------------------------
Message: 4
Date: Thu, 16 Jan 2014 19:13:25 +0330
From: Saeed Farzi <saeedfarzi@gmail.com>
Subject: [Moses-support] filter parallel corpus
To: "corpora@uib.no" <corpora@uib.no>, moses-support
<moses-support@mit.edu>
Message-ID:
<CAOhrzg6rGmxG5faSadEjD1a4vYVePHL2fGviA6wwkuBcg_mtNA@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1
Dear all,
I am working on a translation task with a very large parallel corpus.
Because of computational cost of training such a parallel corpus, i am
going to filter it regarding to the test set ( of course , by the
filtering, the evaluation must be still fair).
I am looking for a solution or a tool for filtering parallel corpus sentences.
Note that i do not need to filter phrase table. I know that the
filter_ moses tool reduces the phrase table size.
cheers
--
S.Farzi, Ph.D. Student
Natural Language Processing Lab,
School of Electrical and Computer Eng.,
Tehran University
Tel: +9821-6111-9719
------------------------------
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
End of Moses-support Digest, Vol 87, Issue 40
*********************************************
Subscribe to:
Post Comments (Atom)
0 Response to "Moses-support Digest, Vol 87, Issue 40"
Post a Comment