Send Moses-support mailing list submissions to
moses-support@mit.edu
To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu
You can reach the person managing the list at
moses-support-owner@mit.edu
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."
Today's Topics:
1. problem with tuning set stalling (Alexander Erdmann)
2. Fwd: Researcher in Machine Learning and Natural Language
Processing at DFKI (Cristina Espa?a-Bonet)
----------------------------------------------------------------------
Message: 1
Date: Mon, 13 Mar 2017 08:56:18 +0400
From: Alexander Erdmann <ae1541@nyu.edu>
Subject: [Moses-support] problem with tuning set stalling
To: moses-support@mit.edu
Message-ID:
<CAGTOGBePCpi_gbCgx-c351ywpuQbXnac5QsSUx2Ae5byyc2P4g@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Hi,
I'm trying to do pivot translation between Arabic dialects via English. The
Egyptian to English side trains and tunes fine in about 4 hours, but the
English to Levantine side (although it is of comparable size and
preprocessed in exactly the same way) stalls before completing the first
run during the tuning step (training finished without error and the tuning
step never yields an error, it just never finishes the first run).
There are about 175,000 sentences in the training corpus for English to
Levantine, all of this data coming from weblogs used in the BOLT corpus.
The LM for Levantine is mostly comprised of the same data, with some
additional data news commentary and twitter. I tried tuning originally with
2000 sentences from BOLT, then 1000, and finally 500, but I ran into the
same stall each time while the Egyptian - English side never had an issue.
Do you have any idea what is going on or how to resolve this?
Here is the output file:
run 1 start at Sun Mar 12 23:26:52 GST 2017
Parsing --decoder-flags: |-v 0|
Saving new config to: ./run1.moses.ini
(1) run decoder to produce n-best lists
params = -v 0
decoder_config = -weight-overwrite 'PhrasePenalty0= 0.043478 WordPenalty0=
-0.217391 TranslationModel0= 0.043478 0.043478 0.043478 0.043478
Distortion0= 0.065217 LM0= 0.108696 LexicalReordering0= 0.065217 0.065217
0.065217 0.065217 0.065217 0.065217'
and here is the error file:
Loading module 'mosesdecoder/3.0'
Loading module 'mgiza/2015.01'
mkdir: cannot create directory
Using SCRIPTS_ROOTDIR: /share/apps/NYUAD3/mosesdecoder/3.0/scripts
Assuming the tables are already filtered, reusing filtered/moses.ini
Using cached features list: ./features.list
MERT starting values and ranges for random generation:
LexicalReordering0 = 0.300 ( 0.00 .. 1.00)
LexicalReordering0 = 0.300 ( 0.00 .. 1.00)
LexicalReordering0 = 0.300 ( 0.00 .. 1.00)
LexicalReordering0 = 0.300 ( 0.00 .. 1.00)
LexicalReordering0 = 0.300 ( 0.00 .. 1.00)
LexicalReordering0 = 0.300 ( 0.00 .. 1.00)
Distortion0 = 0.300 ( 0.00 .. 1.00)
LM0 = 0.500 ( 0.00 .. 1.00)
WordPenalty0 = -1.000 ( 0.00 .. 1.00)
PhrasePenalty0 = 0.200 ( 0.00 .. 1.00)
TranslationModel0 = 0.200 ( 0.00 .. 1.00)
TranslationModel0 = 0.200 ( 0.00 .. 1.00)
TranslationModel0 = 0.200 ( 0.00 .. 1.00)
TranslationModel0 = 0.200 ( 0.00 .. 1.00)
featlist: LexicalReordering0=0.300000
featlist: LexicalReordering0=0.300000
featlist: LexicalReordering0=0.300000
featlist: LexicalReordering0=0.300000
featlist: LexicalReordering0=0.300000
featlist: LexicalReordering0=0.300000
featlist: Distortion0=0.300000
featlist: LM0=0.500000
featlist: WordPenalty0=-1.000000
featlist: PhrasePenalty0=0.200000
featlist: TranslationModel0=0.200000
featlist: TranslationModel0=0.200000
featlist: TranslationModel0=0.200000
featlist: TranslationModel0=0.200000
Saved: ./run1.moses.ini
Normalizing lambdas: 0.300000 0.300000 0.300000 0.300000 0.300000 0.300000
0.300000 0.500000 -1.000000 0.200000 0.200000 0.200000 0.200000 0.200000
DECODER_CFG = -weight-overwrite 'PhrasePenalty0= 0.043478 WordPenalty0=
-0.217391 TranslationModel0= 0.043478 0.043478 0.043478 0.043478
Distortion0= 0.065217 LM0= 0.108696 LexicalReordering0= 0.065217 0.065217
0.065217 0.065217 0.065217 0.065217'
Executing: /share/apps/NYUAD/mosesdecoder/3.0/bin/moses -v 0 -config
filtered/moses.ini -weight-overwrite 'PhrasePenalty0= 0.043478
WordPenalty0= -0.217391 TranslationModel0= 0.043478 0.043478 0.043478
0.043478 Distortion0= 0.065217 LM0= 0.108696 LexicalReordering0= 0.065217
0.065217 0.065217 0.065217 0.065217 0.065217' -n-best-list
run1.best100.out 100 distinct -input-file
/scratch/ae1541/unComparableCorpora/evaluation/silverPhrases/eng-lev/train.bolt.lev.eng
> run1.out
Executing: /share/apps/NYUAD/mosesdecoder/3.0/bin/moses -v 0 -config
filtered/moses.ini -weight-overwrite 'PhrasePenalty0= 0.043478
WordPenalty0= -0.217391 TranslationModel0= 0.043478 0.043478 0.043478
0.043478 Distortion0= 0.065217 LM0= 0.108696 LexicalReordering0= 0.065217
0.065217 0.065217 0.065217 0.065217 0.065217' -n-best-list
run1.best100.out 100 distinct -input-file
/scratch/ae1541/unComparableCorpora/evaluation/silverPhrases/eng-lev/train.bolt.lev.eng
> run1.out
Initializing LexicalReordering..
Loading table into memory...done.
Thanks,
--
Alex Erdmann
PhD Student in Linguistics at The Ohio State University
Visiting Scholar at NYU Abu Dhabi
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20170313/53d3e06f/attachment-0001.html
------------------------------
Message: 2
Date: Mon, 13 Mar 2017 09:15:02 +0100
From: Cristina Espa?a-Bonet <cristinae.uni@gmail.com>
Subject: [Moses-support] Fwd: Researcher in Machine Learning and
Natural Language Processing at DFKI
To: moses-support <moses-support@mit.edu>
Message-ID:
<CAKehm8Agc48VyrmKBFmXbNfhaYXuXXoKS9SVWjJQ6UiP=+_JRg@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
[Apologies for crossposting]
*Researcher in Machine Learning and Natural Language Processing, DFKI,
German Research Center for Artificial Intelligence, Germany *
The Multilingual Technologies (MLT) Lab at DFKI is looking to expand its
expertise in Machine Learning with a focus on Deep Learning,
Machine Translation and possibly other areas of NLP. Depending on
experience, the position is available at the
Junior/Researcher/Senior/Principal Researcher level.
*Key research responsibilities* include:
- machine and deep learning for natural language processing/machine
translation
- software development and integration
- publication in top-tier conferences and journals
*General responsibilities* include:
- engagement with industry partners and contract research
- identification of funding opportunities and engagement in proposal
writing
- contribution to teaching and supervision in accordance with University
and DFKI rules and regulations
- administrative work associated with programmes of research
Requirements:
- MSc/PhD in computer science, machine learning, natural language
processing, computational linguistics or similar
- Strong background and track record in machine learning, neural nets and
deep learning
- Strong background and track record in NLP and MT - Excellent programming
skills
- Excellent problem solving skills, independent and creative thinking
- Excellent team working and communication skills
- Excellent command of written and oral English
- Command of German and other languages not a requirement but helpful
The successful applicant will work in the DFKI MLT lab led by Prof. Josef
van Genabith (Scientific Director MLT, DFKI, and Chair of
Translation-Oriented Language Technologies, Saarland University).
*Working environment*:
DFKI is one of the largest AI research institutes worldwide, with several
sites in Germany, covering basic research and applications.
DFKI is a not-for-profit company with more than 500 researchers from 60+
countries across the globe. DFKI is based on a shareholder model
including globally operating companies such as Intel, Google, Microsoft,
Nuance, SAP, BMW, VW, Bosch, Deutsche Telekom, several
SMEs, three German universities and three German Federal States.
The DFKI Multilingual Technologies lab partners in international, national
and industry funded research projects in all areas of
Language Technologies (including machine translation, question answering,
information extraction, human-robot communication, speech
and the multi-lingual web). The MLT lab currently leads the H2020 European
Research project QT21 (http://www.qt21.eu/ ) on MT, the EU
CEF funded ELRC project (http://lr-coordination.eu/ ) and the EU funded
TRADR project on human-robot collaboration in disaster
scenarios (http://www.tradr-project.eu/ ).
The MLT lab is part of the DFKI site at the Saarland University campus in
Saarbr?cken, Germany. Saarland University has exceptionally
strong Computer Science and Computational Linguistics departments, two Max
Plank Institutes in Computer Science, an Excellence Cluster
in Multimodal Computing and Interaction (http://www.mmci.uni-saarland.
de/en/start) and several International Doctoral and
Master programmes in Computer Science and Computational Linguistics. DFKI
staff regularly engage in teaching and supervision
at Saarland University.
*Geographical environment*:
Saarbr?cken (http://www.saarbruecken.de/en) is the capital of Saarland with
approximately 190,000 inhabitants. It is located right
in the heart of Europe and is the cultural center of this border region of
Germany, France and Luxembourg. Some of the closest larger
cities are Trier, Nancy, Mannheim, Karlsruhe and Frankfurt. Paris can be
reached by train in just under 2 hours. Living costs are modest in
comparison with other large cities in Germany and elsewhere in Europe.
*Starting date, duration, salary*:
Preferred starting date is May/June 2017. The position is available until
June 30, 2020, with opportunities for extension depending on
performance and future funding. Compensation is competitive and reflects
individual competence, seniority and special skills.
*Application*:
Applications are required to include a short cover letter, a CV, list of
publications, a brief summary of research interests, and contact
information for three references. Please send your electronic application
(preferably in PDF format) to mlt-sek@dfki.de referring
to job opening no. "22/17-JvG". Deadline for applications is March 31st,
2017. The position remains open until filled. Please contact
josef.van_genabith@dfki.de for informal inquiries.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20170313/f66dab50/attachment.html
------------------------------
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support
End of Moses-support Digest, Vol 125, Issue 22
**********************************************
Subscribe to:
Post Comments (Atom)
0 Response to "Moses-support Digest, Vol 125, Issue 22"
Post a Comment