Moses-support Digest, Vol 98, Issue 49

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: bjam level of parallelism for various moses binaries
(Hieu Hoang)
2. Re: bjam level of parallelism for various moses binaries
(Hieu Hoang)
3. Re: bjam level of parallelism for various moses binaries
(Christian Hardmeier)
4. Re: bjam level of parallelism for various moses binaries
(Kenneth Heafield)


----------------------------------------------------------------------

Message: 1
Date: Fri, 19 Dec 2014 20:00:59 +0530
From: Hieu Hoang <Hieu.Hoang@ed.ac.uk>
Subject: Re: [Moses-support] bjam level of parallelism for various
moses binaries
To: Kenneth Heafield <moses@kheafield.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAEKMkbgF7pLAAo1SVWHCxoARfi6OaSF0aUa779s08xrM67ArgA@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

What's the solution? Sort out header files? Ditch bjam?

Sent while bumping into things
On 19 Dec 2014 19:10, "Kenneth Heafield" <moses@kheafield.com> wrote:

> You can run bjam and name a specific target. It's also fair to complain
> that these executables draw in way more dependencies than they need,
> which is slowing down compilation.
>
> Kenneth
>
> On 12/19/2014 08:30 AM, Prashant Mathur wrote:
> > Adding to this..
> > Can we have a mode for selective compilation for development purpose?
> > For building moses executable, we don't need these relax-parse,
> > consolidate-parse, extract .. and so on..
> >
> > On Fri, Dec 19, 2014 at 1:54 PM, Martin Li?ka <marxin.liska@gmail.com
> > <mailto:marxin.liska@gmail.com>> wrote:
> >
> > Hello.
> >
> > I've been just building Moses with enabled LTO (-flto) and
> compilation
> > process runs for quite long time just in one thread:
> >
> > There're are many of:
> >
> > gcc.link
> >
> phrase-extract/bin/gcc-5.0.0/release/debug-symbols-on/link-static/threading-multi/relax-parse
> > gcc.link
> >
> phrase-extract/bin/gcc-5.0.0/release/debug-symbols-on/link-static/threading-multi/consolidate-reverse
> >
> phrase-extract/bin/gcc-5.0.0/release/debug-symbols-on/link-static/threading-multi/extract
> >
> > ...
> >
> > And my question is if there are any dependencies between these
> > executables that potentially block parallel build?
> >
> > Thanks,
> > Martin
> > _______________________________________________
> > Moses-support mailing list
> > Moses-support@mit.edu <mailto:Moses-support@mit.edu>
> > http://mailman.mit.edu/mailman/listinfo/moses-support
> >
> >
> >
> >
> > _______________________________________________
> > Moses-support mailing list
> > Moses-support@mit.edu
> > http://mailman.mit.edu/mailman/listinfo/moses-support
> >
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20141219/8a9f1aa4/attachment-0001.htm

------------------------------

Message: 2
Date: Fri, 19 Dec 2014 20:09:32 +0530
From: Hieu Hoang <Hieu.Hoang@ed.ac.uk>
Subject: Re: [Moses-support] bjam level of parallelism for various
moses binaries
To: Kenneth Heafield <moses@kheafield.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID:
<CAEKMkbitT+sBSGLB8v81hRmwWYWEWKTd9UMX0ch23jVEqXnXOg@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

It seems like a lot of the programs now depend on the moses lib so any
change to it trigger mass relinking.

It may be that we should separate out the functionality into different lib
files.

Sent while bumping into things
On 19 Dec 2014 20:00, "Hieu Hoang" <Hieu.Hoang@ed.ac.uk> wrote:

> What's the solution? Sort out header files? Ditch bjam?
>
> Sent while bumping into things
> On 19 Dec 2014 19:10, "Kenneth Heafield" <moses@kheafield.com> wrote:
>
>> You can run bjam and name a specific target. It's also fair to complain
>> that these executables draw in way more dependencies than they need,
>> which is slowing down compilation.
>>
>> Kenneth
>>
>> On 12/19/2014 08:30 AM, Prashant Mathur wrote:
>> > Adding to this..
>> > Can we have a mode for selective compilation for development purpose?
>> > For building moses executable, we don't need these relax-parse,
>> > consolidate-parse, extract .. and so on..
>> >
>> > On Fri, Dec 19, 2014 at 1:54 PM, Martin Li?ka <marxin.liska@gmail.com
>> > <mailto:marxin.liska@gmail.com>> wrote:
>> >
>> > Hello.
>> >
>> > I've been just building Moses with enabled LTO (-flto) and
>> compilation
>> > process runs for quite long time just in one thread:
>> >
>> > There're are many of:
>> >
>> > gcc.link
>> >
>> phrase-extract/bin/gcc-5.0.0/release/debug-symbols-on/link-static/threading-multi/relax-parse
>> > gcc.link
>> >
>> phrase-extract/bin/gcc-5.0.0/release/debug-symbols-on/link-static/threading-multi/consolidate-reverse
>> >
>> phrase-extract/bin/gcc-5.0.0/release/debug-symbols-on/link-static/threading-multi/extract
>> >
>> > ...
>> >
>> > And my question is if there are any dependencies between these
>> > executables that potentially block parallel build?
>> >
>> > Thanks,
>> > Martin
>> > _______________________________________________
>> > Moses-support mailing list
>> > Moses-support@mit.edu <mailto:Moses-support@mit.edu>
>> > http://mailman.mit.edu/mailman/listinfo/moses-support
>> >
>> >
>> >
>> >
>> > _______________________________________________
>> > Moses-support mailing list
>> > Moses-support@mit.edu
>> > http://mailman.mit.edu/mailman/listinfo/moses-support
>> >
>> _______________________________________________
>> Moses-support mailing list
>> Moses-support@mit.edu
>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20141219/83badecc/attachment-0001.htm

------------------------------

Message: 3
Date: Fri, 19 Dec 2014 15:50:28 +0100
From: Christian Hardmeier <ch@rax.ch>
Subject: Re: [Moses-support] bjam level of parallelism for various
moses binaries
To: Hieu Hoang <Hieu.Hoang@ed.ac.uk>
Cc: moses-support <moses-support@mit.edu>
Message-ID: <4764E826-EDE4-4A18-A1F4-B8405C6B24EB@rax.ch>
Content-Type: text/plain; charset=windows-1252

Doing so has the potential to make linking external code to moses a nightmare because of library interdependencies, unless it's really well designed.

Just a word of warning. :)
Christian

On Dec 19, 2014, at 3:39 PM, Hieu Hoang wrote:

> It seems like a lot of the programs now depend on the moses lib so any change to it trigger mass relinking.
>
> It may be that we should separate out the functionality into different lib files.
>
> Sent while bumping into things
>
> On 19 Dec 2014 20:00, "Hieu Hoang" <Hieu.Hoang@ed.ac.uk> wrote:
> What's the solution? Sort out header files? Ditch bjam?
>
> Sent while bumping into things
>
> On 19 Dec 2014 19:10, "Kenneth Heafield" <moses@kheafield.com> wrote:
> You can run bjam and name a specific target. It's also fair to complain
> that these executables draw in way more dependencies than they need,
> which is slowing down compilation.
>
> Kenneth
>
> On 12/19/2014 08:30 AM, Prashant Mathur wrote:
> > Adding to this..
> > Can we have a mode for selective compilation for development purpose?
> > For building moses executable, we don't need these relax-parse,
> > consolidate-parse, extract .. and so on..
> >
> > On Fri, Dec 19, 2014 at 1:54 PM, Martin Li?ka <marxin.liska@gmail.com
> > <mailto:marxin.liska@gmail.com>> wrote:
> >
> > Hello.
> >
> > I've been just building Moses with enabled LTO (-flto) and compilation
> > process runs for quite long time just in one thread:
> >
> > There're are many of:
> >
> > gcc.link
> > phrase-extract/bin/gcc-5.0.0/release/debug-symbols-on/link-static/threading-multi/relax-parse
> > gcc.link
> > phrase-extract/bin/gcc-5.0.0/release/debug-symbols-on/link-static/threading-multi/consolidate-reverse
> > phrase-extract/bin/gcc-5.0.0/release/debug-symbols-on/link-static/threading-multi/extract
> >
> > ...
> >
> > And my question is if there are any dependencies between these
> > executables that potentially block parallel build?
> >
> > Thanks,
> > Martin
> > _______________________________________________
> > Moses-support mailing list
> > Moses-support@mit.edu <mailto:Moses-support@mit.edu>
> > http://mailman.mit.edu/mailman/listinfo/moses-support
> >
> >
> >
> >
> > _______________________________________________
> > Moses-support mailing list
> > Moses-support@mit.edu
> > http://mailman.mit.edu/mailman/listinfo/moses-support
> >
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu
> http://mailman.mit.edu/mailman/listinfo/moses-support




------------------------------

Message: 4
Date: Fri, 19 Dec 2014 10:02:10 -0500
From: Kenneth Heafield <moses@kheafield.com>
Subject: Re: [Moses-support] bjam level of parallelism for various
moses binaries
To: Hieu Hoang <Hieu.Hoang@ed.ac.uk>
Cc: moses-support <moses-support@mit.edu>
Message-ID: <54943DF2.1080401@kheafield.com>
Content-Type: text/plain; charset=utf-8

relax-parse (an example) shouldn't have to depend on lib moses (I don't
think it cares about a reordering table implementation for instance).
If it needs something from that directory, that's a sign that files
should be pulled out of the moses directory into a base lib.

Kenneth

On 12/19/14 09:39, Hieu Hoang wrote:
> It seems like a lot of the programs now depend on the moses lib so any
> change to it trigger mass relinking.
>
> It may be that we should separate out the functionality into different
> lib files.
>
> Sent while bumping into things
>
> On 19 Dec 2014 20:00, "Hieu Hoang" <Hieu.Hoang@ed.ac.uk
> <mailto:Hieu.Hoang@ed.ac.uk>> wrote:
>
> What's the solution? Sort out header files? Ditch bjam?
>
> Sent while bumping into things
>
> On 19 Dec 2014 19:10, "Kenneth Heafield" <moses@kheafield.com
> <mailto:moses@kheafield.com>> wrote:
>
> You can run bjam and name a specific target. It's also fair to
> complain
> that these executables draw in way more dependencies than they need,
> which is slowing down compilation.
>
> Kenneth
>
> On 12/19/2014 08:30 AM, Prashant Mathur wrote:
> > Adding to this..
> > Can we have a mode for selective compilation for development
> purpose?
> > For building moses executable, we don't need these relax-parse,
> > consolidate-parse, extract .. and so on..
> >
> > On Fri, Dec 19, 2014 at 1:54 PM, Martin Li?ka
> <marxin.liska@gmail.com <mailto:marxin.liska@gmail.com>
> > <mailto:marxin.liska@gmail.com
> <mailto:marxin.liska@gmail.com>>> wrote:
> >
> > Hello.
> >
> > I've been just building Moses with enabled LTO (-flto) and
> compilation
> > process runs for quite long time just in one thread:
> >
> > There're are many of:
> >
> > gcc.link
> >
> phrase-extract/bin/gcc-5.0.0/release/debug-symbols-on/link-static/threading-multi/relax-parse
> > gcc.link
> >
> phrase-extract/bin/gcc-5.0.0/release/debug-symbols-on/link-static/threading-multi/consolidate-reverse
> >
> phrase-extract/bin/gcc-5.0.0/release/debug-symbols-on/link-static/threading-multi/extract
> >
> > ...
> >
> > And my question is if there are any dependencies between these
> > executables that potentially block parallel build?
> >
> > Thanks,
> > Martin
> > _______________________________________________
> > Moses-support mailing list
> > Moses-support@mit.edu <mailto:Moses-support@mit.edu>
> <mailto:Moses-support@mit.edu <mailto:Moses-support@mit.edu>>
> > http://mailman.mit.edu/mailman/listinfo/moses-support
> >
> >
> >
> >
> > _______________________________________________
> > Moses-support mailing list
> > Moses-support@mit.edu <mailto:Moses-support@mit.edu>
> > http://mailman.mit.edu/mailman/listinfo/moses-support
> >
> _______________________________________________
> Moses-support mailing list
> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
> http://mailman.mit.edu/mailman/listinfo/moses-support
>


------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 98, Issue 49
*********************************************

Tweet