Moses-support Digest, Vol 124, Issue 24

Send Moses-support mailing list submissions to
moses-support@mit.edu

To subscribe or unsubscribe via the World Wide Web, visit
http://mailman.mit.edu/mailman/listinfo/moses-support
or, via email, send a message with subject or body 'help' to
moses-support-request@mit.edu

You can reach the person managing the list at
moses-support-owner@mit.edu

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Moses-support digest..."


Today's Topics:

1. Re: feature function referring headers in phrase-extract?
(Kenneth Heafield)
2. Re: feature function referring headers in phrase-extract?
(Shuoyang Ding)


----------------------------------------------------------------------

Message: 1
Date: Thu, 23 Feb 2017 17:52:18 +0000
From: Kenneth Heafield <moses@kheafield.com>
Subject: Re: [Moses-support] feature function referring headers in
phrase-extract?
To: Shuoyang Ding <mtsding@gmail.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID: <b6244299-5f16-10b0-5eec-9a62ab3a16c9@kheafield.com>
Content-Type: text/plain; charset=utf-8

Looks like you're compiling a main function into the moses library.

Keep anything containing "int main" in phrase-extract.

On 02/23/2017 04:59 PM, Shuoyang Ding wrote:
> Hi,
>
> I did what Ken described and looks like I?m almost there. The only
> problem now is that I keep getting this weird message about unit test. I
> attached the build log in the email.
>
> It seems like the build system is trying to test binary extract-ghkm
> after compilation but there is no way for it to know how to do it.
>
> Is there anyway I can remove this unit test for the moment?
>
> Shuoyang
>
>
>
>
>> On Feb 22, 2017, at 1:35 PM, Kenneth Heafield <moses@kheafield.com
>> <mailto:moses@kheafield.com>> wrote:
>>
>> The main moses target already includes moses/*.cpp (with some exceptions
>> that you don't care about). So just moving files should work albeit
>> hackily.
>>
>> Kenneth
>>
>> On 02/22/2017 05:24 PM, Shuoyang Ding wrote:
>>> Hi,
>>>
>>> Thanks for the timely reply.
>>>
>>> To make sure I?m doing this correctly, here are some more questions:
>>>
>>> 1) should the moved files in phrase-extract/ be added to moses/Jamfile
>>> so they?ll be compiled?
>>> 2) shall I move the Jamfiles in phrase-extract/ and edit the paths in
>>> them as appropriate?
>>>
>>> Shuoyang
>>>
>>>> On Feb 22, 2017, at 12:15 PM, Hieu Hoang <hieuhoang@gmail.com
>>>> <mailto:hieuhoang@gmail.com>
>>>> <mailto:hieuhoang@gmail.com>> wrote:
>>>>
>>>> apologies, ken pointed out why there are multiple definitions. You
>>>> should be aware of that
>>>>
>>>> Hieu Hoang
>>>> http://moses-smt.org/
>>>>
>>>> On 22 February 2017 at 17:11, Hieu Hoang <hieuhoang@gmail.com
>>>> <mailto:hieuhoang@gmail.com>> wrote:
>>>>
>>>> too complicated imo. Copy what u need into moses/ so you know
>>>> exactly what the dependencies are.
>>>>
>>>> Do what Ken suggested later when it's working
>>>>
>>>> Hieu Hoang
>>>> http://moses-smt.org/
>>>>
>>>> On 22 February 2017 at 17:10, Kenneth Heafield
>>>> <moses@kheafield.com <mailto:moses@kheafield.com>> wrote:
>>>>
>>>> Hi,
>>>>
>>>> phrase-extract depends on moses c.f.
>>>> phrase-extract/Jamfile:7.
>>>>
>>>> alias deps : $(most-deps:B).o ..//z ..//boost_iostreams
>>>> ..//boost_filesystem ../moses//moses ../moses//ThreadPool
>>>> ../moses//Util
>>>> ../util//kenutil ;
>>>>
>>>> So rather than copy, move it to moses. More cleanly, you
>>>> could extract
>>>> a target for the part you want to share and point both at it.
>>>>
>>>> Kenneth
>>>>
>>>> On 02/22/2017 04:57 PM, Shuoyang Ding wrote:
>>>>> Hi All,
>>>>>
>>>>> Sorry for bringing up this again after a long time. Last
>>>> time I worked
>>>>> around the problem by re-designing my feature function, which
>>>>> unfortunately does not apply this time.
>>>>>
>>>>> To quickly remind you what the problem was, I was developing
>>>> a feature
>>>>> function that involves some headers in phrase-extract/, and
>>>> the headers
>>>>> seems not correctly referenced.
>>>>>
>>>>> As Hieu suggested, I copied code in phrase-extract/ into
>>>> moses/ and it
>>>>> caused massive re-definition error so seems like that is not a
>>>>> straight-forward solution.
>>>>>
>>>>> Having read some feature functions, I noticed that a feature
>>>> function
>>>>> PhraseOrientation actually reference headers in
>>>> phrase-extract/, and it
>>>>> seems to work well. I?m wondering if it is actually easier to do
>>>>> something similar to that feature function (i.e. by changing
>>>> some bjam
>>>>> configurations) and which config files should I be looking at.
>>>>>
>>>>> Thanks!
>>>>>
>>>>> Regards,
>>>>> Shuoyang Ding
>>>>>
>>>>> Ph.D. Student
>>>>> Center for Language and Speech Processing
>>>>> Department of Computer Science
>>>>> Johns Hopkins University
>>>>>
>>>>> Hackerman Hall 225A
>>>>> 3400 N. Charles St.
>>>>> Baltimore, MD 21218
>>>>>
>>>>> http://cs.jhu.edu/~sding
>>>>>
>>>>>
>>>>>> On Apr 29, 2016, at 3:59 AM, Hieu Hoang
>>>> <hieuhoang@gmail.com <mailto:hieuhoang@gmail.com>
>>>>>> <mailto:hieuhoang@gmail.com <mailto:hieuhoang@gmail.com>>>
>>>> wrote:
>>>>>>
>>>>>> You should be careful of referencing files in
>>>> phrase-extract/ if you
>>>>>> are developing code that is in moses/
>>>>>>
>>>>>> The code in phrase-extract/ will not be linked.
>>>>>>
>>>>>> It's probably safer to simply copy what you need from
>>>> phrase-extract/
>>>>>> into moses/ and use the copy
>>>>>>
>>>>>> On 25/04/2016 17:16, Shuoyang Ding wrote:
>>>>>>> Hi all,
>>>>>>>
>>>>>>> I?m developing a feature function that involves some
>>>> headers in
>>>>>>> phrase-extract:
>>>>>>>
>>>>>>> phrase-extract/extract-ghkm/AlignmentGraph.h
>>>>>>> phrase-extract/extract-ghkm/Rule.h
>>>>>>> phrase-extract/extract-ghkm/ScfgRule.h
>>>>>>> phrase-extract/syntax-common/xml_tree_parser.h
>>>>>>>
>>>>>>> Now I?m having some issue with recompiling with this new
>>>> feature
>>>>>>> function ? these headers seems not correctly referenced. I
>>>> attached
>>>>>>> the build log with this email. Specifically:
>>>>>>>
>>>>>>> ./phrase-extract/extract-ghkm/AlignmentGraph.h:28:24:
>>>> fatal error:
>>>>>>> SyntaxTree.h: No such file or directory
>>>>>>>
>>>>>>> I guess I may have to change moses/Jamfile to accommodate this
>>>>>>> change. Is there anyone who is familiar with bjam who can
>>>> help me?
>>>>>>>
>>>>>>> Thanks!
>>>>>>>
>>>>>>> Regards,
>>>>>>> Shuoyang Ding
>>>>>>>
>>>>>>> Ph.D. Student
>>>>>>> Center for Language and Speech Processing
>>>>>>> Department of Computer Science
>>>>>>> Johns Hopkins University
>>>>>>>
>>>>>>> Hackerman Hall 225A
>>>>>>> 3400 N. Charles St.
>>>>>>> Baltimore, MD 21218
>>>>>>>
>>>>>>> http://cs.jhu.edu/~sding <http://cs.jhu.edu/%7Esding>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Moses-support mailing list
>>>>>>> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
>>>>>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>> <http://mailman.mit.edu/mailman/listinfo/moses-support>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Moses-support mailing list
>>>>> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
>>>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>> <http://mailman.mit.edu/mailman/listinfo/moses-support>
>>>>>
>>>> _______________________________________________
>>>> Moses-support mailing list
>>>> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
>>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>> <http://mailman.mit.edu/mailman/listinfo/moses-support>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Moses-support mailing list
>>>> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
>>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>
>


------------------------------

Message: 2
Date: Thu, 23 Feb 2017 13:08:45 -0500
From: Shuoyang Ding <mtsding@gmail.com>
Subject: Re: [Moses-support] feature function referring headers in
phrase-extract?
To: Kenneth Heafield <moses@kheafield.com>
Cc: moses-support <moses-support@mit.edu>
Message-ID: <0E7CA194-04C0-4A05-9D68-36F7EA708FCF@gmail.com>
Content-Type: text/plain; charset="utf-8"

Removed and works now.

Thanks again!

Shuoyang

> On Feb 23, 2017, at 12:52 PM, Kenneth Heafield <moses@kheafield.com> wrote:
>
> Looks like you're compiling a main function into the moses library.
>
> Keep anything containing "int main" in phrase-extract.
>
> On 02/23/2017 04:59 PM, Shuoyang Ding wrote:
>> Hi,
>>
>> I did what Ken described and looks like I?m almost there. The only
>> problem now is that I keep getting this weird message about unit test. I
>> attached the build log in the email.
>>
>> It seems like the build system is trying to test binary extract-ghkm
>> after compilation but there is no way for it to know how to do it.
>>
>> Is there anyway I can remove this unit test for the moment?
>>
>> Shuoyang
>>
>>
>>
>>
>>> On Feb 22, 2017, at 1:35 PM, Kenneth Heafield <moses@kheafield.com
>>> <mailto:moses@kheafield.com <mailto:moses@kheafield.com>>> wrote:
>>>
>>> The main moses target already includes moses/*.cpp (with some exceptions
>>> that you don't care about). So just moving files should work albeit
>>> hackily.
>>>
>>> Kenneth
>>>
>>> On 02/22/2017 05:24 PM, Shuoyang Ding wrote:
>>>> Hi,
>>>>
>>>> Thanks for the timely reply.
>>>>
>>>> To make sure I?m doing this correctly, here are some more questions:
>>>>
>>>> 1) should the moved files in phrase-extract/ be added to moses/Jamfile
>>>> so they?ll be compiled?
>>>> 2) shall I move the Jamfiles in phrase-extract/ and edit the paths in
>>>> them as appropriate?
>>>>
>>>> Shuoyang
>>>>
>>>>> On Feb 22, 2017, at 12:15 PM, Hieu Hoang <hieuhoang@gmail.com <mailto:hieuhoang@gmail.com>
>>>>> <mailto:hieuhoang@gmail.com <mailto:hieuhoang@gmail.com>>
>>>>> <mailto:hieuhoang@gmail.com <mailto:hieuhoang@gmail.com>>> wrote:
>>>>>
>>>>> apologies, ken pointed out why there are multiple definitions. You
>>>>> should be aware of that
>>>>>
>>>>> Hieu Hoang
>>>>> http://moses-smt.org/ <http://moses-smt.org/>
>>>>>
>>>>> On 22 February 2017 at 17:11, Hieu Hoang <hieuhoang@gmail.com <mailto:hieuhoang@gmail.com>
>>>>> <mailto:hieuhoang@gmail.com <mailto:hieuhoang@gmail.com>>> wrote:
>>>>>
>>>>> too complicated imo. Copy what u need into moses/ so you know
>>>>> exactly what the dependencies are.
>>>>>
>>>>> Do what Ken suggested later when it's working
>>>>>
>>>>> Hieu Hoang
>>>>> http://moses-smt.org/ <http://moses-smt.org/>
>>>>>
>>>>> On 22 February 2017 at 17:10, Kenneth Heafield
>>>>> <moses@kheafield.com <mailto:moses@kheafield.com> <mailto:moses@kheafield.com <mailto:moses@kheafield.com>>> wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> phrase-extract depends on moses c.f.
>>>>> phrase-extract/Jamfile:7.
>>>>>
>>>>> alias deps : $(most-deps:B).o ..//z ..//boost_iostreams
>>>>> ..//boost_filesystem ../moses//moses ../moses//ThreadPool
>>>>> ../moses//Util
>>>>> ../util//kenutil ;
>>>>>
>>>>> So rather than copy, move it to moses. More cleanly, you
>>>>> could extract
>>>>> a target for the part you want to share and point both at it.
>>>>>
>>>>> Kenneth
>>>>>
>>>>> On 02/22/2017 04:57 PM, Shuoyang Ding wrote:
>>>>>> Hi All,
>>>>>>
>>>>>> Sorry for bringing up this again after a long time. Last
>>>>> time I worked
>>>>>> around the problem by re-designing my feature function, which
>>>>>> unfortunately does not apply this time.
>>>>>>
>>>>>> To quickly remind you what the problem was, I was developing
>>>>> a feature
>>>>>> function that involves some headers in phrase-extract/, and
>>>>> the headers
>>>>>> seems not correctly referenced.
>>>>>>
>>>>>> As Hieu suggested, I copied code in phrase-extract/ into
>>>>> moses/ and it
>>>>>> caused massive re-definition error so seems like that is not a
>>>>>> straight-forward solution.
>>>>>>
>>>>>> Having read some feature functions, I noticed that a feature
>>>>> function
>>>>>> PhraseOrientation actually reference headers in
>>>>> phrase-extract/, and it
>>>>>> seems to work well. I?m wondering if it is actually easier to do
>>>>>> something similar to that feature function (i.e. by changing
>>>>> some bjam
>>>>>> configurations) and which config files should I be looking at.
>>>>>>
>>>>>> Thanks!
>>>>>>
>>>>>> Regards,
>>>>>> Shuoyang Ding
>>>>>>
>>>>>> Ph.D. Student
>>>>>> Center for Language and Speech Processing
>>>>>> Department of Computer Science
>>>>>> Johns Hopkins University
>>>>>>
>>>>>> Hackerman Hall 225A
>>>>>> 3400 N. Charles St.
>>>>>> Baltimore, MD 21218
>>>>>>
>>>>>> http://cs.jhu.edu/~sding <http://cs.jhu.edu/~sding>
>>>>>>
>>>>>>
>>>>>>> On Apr 29, 2016, at 3:59 AM, Hieu Hoang
>>>>> <hieuhoang@gmail.com <mailto:hieuhoang@gmail.com> <mailto:hieuhoang@gmail.com <mailto:hieuhoang@gmail.com>>
>>>>>>> <mailto:hieuhoang@gmail.com <mailto:hieuhoang@gmail.com> <mailto:hieuhoang@gmail.com <mailto:hieuhoang@gmail.com>>>>
>>>>> wrote:
>>>>>>>
>>>>>>> You should be careful of referencing files in
>>>>> phrase-extract/ if you
>>>>>>> are developing code that is in moses/
>>>>>>>
>>>>>>> The code in phrase-extract/ will not be linked.
>>>>>>>
>>>>>>> It's probably safer to simply copy what you need from
>>>>> phrase-extract/
>>>>>>> into moses/ and use the copy
>>>>>>>
>>>>>>> On 25/04/2016 17:16, Shuoyang Ding wrote:
>>>>>>>> Hi all,
>>>>>>>>
>>>>>>>> I?m developing a feature function that involves some
>>>>> headers in
>>>>>>>> phrase-extract:
>>>>>>>>
>>>>>>>> phrase-extract/extract-ghkm/AlignmentGraph.h
>>>>>>>> phrase-extract/extract-ghkm/Rule.h
>>>>>>>> phrase-extract/extract-ghkm/ScfgRule.h
>>>>>>>> phrase-extract/syntax-common/xml_tree_parser.h
>>>>>>>>
>>>>>>>> Now I?m having some issue with recompiling with this new
>>>>> feature
>>>>>>>> function ? these headers seems not correctly referenced. I
>>>>> attached
>>>>>>>> the build log with this email. Specifically:
>>>>>>>>
>>>>>>>> ./phrase-extract/extract-ghkm/AlignmentGraph.h:28:24:
>>>>> fatal error:
>>>>>>>> SyntaxTree.h: No such file or directory
>>>>>>>>
>>>>>>>> I guess I may have to change moses/Jamfile to accommodate this
>>>>>>>> change. Is there anyone who is familiar with bjam who can
>>>>> help me?
>>>>>>>>
>>>>>>>> Thanks!
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Shuoyang Ding
>>>>>>>>
>>>>>>>> Ph.D. Student
>>>>>>>> Center for Language and Speech Processing
>>>>>>>> Department of Computer Science
>>>>>>>> Johns Hopkins University
>>>>>>>>
>>>>>>>> Hackerman Hall 225A
>>>>>>>> 3400 N. Charles St.
>>>>>>>> Baltimore, MD 21218
>>>>>>>>
>>>>>>>> http://cs.jhu.edu/~sding <http://cs.jhu.edu/%7Esding>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Moses-support mailing list
>>>>>>>> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
>>>>>>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>>> <http://mailman.mit.edu/mailman/listinfo/moses-support>
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Moses-support mailing list
>>>>>> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
>>>>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>>> <http://mailman.mit.edu/mailman/listinfo/moses-support>
>>>>>>
>>>>> _______________________________________________
>>>>> Moses-support mailing list
>>>>> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
>>>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>>> <http://mailman.mit.edu/mailman/listinfo/moses-support>
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Moses-support mailing list
>>>>> Moses-support@mit.edu <mailto:Moses-support@mit.edu>
>>>>> http://mailman.mit.edu/mailman/listinfo/moses-support

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/mailman/private/moses-support/attachments/20170223/ae5f6d7e/attachment.html

------------------------------

_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support


End of Moses-support Digest, Vol 124, Issue 24
**********************************************

Tweet