LinuxQuestions.org
Latest LQ Deal: Linux Power User Bundle
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 11-27-2010, 10:57 AM   #1
Andrew Benton
Senior Member
 
Registered: Aug 2003
Location: Birkenhead/Britain
Distribution: Linux From Scratch
Posts: 2,073

Rep: Reputation: 64
Gnu make parallel problem


Hello World!
I use an application called redbutton-browser to access some of the things available on the redbutton digital tv channels. It compiles fine if I use a simple make but fails if I try to run parallel jobs with make -j4. I'd like some help altering the Makefile so that it does a few commands sequentially before it does the rest of the Makefile in parallel.

The relevant bit of the Makefile looks like this:
Code:
ISO13522-MHEG-5.c:	xsd2c.c ISO13522-MHEG-5.xsd add_instance_vars.conf add_rtti.conf
	make xsd2c
	./xsd2c ISO13522-MHEG-5.xsd
	./add_instance_vars ISO13522-MHEG-5.c ISO13522-MHEG-5.h
	./add_rtti ISO13522-MHEG-5.c > rtti.h

xsd2c:	xsd2c.c
	${CC} ${CFLAGS} ${DEFS} -o xsd2c xsd2c.c -lexpat
If I run it with make -j1 it compiles a small binary called xsd2c which it then uses to create ISO13522-MHEG-5.c and ISO13522-MHEG-5.h from the file ISO13522-MHEG-5.xsd and then it manipulates the ISO13522-MHEG-5.c ISO13522-MHEG-5.h some more with a couple of scripts. If I try to run make -j4 it falls over itself because it tries to run xsd2c before it has been compiled. How do I make it execute just these commands one at a time?
 
Old 11-27-2010, 11:57 AM   #2
ntubski
Senior Member
 
Registered: Nov 2005
Distribution: Debian, Arch
Posts: 3,390

Rep: Reputation: 1554Reputation: 1554Reputation: 1554Reputation: 1554Reputation: 1554Reputation: 1554Reputation: 1554Reputation: 1554Reputation: 1554Reputation: 1554Reputation: 1554
Shouldn't ISO13522-MHEG-5.c depend on xsd2c?
Code:
ISO13522-MHEG-5.c: xsd2c ISO13522-MHEG-5.xsd add_instance_vars.conf add_rtti.conf
	# no need to explicitly call make here
	./xsd2c ISO13522-MHEG-5.xsd
	./add_instance_vars ISO13522-MHEG-5.c ISO13522-MHEG-5.h
	./add_rtti ISO13522-MHEG-5.c > rtti.h

xsd2c:	xsd2c.c
	${CC} ${CFLAGS} ${DEFS} -o xsd2c xsd2c.c -lexpat
 
Old 11-27-2010, 12:52 PM   #3
Andrew Benton
Senior Member
 
Registered: Aug 2003
Location: Birkenhead/Britain
Distribution: Linux From Scratch
Posts: 2,073

Original Poster
Rep: Reputation: 64
That's true, the changes you suggest make sense but it's a bit tangential to the topic I was asking for help with. make -j4 still fails as before.
 
Old 11-27-2010, 12:59 PM   #4
ntubski
Senior Member
 
Registered: Nov 2005
Distribution: Debian, Arch
Posts: 3,390

Rep: Reputation: 1554Reputation: 1554Reputation: 1554Reputation: 1554Reputation: 1554Reputation: 1554Reputation: 1554Reputation: 1554Reputation: 1554Reputation: 1554Reputation: 1554
Quote:
Originally Posted by Andrew Benton
If I try to run make -j4 it falls over itself because it tries to run xsd2c before it has been compiled
...
That's true, the changes you suggest make sense but it's a bit tangential to the topic I was asking for help with. make -j4 still fails as before.
Well, I though those changes would fix the problem actually. In fact, if those changes don't fix it I can't see what would.
 
Old 11-28-2010, 07:06 AM   #5
Andrew Benton
Senior Member
 
Registered: Aug 2003
Location: Birkenhead/Britain
Distribution: Linux From Scratch
Posts: 2,073

Original Poster
Rep: Reputation: 64
My apologies, you were correct, make -j4 ISO13522-MHEG-5.c works every time no, the makefile is falling over itself elsewhere now.
Code:
OBJS=	rb-browser.o		\
	MHEGEngine.o		\
	MHEGDisplay.o		\
	MHEGCanvas.o		\
	MHEGBackend.o		\
	MHEGApp.o		\
	MHEGColour.o		\
	MHEGFont.o		\
	MHEGTimer.o		\
	MHEGStreamPlayer.o	\
	MHEGVideoOutput.o	\
	videoout_null.o		\
	videoout_xshm.o		\
	MHEGAudioOutput.o	\
	${CLASSES}		\
	ISO13522-MHEG-5.o	\
	der_decode.o		\
	clone.o			\
	si.o			\
	readpng.o		\
	mpegts.o		\
	utils.o

default: rb-browser rb-keymap

rb-browser:	ISO13522-MHEG-5.c clone.c ${OBJS}
	${CC} ${CFLAGS} ${DEFS} ${INCS} -o rb-browser ${OBJS} ${LIBS}
It starts trying to compile rb-browser which depends on ISO13522-MHEG-5.c clone.c ${OBJS}, so it does them in parallel, but many of the ${OBJS} need ISO13522-MHEG-5.h (which is created at the same time as ISO13522-MHEG-5.c) and so it falls over because it's not ready yet. How do I make ${OBJS} depend on ISO13522-MHEG-5.c?

The whole Makefile is here

Last edited by Andrew Benton; 11-28-2010 at 07:12 AM. Reason: Added link to the original Makefile
 
Old 11-28-2010, 08:29 AM   #6
ntubski
Senior Member
 
Registered: Nov 2005
Distribution: Debian, Arch
Posts: 3,390

Rep: Reputation: 1554Reputation: 1554Reputation: 1554Reputation: 1554Reputation: 1554Reputation: 1554Reputation: 1554Reputation: 1554Reputation: 1554Reputation: 1554Reputation: 1554
Ah, I see that's a pretty broken makefile.

Code:
# Possibly not all ${OBJS} need this header
${OBJS} : ISO13522-MHEG-5.h

...

# rb-browser only needs object files, not sources
rb-browser: ${OBJS} # no .c 
	${CC} ${CFLAGS} ${DEFS} ${INCS} -o rb-browser ${OBJS} ${LIBS}

...

# "%" rule is needed because this is the only way that make can
# understand a rule that creates more than one file. See
# http://www.cmcrossroads.com/ask-mr-make/12908-rules-with-multiple-outputs-in-gnu-make
%.h %.c: %.xsd
	./xsd2c $*.xsd
	./add_instance_vars $*.c $*.h
	./add_rtti $*.c > rtti.h

ISO13522-MHEG-5.h ISO13522-MHEG-5.c: xsd2c ISO13522-MHEG-5.xsd add_instance_vars.conf add_rtti.conf

...
dertest-mheg.c:	xsd2c ISO13522-MHEG-5.xsd
	# don't call make here
	./xsd2c -c dertest-mheg.c -h dertest-mheg.h ISO13522-MHEG-5.xsd
...
tar: clean
	# don't call make here
	(cd ..; tar zcvf ${TARDIR}.tar.gz --exclude .svn ${TARDIR})

# these aren't real files, tell make not to look for them
.PHONY: clean install tar
 
Old 11-28-2010, 10:17 AM   #7
Andrew Benton
Senior Member
 
Registered: Aug 2003
Location: Birkenhead/Britain
Distribution: Linux From Scratch
Posts: 2,073

Original Poster
Rep: Reputation: 64
Many thanks for your help. Strangely it still falls over if I try make -j64 but it works fine at make -j16 so I think that's good enough. Thanks again for your help
 
Old 11-28-2010, 10:56 AM   #8
ntubski
Senior Member
 
Registered: Nov 2005
Distribution: Debian, Arch
Posts: 3,390

Rep: Reputation: 1554Reputation: 1554Reputation: 1554Reputation: 1554Reputation: 1554Reputation: 1554Reputation: 1554Reputation: 1554Reputation: 1554Reputation: 1554Reputation: 1554
Hmm, maybe it fails at -j64 because I forgot about rtti.h

Code:
${OBJS} : ISO13522-MHEG-5.h rtti.h

rttii.h %.h %.c: %.xsd
	./xsd2c $*.xsd
	./add_instance_vars $*.c $*.h
	./add_rtti $*.c > rtti.h

rtti.h ISO13522-MHEG-5.h ISO13522-MHEG-5.c: xsd2c ISO13522-MHEG-5.xsd add_instance_vars.conf add_rtti.conf
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
GNU Make and GNU GCC DEBUG vs RELEASE compiler options aryan1 Programming 1 01-12-2010 01:29 PM
gnu make separator problem with commands only Jayse66 Linux - Software 4 09-12-2009 12:31 PM
directory access problem when compile .c into .o using GNU make George2 Programming 2 08-31-2006 11:26 AM
Problem running ifdef in GNU make 3.80 jennifer29 Linux - Newbie 1 03-03-2006 05:56 AM
How to make a parallel repository (e.g. with CVS) esben Linux - Software 0 01-18-2004 12:33 PM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 12:22 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration