LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 08-04-2009, 04:29 PM   #1
Matthew Applegate
LQ Newbie
 
Registered: Aug 2009
Posts: 5

Rep: Reputation: 0
GNU Make - Variables as Targets


Hi all,

My never ending battle with GNU make continues. I am having another problem (infuriatingly a seemingly very simple problem) using GNU make which I am hoping is due to my ignorance, and hence not a show stopper.

Let me start by introducing the problematic rules in my makefile, as well as anything else that is necessary to understand my particular case.

Code:
.PHONY: all ${librarylist_variable}

all: ${librarylist_variable}

librarylist_variable = $(shell if [ -f librarylist ]; then cat librarylist; fi)

${librarylist_variable}: librarylist
    # Make all the libraries which are dependencies
	@${MAKE} --no-print-directory -C ${@} ${MAKECMDGOALS}

librarylist: file1
	@create_librarylist.sh
Basically my problem is as follows, if I change file1 and then make all, librarylist will be re-made, which causes the elements of ${librarylist_variable} to be made. This is all as expected. ${librarylist_variable} is of course just be the contents of the librarylist file, but, GNU make does not use the most up-to-date contents, having just re-made librarylist.

i.e. If librarylist contains the following:

A
B

If I then make all, having changed file1, and imagine that when librarylist is re-made by make it now contains:

A
B
C

make will continue as if librarylist contained A and B only!

I want make to use the most up-to-date contents of the variable librarylist_variable during operation of make, is this possible? How do I achieve this?

I do hope my description make sense, please can someone help me?

-Matt

Last edited by Matthew Applegate; 08-07-2009 at 12:52 AM.
 
Old 08-04-2009, 05:50 PM   #2
tuxdev
Senior Member
 
Registered: Jul 2005
Distribution: Slackware
Posts: 2,012

Rep: Reputation: 111Reputation: 111
It should be possible to use the 'include' directive and an implicit rule to do what you want. Also read "Recursive Make Considered Harmful" (it might be okay in this case, though).
 
Old 08-05-2009, 01:07 AM   #3
Matthew Applegate
LQ Newbie
 
Registered: Aug 2009
Posts: 5

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by tuxdev View Post
It should be possible to use the 'include' directive and an implicit rule to do what you want. Also read "Recursive Make Considered Harmful" (it might be okay in this case, though).
Thanks for your response Tuxdev. As it happens I am already using the include directive in the makefile. The setup that I have is that I have just one master makefile to build all libraries. A dummy makefile is placed in each library, which merely includes the master makefile. I am not sure how I can use include to solve my immediate problem, did you have anything in mind?

I suppose the only way I could avoid recursive calls to make would be to write a script which generated the master makefile which would contain all the rules to make all the libraries. Writing the script might not be such a problem, however, I do not want to have to remember to run this script before I make, and moreover, running the script would likely take a long time. The makefile I have currently is simple to understand, and easy to maintain, a makefile generated from a script becomes something else. So that said, unless you can twist my arm with a very good reason, I would like to stay with recursion.

So the question remains, how do I update the contents the variable librarylist_variable after librarylist has been re-made? Or alternatively, how do I achieve the behaviour I want?

Thanks for the comments, and please keep 'em coming,

-Matt
 
Old 08-05-2009, 09:16 AM   #4
tuxdev
Senior Member
 
Registered: Jul 2005
Distribution: Slackware
Posts: 2,012

Rep: Reputation: 111Reputation: 111
Could you post all of the actual Makefiles you've got? I can't really see what's really going on with just hypothetical (also please use [code] tags).

This is the Makefile I use for my smaller projects:
Code:
CXXFLAGS = -O3 -g -Wall -Wextra -ansi -pedantic -MD
LINK.o = $(LINK.cpp)
.PHONY = all clean

ifeq ($(GPROF),yes)
CXXFLAGS += -pg
endif

ifeq ($(GCOV),yes)
CXXFLAGS += -O0 -DNDEBUG -fno-elide-constructors -fno-inline -fprofile-arcs -ftest-coverage
endif

SRCS = $(wildcard src/*.cpp)

all: foo

foo: $(SRCS:%.cpp=%.o)

clean:
	$(RM) src/foo src/core src/gmon.out src/*.o src/*.d src/*.gcno src/*.gcov src/*.gcda

-include src/*.d
The -MD flag to GCC generates dependency information, which is then included (I was thinking about something like this technique in my last post). You can also turn on or off profiling and coverage analysis (valgrind is better a profiling, though).

Once a project gets sufficiently complicated that writing a Makefile manually becomes a major headache, I switch to CMake. It generates Makefiles, but does it a lot better than writing a script from scratch. The issue you mentioned with a shell script doesn't really happen in practice with CMake. Oh, and that it can also generate Visual Studio solutions is *really* nice for cross-platform development.

Last edited by tuxdev; 08-05-2009 at 09:37 AM.
 
Old 08-06-2009, 02:59 AM   #5
Matthew Applegate
LQ Newbie
 
Registered: Aug 2009
Posts: 5

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by tuxdev View Post
The -MD flag to GCC generates dependency information, which is then included (I was thinking about something like this technique in my last post). You can also turn on or off profiling and coverage analysis (valgrind is better a profiling, though).

Once a project gets sufficiently complicated that writing a Makefile manually becomes a major headache, I switch to CMake. It generates Makefiles, but does it a lot better than writing a script from scratch. The issue you mentioned with a shell script doesn't really happen in practice with CMake. Oh, and that it can also generate Visual Studio solutions is *really* nice for cross-platform development.
Hi Tuxdev, I am slightly hesitant to upload the entire makefile, for fear of moving the topic away from my immediate problem, but here goes anyway. I suppose there is nothing proprietary in a makefile.

As you will see I am not compiling C, but Verilog. I already have work arounds to my initial problem, however I am not happy with them. The first work around involves always making the librarylist. However this is not what I want to do, for performance reasons. My second work around involved building each element of the variable librarylist_variable using a loop. This seemed to work. However, the loop made things very slow, and I could not use make -j using this approach. As it happens, I have left things as is. I keep finding anomalies with GNU make like this and it really chews up my time, and most often, I end up back where I started. I shall refrain from a broadside on GNU make for now. Sigh.

If you know of any other way to solve my problem, please let me know.

-Matt
Code:
################################################################################
# Variables
################################################################################
SHELL         := /usr/bin/bash
.DEFAULT_GOAL := all

get_design_library_base_name_output = $(shell get_design_library_base_name.sh 2>&1)
# Check if the script errored
ifeq ($(filter ERROR:, ${get_design_library_base_name_output}), )
    design_library_base_name := ${get_design_library_base_name_output}
else
    $(error $(shell get_design_library_base_name.sh > /dev/null))
endif

filelist_variable    = $(shell if [ -f filelist ]; then cat filelist; fi)
librarylist_variable = $(shell if [ -f librarylist ]; then cat librarylist; fi)

################################################################################
# Targets
################################################################################
.PHONY: fresh all ${librarylist_variable} banner clean debug

fresh: clean all

all: ${librarylist_variable} banner work 

work: librarylist filelist ${filelist_variable}
    # If work does not exist, create and map it
	@if [ ! -d work ]; then echo "[${design_library_base_name}_lib] Creating design library work..."; vlib work; fi
	@map_design_library.sh
    # Compile the design files
	@echo "[${design_library_base_name}_lib] Compiling file(s) ${filelist_variable}, $? has/have changed..."
	@vlog -timescale "1 ps / 1 ps" \
		-lint \
		${filelist_variable}

${librarylist_variable}: librarylist
    # Make all the libraries which are dependencies
	@${MAKE} --no-print-directory -C `dirname $(shell get_design_library_mapping.sh ${@})` ${MAKECMDGOALS}

librarylist: filelist ${filelist_variable}
	@echo "[${design_library_base_name}_lib] Creating librarylist, $? has/have changed..."
	@create_librarylist.sh
    # If the librarylist already exists and if re-made would be different, this make won't work!

filelist:
	@echo "[${design_library_base_name}_lib] Creating filelist..."
	@ls -1 *.v > filelist

banner :
    # Print a banner
	@echo ""
	@echo "================================================================================"
	@echo "| ${design_library_base_name}_lib"
	@echo "================================================================================"

clean: ${librarylist_variable} banner
	@echo "[${design_library_base_name}_lib] Deleting design library work..."
	@rm -rf ./work

debug: ${librarylist_variable} banner
	@echo "SHELL                    = ${SHELL}"
	@echo ".DEFAULT_GOAL            = ${.DEFAULT_GOAL}"
	@echo MAKECMDGOALS              = ${MAKECMDGOALS}
	@echo "design_library_base_name = ${design_library_base_name}"
	@echo "filelist_variable        = ${filelist_variable}"
	@echo "librarylist_variable     = ${librarylist_variable}"

Last edited by Matthew Applegate; 08-07-2009 at 12:51 AM. Reason: Adding code tags
 
Old 08-06-2009, 09:02 AM   #6
tuxdev
Senior Member
 
Registered: Jul 2005
Distribution: Slackware
Posts: 2,012

Rep: Reputation: 111Reputation: 111
I think I also need to see what is in those shell scripts as well.

It looks like Make is having a lot of trouble because it doesn't really have a decent clue of the real dependencies (probably because a lot of that work is in those shell scripts). For example, the "filelist" target really depends on every .v file, but make doesn't know that, so it won't redo that target properly.

Rather than forcing make to do things in a particular way, try to instead think of giving make enough information to deduce as much as possible. The Makefile I posted shows this ideal rather well in that I have exactly one explicit command, the clean target.

Please, please, use [code] tags

Last edited by tuxdev; 08-06-2009 at 09:09 AM.
 
Old 08-08-2009, 01:29 AM   #7
Matthew Applegate
LQ Newbie
 
Registered: Aug 2009
Posts: 5

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by tuxdev View Post
I think I also need to see what is in those shell scripts as well.
The shell scripts do what they say on the tin:

get_design_library_base_name.sh- Gets the name of the current library being made. This is just a function of the absolute path.

map_design_library.sh - Appends a line to a file which specifies where all libraries are. This is needed for some tools I am using.

create_librarylist.sh - Creates the librarylist file. Parses all verilog files in the current library which also exist in the filelist and deduces which libraries they need (looks for #include effectively).

get_design_library_mapping.sh - Opens the said file which map_design_library.sh writes to, and retrieves the path to a given library.

I still think that the intial post that I made best describes the problem I am having, having elaborated more on my makefile in general I am fearful that we are getting off track. I really appreciate your help Tuxdev, please keep the comments coming,

-Matt

Last edited by Matthew Applegate; 08-08-2009 at 01:48 AM.
 
Old 08-08-2009, 10:22 AM   #8
tuxdev
Senior Member
 
Registered: Jul 2005
Distribution: Slackware
Posts: 2,012

Rep: Reputation: 111Reputation: 111
I'd really like to see the actual code in the shell scripts. The description of the first script is exactly what "pwd" does, and the rest of the other scripts should really be embedded into the Makefile as well.
 
Old 08-08-2009, 08:26 PM   #9
ntubski
Senior Member
 
Registered: Nov 2005
Distribution: Debian
Posts: 2,466

Rep: Reputation: 845Reputation: 845Reputation: 845Reputation: 845Reputation: 845Reputation: 845Reputation: 845
Quote:
Originally Posted by Matthew Applegate View Post
I still think that the intial post that I made best describes the problem I am having
I think tuxdev's initial suggestion (perhaps too vague) to use include makes sense. Instead of
Code:
librarylist_variable = $(shell if [ -f librarylist ]; then cat librarylist; fi)
You should have
Code:
include librarylist
The file librarylist should read
Code:
librarylist_variable = A B
If the librarylist file gets remade, make will restart as described in 3.7 How Makefiles Are Remade, so it will use the updated value of librarylist_variable.
 
Old 08-08-2009, 11:31 PM   #10
ta0kira
Senior Member
 
Registered: Sep 2004
Distribution: FreeBSD 9.1, Kubuntu 12.10
Posts: 3,078

Rep: Reputation: Disabled
It's hard to tell if this will be of help or not. You might be able to adapt either this or your problem so that it can be of some help:
http://developer.berlios.de/snippet/...ppet&id=100038

Kevin Barry
 
Old 08-09-2009, 04:38 AM   #11
Matthew Applegate
LQ Newbie
 
Registered: Aug 2009
Posts: 5

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by ntubski View Post
I think tuxdev's initial suggestion (perhaps too vague) to use include makes sense. Instead of
Code:
librarylist_variable = $(shell if [ -f librarylist ]; then cat librarylist; fi)
You should have
Code:
include librarylist
The file librarylist should read
Code:
librarylist_variable = A B
If the librarylist file gets remade, make will restart as described in 3.7 How Makefiles Are Remade, so it will use the updated value of librarylist_variable.
That's a good suggestion. Having just tried implementing this I am stuck with a load of warnings from make if filelist or librarylist do not exist. It is only superficial, but I would prefer not to have them.

I tried using an ifeq statement to get around this but that resulted in me having to make twice, one to make the filelist for example, the other time to compile the design files.

Any ideas?

Last edited by Matthew Applegate; 08-09-2009 at 06:07 AM.
 
Old 08-09-2009, 08:28 AM   #12
dwhitney67
Senior Member
 
Registered: Jun 2006
Location: Maryland
Distribution: Kubuntu, Fedora, RHEL
Posts: 1,522

Rep: Reputation: 332Reputation: 332Reputation: 332Reputation: 332
Put a minus-sign character in front of the include statement:
Code:
-include librarylist
 
  


Reply

Tags
gnu, make, variable


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
gnu make .. 91change Linux - Newbie 1 09-15-2008 02:31 AM
GNU Make - get list of targets to be made jrfk2 Linux - General 1 08-12-2006 10:03 PM
tinkering with dkms: make error no targets. Makefile issue? Emmanuel_uk Linux - Newbie 1 06-03-2005 01:26 AM
Adding more targets to Make oulevon Programming 6 02-09-2005 05:11 PM
make with multiple targets eskimo22 Programming 1 02-26-2004 11:42 AM


All times are GMT -5. The time now is 09:55 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration