LinuxQuestions.org - [SOLVED] Script to list dependencies

- Programming (https://www.linuxquestions.org/questions/programming-9/)

- - Script to list dependencies (https://www.linuxquestions.org/questions/programming-9/script-to-list-dependencies-777119/)

MTK358

12-20-2009 07:29 PM

Script to list dependencies

I would like to make a script (in whatever language is most suitable) that extracts the contents of "#include "Something.h"" statements.

Example Input File:

Code:

some stuff

/*more stuff*/

#include "Something.h"

#include <stdio.h>

#include "AnotherHeader.h"

more code

Output:

Code:

Something.h

AnotherHeader.h

It would also be nice if it could recursively find the dependencies of these headers, and so on.

How should I do this (or maybe there already is such a thing)?

JohnGraham

12-20-2009 08:07 PM

find, grep and sed seem like apt tools for this (in that order) - find to get the files, grep to search for #include lines, and sed to edit-out the bits you don't want.

Here's a single command-line that works for a bunch of files I have in one of my projects. It'll probably have bugs, and you'll probably want to wrap it in script-related niceness, but it's a start:

Code:

find | xargs egrep '#include [<"].*[>"]' | sed -e 's|.*#include [<"]||g' -e 's|[>"].*||g' | sort | uniq

ntubski

12-20-2009 08:52 PM

gcc can do this: see Preprocessor Options (-M and -MM). The output is in Makefile format:

Code:

~/tmp$ cat dep.c 

/*more stuff*/

#include "Something.h"

#include <stdio.h>

#include "AnotherHeader.h"

~/tmp$ gcc -MM -MG dep.c 

dep.o: dep.c Something.h AnotherHeader.h

~/tmp$ gcc -MM -MG dep.c | sed 's/^.*dep\.c //'

Something.h AnotherHeader.h

-MG just tells gcc not to complain about Something.h not existing. If Something.h #includes another header it will be in the dependancy list for dep.c.

ta0kira

12-21-2009 12:03 AM

It's hard to track down dependencies with local includes unless you're privy to the include paths to be specified at compile time, in which case you might as well just use automake and see if you can find where it stores dependency information (I've never bothered to look for it, but it's there somewhere.)
Kevin Barry

MTK358

12-21-2009 07:34 AM

I've started playing with JohnGraham's script, even thought it doesn't do what I want I think I can make it do what I want.

The main thing is that the script fount all the dependencies of all the files, I just need the dependencies of one single .c file recursively.

Also, I only want to catch #include's with quotes (not brackets).

Here's what I came up with so far:

Code:

cat lsd

#!/bin/bash



grep '#include ["].*["]' $1

$ lsd Cat.c

#include "Cat.h"

Wanted Output:

Code:

lsd Cat.c

Cat.h

Animal.h

Object.h

All I need to do now is to strip off the #include statement to just leave the filename, and then run this script on all the dependencies that it found.

MTK358

12-21-2009 08:07 AM

Some more progress:

Code:

#!/bin/bash



grep '#include ["].*["]' $1 | sed 's:[^"]*"\([^"]*\)"[^"]*:\1:'

It now lists all the filenames of the dependencies (for some reason it only works when there is a space between #include and the first quotation mark, I don't know why).

Now just the recursive part.

EDIT: I figured out that the grep command is searching for the space, this is how I fixed it:

Code:

grep '#include *["].*["]' $1 | sed 's:[^"]*"$[^"]*$"[^"]*:\1:'

MTK358

12-21-2009 09:42 AM

I really have no idea how to do this.

Is it possible for a for loop in bash to iterate through individual lines of a block text stored in a variable?

MTK358

12-21-2009 12:32 PM

I found that you can use exec to redirect a file to stdin and then read to read stdin line by line, but how do you redirect the contents of a variable to stdin?

tuxdev

12-21-2009 01:23 PM

Code:

#!/usr/bin/env bash



while read -r file ; do

  :

done < <(grep -rh '^#include *".*"$' | sed 's/^#include *"\(.*\)"$/\1/' | sort | uniq)

I do wonder why you want to do this. Any one of the alternative methods given in this thread is likely to work much better for you.

MTK358

12-21-2009 01:42 PM

You still don't understand that I don't want to get all the dependencies of all the files. I just want the dependencies of 1 file, and the dependencies of those, and so on.

Also the example creates syntax errors.

tuxdev

12-21-2009 01:51 PM

That's not really what you want to do. What you want to do is something like "draw a dependency graph", which doxygen can do for you with the help of graphvis.

What is it that you *really* want to do?

MTK358

12-21-2009 01:57 PM

Actually this is supposed to be a part of an automated script that checks which .c files must be compiled.

For this purpose, I want it so that you just apply a .c file as the argument and it lists out all the .h files it includes.

Disillusionist

12-21-2009 02:15 PM

Code:

#!/bin/bash



read -p "Enter file to be checked ==> " l_file

if [ -f "$l_file" ]

then

  grep '#include' "$l_file"|sed 's/#include *["<]//'|sed 's/[">]//'

else

  echo "File: $l_file not found or not a regular file"

fi

MTK358

12-21-2009 02:17 PM

Doesn't that go only 2 levels deep?

Here's my recursive version, but it just freezes at the highlited line and doesn't execute further:

Code:

#!/bin/bash



deps=''

files=$*

filedeps=''



getdeps() {

        $filedeps=$(grep '#include *".*"' $* | sed 's:[^"]*"\([^"]*\)"[^"]*:\1:' | uniq)

        if [ $filedeps ]

        then

                echo $filedeps # Actually this should append $filedefs to $deps, I don't know how to do that

                $files = $filedeps

                getdeps

        fi

}



getdeps

echo $deps



exit 0

Disillusionist

12-21-2009 02:36 PM

Quote:

Originally Posted by MTK358 (Post 3800306)

Doesn't that go only 2 levels deep?

Here's my recursive version, but it just freezes at the highlited line and doesn't execute further:

Code:

#!/bin/bash



deps=''

files=$*

filedeps=''



getdeps() {

        $filedeps=$(grep '#include *".*"' $files | sed 's:[^"]*"\([^"]*\)"[^"]*:\1:' | uniq)

        if [ $filedeps ]

        then

                echo $filedeps # Actually this should append $filedefs to $deps, I don't know how to do that

                $files = $filedeps

                getdeps

        fi

}



getdeps

echo $deps



exit 0

You need to use the variable $files instead of $*

MTK358

12-21-2009 02:50 PM

The line "$files = $filedeps" doesn't work!?!?

Disillusionist

12-21-2009 04:12 PM

Code:

#!/bin/bash

files=$*



getdeps(){

  filedeps=$(grep '#include' $file|sed 's:#include *[<"]::'|sed 's:[>"]::'|uniq)

  echo $filedeps|sed 's/ /\n/g'|while read file

  do

      echo $file

      if [ -f $file ]

      then

        getdeps

      fi

  done

}



echo $files|sed 's/ /\n/g'|while read file

do

  if [ -f $file ]

  then

      echo $file

      getdeps

  fi

done

MTK358

12-21-2009 04:51 PM

Quote:

Originally Posted by Disillusionist (Post 3800426)

Code:

#!/bin/bash

files=$*



getdeps(){

  filedeps=$(grep '#include' $file|sed 's:#include *[<"]::'|sed 's:[>"]::'|uniq)

  echo $filedeps|sed 's/ /\n/g'|while read file

  do

      echo $file

      if [ -f $file ]

      then

        getdeps

      fi

  done

}



echo $files|sed 's/ /\n/g'|while read file

do

  if [ -f $file ]

  then

      echo $file

      getdeps

  fi

done

I really wish that there would be some better bash tutorial that actually explains how it works, because it is so not like normal languages, as it seems to be made of programs stuck together instead of predefined constructs.

for example I am used to "normal" programming languages and this completely freaks me out:

Code:

echo $files|sed 's/ /\n/g'|while read file

do

And, of course, the way you cannot just set the value of a variable.

The problem with most bash tutorials is that they treat it as a normal programming language, not even saying that things like the fact that "while" is actually independent command-line program that takes input and output, not a built-in construct.

tuxdev

12-21-2009 05:10 PM

uh, while is a built-in construct. The problem is that piping creates subshells, and stuff that happens inside a subshell doesn't effect the parent.

You're right that pretty much all bash resources are misleading, especially ABS..
http://mywiki.wooledge.org/BashGuide
..much better

Disillusion's script is more POSIX shell than Bash, but even there it's got problems, particularly with handling filenames with spaces.

Code:

#!/usr/bin/env bash



getdeps() {

  while read -r file ; do

      echo "$file"

      getdeps "$file"

  done < <(grep -h '^#include *".*"$' "$1" | sed 's/^#include *"\(.*\)"$/\1/')

}



getdeps "$1"

ghostdog74

12-21-2009 06:39 PM

Quote:

Originally Posted by MTK358 (Post 3799369)

It would also be nice if it could recursively find the dependencies of these headers, and so on.

what do you mean by "dependencies"? can you show examples?

MTK358

12-21-2009 07:12 PM

Quote:

Originally Posted by tuxdev (Post 3800477)

You're right that pretty much all bash resources are misleading, especially ABS..
http://mywiki.wooledge.org/BashGuide
..much better
[/code]

Wow, that tutorial is so, so much better than the "Advanced Bash Scripting Guide" advertised everywhere. It actually makes some sense!

Quote:

Originally Posted by ghostdog74 (Post 3800540)

what do you mean by "dependencies"? can you show examples?

I mean .h files that the .c file includes.

ghostdog74

12-21-2009 07:33 PM

Code:

awk 'BEGIN{s="find /usr/include -type f \\( "}

/include/{

    gsub(/\042|[<>]/,"",$2)

    s=s"-iname "$2" -o "

}

END{

    gsub(/-o $/,"\\)",s)

    while( ( s | getline found) > 0 ) {

        print "found: "found

    }

}' file.c

MTK358

12-21-2009 07:36 PM

Here's my current version, but the highlighted part doesn't seem to take effect:

Code:

#!/usr/bin/env bash



deps=''



getdeps() {

        while read -r file ; do

                deps=$(echo -e "$deps""$file"'\n')

                getdeps "$file"

        done < <(grep '#include *".*"' "$1" | sed 's:[^"]*"\([^"]*\)"[^"]*:\1:')

}



getdeps "$1"

echo -n "$(echo "$deps" | uniq | sort)"

ta0kira

12-21-2009 07:38 PM

Quote:

Originally Posted by MTK358 (Post 3800284)

This is half the reason I use automake and autoconf. If you insist on rewriting them yourself, you might as well look at the source code.
Kevin Barry

tuxdev

12-21-2009 08:33 PM

Use arrays.

Code:

#!/usr/bin/env bash



deps=()



getdeps() {

        while read -r file ; do

                deps+=("$file")

                getdeps "$file"

        done < <(grep '#include *".*"' "$1" | sed 's:[^"]*"\([^"]*\)"[^"]*:\1:')

}



getdeps "$1"

printf "%s\n" "${deps[@]}" | sort | uniq

I do share ta0kira's concerns over why you are trying to resolve this problem (I prefer use CMake over autotools, though). I also sometimes use ntubski's suggestion to use gcc's -MD for simpler projects. After all, make's job is to resolve dependencies, but you have to actually tell make what they are.

MTK358

12-22-2009 08:23 AM

It works now.

All times are GMT -5. The time now is 04:36 PM.